“The actual paper has a section on error analysis that is particularly enlightening: https://arxiv.org/abs/1610.05256 On the CallHome dataset humans confuse words 4.1% of the time, but delete 6.5% of words, most commonly deleting the word "I". Their ASR system confuses 6.5% of words on this dataset, but only deletes 3.3% of words, so depending on how you…”
Achieving Human Parity in Conversational Speech Recognition
Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human error rate on the widely used NIST 2000 test set, and find that our latest automated system has reached human parity. The error rate of professional transcribers is 5.9% for the Switchboard portion of the data, in which newly acquainted pairs of people discuss an assigned topic, and 11.3% for the CallHome portion where friends and family members have open-ended conversations. In both cases, our automated system establishes a new state of the art, and edges past the human benchmark, achieving error rates of 5.8% and 11.0%, respectively. The key to our system's performance is the use of various convolutional and LSTM acoustic model architectures, combined with a novel spatial smoothing method and lattice-free MMI acoustic training, multiple recurrent neural network language modeling approaches, and a systematic use of system combination.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Section 9 in the paper[1] is all about comparing these mistakes between the system and humans. The most common mistakes for humans and the system are in tables 9—11. We find that the artificial errors are substantially the same as human ones with one large exception confusions between backchannel words [acknowledgment words like “uh-huh”] and hesitations…”
“This (https://arxiv.org/abs/2010.03449) is by far not the first publication which claims super human speech recognition, or human parity. For example https://arxiv.org/abs/1610.05256 or https://arxiv.org/abs/1703.02136. And since then, the WER has dropped much more on the same dataset (Switchboard), for example https://arxiv.org/abs/2001.07263. They high…”
- Achieving Human Parity in Conversational Speech Recognition ↗sagivo · 2016-10-18 · 2 分