“Here’s a direct link to the paper: https://arxiv.org/pdf/1904.01624.pdf There was another paper two weeks ago: https://arxiv.org/pdf/1901.02348.pdf For some reason, this submission isn’t making it to the front page, even with 7 upvotes in the first 10 minutes.”
Lessons from Building Acoustic Models with a Million Hours of Speech
This is a report of our lessons learned building acoustic models from 1 Million hours of unlabeled speech, while labeled speech is restricted to 7,000 hours. We employ student/teacher training on unlabeled data, helping scale out target generation in comparison to confidence model based methods, which require a decoder and a confidence model. To optimize storage and to parallelize target generation, we store high valued logits from the teacher model. Introducing the notion of scheduled learning, we interleave learning on unlabeled and labeled data. To scale distributed training across a large number of GPUs, we use BMUF with 64 GPUs, while performing sequence training only on labeled data with gradient threshold compression SGD using 16 GPUs. Our experiments show that extremely large amounts of data are indeed useful; with little hyper-parameter tuning, we obtain relative WER improvements in the 10 to 20% range, with higher gains in noisier conditions.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“> M-AILABS data contains 9 language varieties with a modified BSD 3-Clause License, however there is no community-driven aspect. There is certainly an aspect, as M-AILABS sources their data mainly from the librivox project, which is community-driven. To give ballpark numbers on what it would have cost if you had had to pay people for providing the data …”