按年被提到
20252025 · 2 次2026
大模型与语言模型›量化、解码与分词·arXiv:2502.03382 · cs.CL

High-Fidelity Simultaneous Speech-To-Speech Translation

Tom Labiausse, Laurent Mazaré, Edouard Grave, Patrick Pérez, Alexandre Défossez, Neil Zeghidour · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2025-02-06 · 最近 2025-07-03
2不同的人
2活跃月份
2讨论串
9被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We introduce Hibiki, a decoder-only model for simultaneous speech translation. Hibiki leverages a multistream language model to synchronously process source and target speech, and jointly produces text and audio tokens to perform speech-to-text and speech-to-speech translation. We furthermore address the fundamental challenge of simultaneous interpretation, which unlike its consecutive counterpart, where one waits for the end of the source utterance to start translating, adapts its flow to accumulate just enough context to produce a correct translation in real-time, chunk by chunk. To do so, we introduce a weakly-supervised method that leverages the perplexity of an off-the-shelf text translation system to identify optimal delays on a per-word basis and create aligned synthetic data. After supervised training, Hibiki performs adaptive, simultaneous speech translation with vanilla temperature sampling. On a French-English simultaneous speech translation task, Hibiki demonstrates state-of-the-art performance in translation quality, speaker fidelity and naturalness. Moreover, the simplicity of its inference process makes it compatible with batched translation and even real-time on-device deployment. We provide examples as well as models and inference code.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论iambateman · 2025-07

“This is why I wonder about the value of language learning for reasons other than “I’m really passionate about it.” We are so close to interfaces that reduce the language barrier by a lot…”

High-fidelity simultaneous speech-to-speech translation原评论 ↗
讨论串评论Grosvenor · 2025-07

“This is so cool. The future is cool! I wonder how it will work on languages that have different grammatical structure than french/english? Like Finno-Ugric languages which have sort of a Yoda speech to them. Edit: In Finno-Ugric languages words later on in a sentence can completely change the meaning. Will be interesting to look at. It's considerate of …”

High-fidelity simultaneous speech-to-speech translation原评论 ↗
讨论串评论lukax · 2025-07

“Soniox also supports real-time speech-to-text translation with 60 languages. You can hook that to a TTS and you have Speech-to-Speech translation. That failed Google I/O real-time translation demo? With Soniox it just works. You can try it out here (select translation instead of transcription) https://soniox.com/ Disclaimer: I work at Soniox.”

High-fidelity simultaneous speech-to-speech translation原评论 ↗
被提交为 story · 2 次