按年被提到
20192019 · 3 次2026
视觉、语音与图像生成›语音与音频生成·arXiv:1905.09773 · cs.CV

Speech2Face: Learning the Face Behind a Voice

Tae-Hyun Oh, Tali Dekel, Changil Kim, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Wojciech Matusik · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 4 人首次 2019-05-26 · 最近 2022-12-19
4不同的人
2活跃月份
4讨论串
167被引(OpenAlex)
arXiv 摘要(候选层不写导读)

How much can we infer about a person's looks from the way they speak? In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking. We design and train a deep neural network to perform this task using millions of natural Internet/YouTube videos of people speaking. During training, our model learns voice-face correlations that allow it to produce images that capture various physical attributes of the speakers such as age, gender and ethnicity. This is done in a self-supervised manner, by utilizing the natural co-occurrence of faces and speech in Internet videos, without the need to model attributes explicitly. We evaluate and numerically quantify how--and in what manner--our Speech2Face reconstructions, obtained directly from audio, resemble the true face images of the speakers.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论daenz · 2019-05

“One of the really cool things from the pdf is the ability to construct a normalized face from a video frame: https://i.imgur.com/SdRHCJ0.png Basically they are taking a video frame, making it face forward, cancelling out lighting for pure albedo, and removing accessories like glasses. This is incredible in terms of tracking individuals. It's a digital mug…”

Speech2Face: Learning the Face Behind a Voice原评论 ↗
讨论串评论echopom · 2019-05

“Absolutely terrifying. If you add this to the model that guesses identity based on the sound produced by inputs (keyboard, mouse...) you basically end up with an "ambiant sound fingerprinting" tech, where the sounds emitted nearby a device can be used to accurately determine the individual that's standing close to it... If you add this to china's faci…”

Speech2Face: Learning the Face Behind a Voice原评论 ↗
讨论串评论wanderfowl · 2019-05

“I'm a speech scientist. This paper is a neat idea, and the results are interesting, but not in the way I'd expected. I had hoped it would the domain of how much person-specific information this can deduce from a voice, e.g. lip aperture, overbite, size of the vocal tract, openness of the nares. This is interesting from a speech perception standpoint. Inst…”

Speech2Face: Learning the Face Behind a Voice原评论 ↗
被提交为 story · 4 次