“Looks like the Toronto group have been working on something very similar as well: http://lanl.arxiv.org/abs/1411.2539 Has anybody been able to find the Google paper? The article says it's on arxiv, but I can't seem to find it there. All that seems to be published so far is this blog post: http://googleresearch.blogspot.be/2014/11/a-picture-is-worth-thous…”
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Inspired by recent advances in multimodal learning and machine translation, we introduce an encoder-decoder pipeline that learns (a): a multimodal joint embedding space with images and text and (b): a novel language model for decoding distributed representations from our space. Our pipeline effectively unifies joint image-text embedding models with multimodal neural language models. We introduce the structure-content neural language model that disentangles the structure of a sentence to its content, conditioned on representations produced by the encoder. The encoder allows one to rank images and sentences while the decoder can generate novel descriptions from scratch. Using LSTM to encode sentences, we match the state-of-the-art performance on Flickr8K and Flickr30K without using object detections. We also set new best results when using the 19-layer Oxford convolutional network. Furthermore we show that with linear encoders, the learned embedding space captures multimodal regularities in terms of vector space arithmetic e.g. *image of a blue car* - "blue" + "red" is near images of red cars. Sample captions generated for 800 images are made available for comparison.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Hi all, I'm the author on the Stanford article, so I can answer any more detailed/technical questions. The article doesn't mention this, but closely related work I've only very recently become aware of (if anyone is interested) are additionally: From UofT: http://arxiv.org/pdf/1411.2539v1.pdf From Baidu/UCLA: http://arxiv.org/pdf/1410.1090v1.pdf From G…”
“There are a number of papers tackling a similar task [0][1][2] for anyone who is interested. There isn't enough information to tell exactly what is going on with Deepgram, but one way to approach this would be to construct a shared embedding space for words/phrases and speech. These types of embedding spaces are powerful [3][4][5], but not magic. Cool de…”