按年被提到
20242024 · 2 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:2410.23262 · cs.CV

EMMA: End-to-End Multimodal Model for Autonomous Driving

Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, Yin Zhou, James Guo 等 14 人 · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2024-11-25 · 最近 2025-02-18
3不同的人
2活跃月份
3讨论串
48被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We introduce EMMA, an End-to-end Multimodal Model for Autonomous driving. Built upon a multi-modal large language model foundation like Gemini, EMMA directly maps raw camera sensor data into various driving-specific outputs, including planner trajectories, perception objects, and road graph elements. EMMA maximizes the utility of world knowledge from the pre-trained large language models, by representing all non-sensor inputs (e.g. navigation instructions and ego vehicle status) and outputs (e.g. trajectories and 3D locations) as natural language text. This approach allows EMMA to jointly process various driving tasks in a unified language space, and generate the outputs for each task using task-specific prompts. Empirically, we demonstrate EMMA's effectiveness by achieving state-of-the-art performance in motion planning on nuScenes as well as competitive results on the Waymo Open Motion Dataset (WOMD). EMMA also yields competitive results for camera-primary 3D object detection on the Waymo Open Dataset (WOD). We show that co-training EMMA with planner trajectories, object detection, and road graph tasks yields improvements across all three domains, highlighting EMMA's potential as a generalist model for autonomous driving applications. We hope that our results will inspire research to further evolve the state of the art in autonomous driving model architectures.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论Animats · 2024-11

“Waymo tried cameras-only recently as a research project.[1][2] They seem to do about as well as Tesla, which they don't consider good enough. [1] https://www.forbes.com/sites/bradtempleton/2024/10/30/waymo-builds-a-vision-based-end-to-end-driving-model-like-teslawayve/ [2] https://arxiv.org/pdf/2410.23262”

A Short Introduction to Automotive Lidar Technology原评论 ↗
评论m_ke · 2024-11

“hey just got back on and the papers you shared are the main works that I was about to link. There's also a new VLA paper fro waymo https://arxiv.org/abs/2410.23262v2 and some recent talks on youtube: - OpenVLA: https://www.youtube.com/watch?v=-0s0v3q7mBk - The current state of robotics by Alex Irpan: ‬https://www.youtube.com/watch?v=XocmVe1FCMY - Robo…”

A Revolution in How Robots Learn原评论 ↗
评论famouswaffles · 2025-02

“Seems to be the case to me, reading this and waymo's attempts. There's paper on EMMA here - https://arxiv.org/abs/2410.23262 And there are state of the art weather prediction transformers. https://arxiv.org/abs/2312.03876”

The Generative AI Con原评论 ↗