按年被提到
20212023 · 2 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:2111.15664 · cs.LG

OCR-free Document Understanding Transformer

Geewook Kim, Teakgyu Hong, Moonbin Yim, Jeongyeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, Seunghyun Park · 2021
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-01-21 · 最近 2023-05-24
2不同的人
2活跃月份
2讨论串
547被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Understanding document images (e.g., invoices) is a core but challenging task since it requires complex functions such as reading text and a holistic understanding of the document. Current Visual Document Understanding (VDU) methods outsource the task of reading text to off-the-shelf Optical Character Recognition (OCR) engines and focus on the understanding task with the OCR outputs. Although such OCR-based approaches have shown promising performance, they suffer from 1) high computational costs for using OCR; 2) inflexibility of OCR models on languages or types of document; 3) OCR error propagation to the subsequent process. To address these issues, in this paper, we introduce a novel OCR-free VDU model named Donut, which stands for Document understanding transformer. As the first step in OCR-free VDU research, we propose a simple architecture (i.e., Transformer) with a pre-training objective (i.e., cross-entropy loss). Donut is conceptually simple yet effective. Through extensive experiments and analyses, we show a simple OCR-free VDU model, Donut, achieves state-of-the-art performances on various VDU tasks in terms of both speed and accuracy. In addition, we offer a synthetic data generator that helps the model pre-training to be flexible in various languages and domains. The code, trained model and synthetic data are available at https://github.com/clovaai/donut.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论visarga · 2023-01

“Adept.ai is rumoured to have a model that understands screens and web pages, but they are still in pre-release stage. If you use the image as input, you can try the DONUT model [1] that directly answers questions without external OCR. It can be fine-tuned to suit your needs. You can get a pretty decent reading of screens and text documents with Amazon T…”

General technology for enabling AI capabilities with LLMs and Generative models原评论 ↗
被提交为 story · 1 次