按年被提到
20242024 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2409.04701 · cs.CL

Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models

Michael Günther, Isabelle Mohr, Daniel James Williams, Bo Wang, Han Xiao · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-09-20 · 最近 2025-06-09
2不同的人
2活跃月份
2讨论串
39被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Many use cases require retrieving smaller portions of text, and dense vector-based retrieval systems often perform better with shorter text segments, as the semantics are less likely to be over-compressed in the embeddings. Consequently, practitioners often split text documents into smaller chunks and encode them separately. However, chunk embeddings created in this way can lose contextual information from surrounding chunks, resulting in sub-optimal representations. In this paper, we introduce a novel method called late chunking, which leverages long context embedding models to first embed all tokens of the long text, with chunking applied after the transformer model and just before mean pooling - hence the term late in its naming. The resulting chunk embeddings capture the full contextual information, leading to superior results across various retrieval tasks. The method is generic enough to be applied to a wide range of long-context embedding models and works without additional training. To further increase the effectiveness of late chunking, we propose a dedicated fine-tuning approach for embedding models.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论snyy · 2025-06

“Hey HN! We're Shreyash and Bhavnick. We're building Chonkie (https://chonkie.ai), an open-source library for chunking and embedding data. Python: https://github.com/chonkie-inc/chonkie TypeScript: https://github.com/chonkie-inc/chonkie-ts Here's a video showing our code chunker: https://youtu.be/Xclkh6bU1P0. Bhavnick and I have been building personal …”

Launch HN: Chonkie (YC X25) – Open-Source Library for Advanced Chunking原评论 ↗
被提交为 story · 1 次