按年被提到
20242024 · 1 次2026
大模型与语言模型›词向量与预训练模型·arXiv:2402.16785 · cs.LG

CARTE: Pretraining and Transfer for Tabular Learning

Myung Jun Kim, Léo Grinsztajn, Gaël Varoquaux · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-03-06 · 最近 2025-01-09
2不同的人
2活跃月份
2讨论串
7被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Pretrained deep-learning models are the go-to solution for images or text. However, for tabular data the standard is still to train tree-based models. Indeed, transfer learning on tables hits the challenge of data integration: finding correspondences, correspondences in the entries (entity matching) where different words may denote the same entity, correspondences across columns (schema matching), which may come in different orders, names... We propose a neural architecture that does not need such correspondences. As a result, we can pretrain it on background data that has not been matched. The architecture -- CARTE for Context Aware Representation of Table Entries -- uses a graph representation of tabular (or relational) data to process tables with different columns, string embedding of entries and columns names to model an open vocabulary, and a graph-attentional network to contextualize entries with column names and neighboring entries. An extensive benchmark shows that CARTE facilitates learning, outperforming a solid set of baselines including the best tree-based models. CARTE also enables joint learning across tables with unmatched columns, enhancing a small table with bigger ones. CARTE opens the door to large pretrained models for tabular data.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论GVaroquaux · 2024-03

“As an author, I'm happy to see this work here. For those curious about what we have been up to on the topic of tabular learning, we have found a setting where deep learning does seem to bring sizable benefits (spoiler alert, it's about being able to pre-train, and transferring to new data works best when there are some strings to be recognized): https://…”

Why do tree-based models still outperform deep learning on tabular data? (2022)原评论 ↗
评论instanceofme · 2025-01

“Related: CARTE-AI, which can also deal with multiple tables. https://soda-inria.github.io/carte/ https://arxiv.org/pdf/2402.16785 The paper includes a comparison to TabPFN v1 (among others), noting the lack of categorical & missing values handling which v2 now seems to have. Would be curious to see an updated comparison.”

Show HN: TabPFN v2 – A SOTA foundation model for small tabular data原评论 ↗