按年被提到
20242024 · 3 次2026
数学、逻辑与理论›AI 数学与定理证明·arXiv:2411.04872 · cs.AI

FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett 等 24 人 · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2024-11-10 · 最近 2024-12-22
3不同的人
2活跃月份
3讨论串
66被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians. The questions cover most major branches of modern mathematics -- from computationally intensive problems in number theory and real analysis to abstract questions in algebraic geometry and category theory. Solving a typical problem requires multiple hours of effort from a researcher in the relevant branch of mathematics, and for the upper end questions, multiple days. FrontierMath uses new, unpublished problems and automated verification to reliably evaluate models while minimizing risk of data contamination. Current state-of-the-art AI models solve under 2% of problems, revealing a vast gap between AI capabilities and the prowess of the mathematical community. As AI systems advance toward expert-level mathematical abilities, FrontierMath offers a rigorous testbed that quantifies their progress.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论westurner · 2024-11

“ScholarlyArticle: "FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI" (2024) https://arxiv.org/abs/2411.04872.. https://epochai.org/frontiermath/the-benchmark: > [Not even 2%] > Abstract: We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by exp…”

FrontierMath: A benchmark for evaluating advanced mathematical reasoning in AI原评论 ↗
评论n4r9 · 2024-11

“There's the Frontier Math benchmarks [0] demonstrating that AI is currently quite far from human performance at research-level mathematics. [0] https://arxiv.org/abs/2411.04872”

AlphaProof's Greatest Hits原评论 ↗
评论pegasus · 2024-12

“When can infer somewhat from this section in the FrontierMath paper [1]: "Assessment of FrontierMath difficulty All four mathematicians characterized the research problems in the FrontierMath benchmark as exceptionally challenging, noting that the most difficult questions require deep domain expertise and significant time investment. For example, referr…”

Reimagining mathematics in a world of reasoning machines [video]原评论 ↗