按年被提到
20242024 · 1 次2026
AI 安全、伦理与社会影响›大模型与人类行为·arXiv:2403.04132 · cs.AI

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, Ion Stoica · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-03-10 · 最近 2025-04-29
2不同的人
2活跃月份
2讨论串
391被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large Language Models (LLMs) have unlocked new capabilities and applications; however, evaluating the alignment with human preferences still poses significant challenges. To address this issue, we introduce Chatbot Arena, an open platform for evaluating LLMs based on human preferences. Our methodology employs a pairwise comparison approach and leverages input from a diverse user base through crowdsourcing. The platform has been operational for several months, amassing over 240K votes. This paper describes the platform, analyzes the data we have collected so far, and explains the tried-and-true statistical methods we are using for efficient and accurate evaluation and ranking of models. We confirm that the crowdsourced questions are sufficiently diverse and discriminating and that the crowdsourced human votes are in good agreement with those of expert raters. These analyses collectively establish a robust foundation for the credibility of Chatbot Arena. Because of its unique value and openness, Chatbot Arena has emerged as one of the most referenced LLM leaderboards, widely cited by leading LLM developers and companies. Our demo is publicly available at \url{https://chat.lmsys.org}.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论bhelkey · 2025-04

“If September 2024 marks the date in your mind stagnation was obvious, surely the last improvement must have come before? Whatever the case, there are open platforms that give users a chance to compare two anonymous LLMs and rank the models as a result [1]. What I observe when I look for these rankings is that none of the top ranked models come from befo…”

Generative AI is not replacing jobs or hurting wages at all, say economists原评论 ↗
被提交为 story · 1 次