按年被提到
20252025 · 2 次2026
大模型与语言模型›大模型推理与思维链·arXiv:2505.19590 · cs.LG

Learning to Reason without External Rewards

Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, Dawn Song · 2025
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2025-05-27 · 最近 2025-06-02
2不同的人
2活跃月份
2讨论串
33被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Training large language models (LLMs) for complex reasoning via Reinforcement Learning with Verifiable Rewards (RLVR) is effective but limited by reliance on costly, domain-specific supervision. We explore Reinforcement Learning from Internal Feedback (RLIF), a framework that enables LLMs to learn from intrinsic signals without external rewards or labeled data. We propose Intuitor, an RLIF method that uses a model's own confidence-termed self-certainty-as its sole reward signal. Intuitor replaces external rewards in Group Relative Policy Optimization (GRPO) with self-certainty scores, enabling fully unsupervised learning. Experiments demonstrate that Intuitor matches GRPO's performance on mathematical benchmarks while achieving better generalization to out-of-domain tasks like code generation, without requiring gold solutions or test cases. Our findings show that intrinsic model signals can drive effective learning across domains, offering a scalable alternative to RLVR for autonomous AI systems where verifiable rewards are unavailable. Code is available at https://github.com/sunblaze-ucb/Intuitor

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论jimmySixDOF · 2025-06

“RL is proving to be a weird science lately: >Spurious Rewards: Rethinking Training Signals in RLVR ### *TL;DR* We show that you can do RLVR on Qwen2.5-Math models with *completely random or incorrect rewards*, and still get massive math benchmark gains. All of the following spurious rewards give 15-20+ points on MATH-500 when RLVR training Qwen2.5-Math…”

ReasoningGym: Reasoning Environments for RL with Verifiable Rewards原评论 ↗
被提交为 story · 1 次