“Internal vector recurrence is more or less what the Universal Transformer paper proposes. This recent paper does so by sharing the transformer weights in a single recurrence setup: https://arxiv.org/abs/2607.22083, and this one does so at a per-token level: https://arxiv.org/abs/2507.10524 I agree with you about not trusting Big Ai. I'm amazed that so mu…”
Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model
We present Nanbeige4.2-3B, a compact general agentic model with 3B non-embedding parameters. It delivers strong performance across code-agent, office-agent, and complex tool-use tasks while maintaining highly competitive reasoning capabilities in mathematics, coding, and science. Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters. For SFT data and trajectory construction, we expand the diversity of executable environments, task assets, and agentic scaffolds through real-world deployment and large-scale synthesis. Our RL pipeline applies mixed-mode RLHF over Think and Non-Think responses to improve overall model quality and reduce failure cases, length-controlled reasoning RL to balance accuracy and reasoning efficiency, and agentic RL with outcome and process rewards to stabilize long-horizon training. Extensive evaluations show that Nanbeige4.2-3B outperforms larger models, including Qwen3.5-9B and Gemma4-12B, across diverse agentic benchmarks while remaining competitive on reasoning and alignment tasks. Performance with OpenClaw further supports its use as a compact local personal assistant.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Related: Looped Transformer: https://sebastianraschka.com/llm-architecture-gallery/looped-depth-sharing/ Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts: https://arxiv.org/abs/2607.22083 Nanbeige-4.2-3B https://huggingface.co/Nanbeige Ouro: https://ouro-llm.github.io/”