按年被提到
20262026 · 6 次2026
系统、数据库与网络›计算机体系结构与硬件·arXiv:2608.16157 · cs.DC

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution

Shuo Yang, Xiaoze Fan, Melissa Pan, Haocheng Xi, Zhe Wang, Shanlin Sun, Kurt Keutzer, Song Han, Matei Zaharia, Chenfeng Xu, Ion Stoica · 2026
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 6 人首次 2026-08-21 · 最近 2026-11-02
6不同的人
2活跃月份
6讨论串
1被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack, including model layout and loading, expert residency, CPU--GPU execution, agentic state reuse, and runtime memory management, around two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine. Rather than committing to a fixed offloading strategy, FreeToken continuously maps computation and model state onto the resources actually available. FreeToken supports more than 20 MoE models and real coding and tool-using agents across hardware ranging from an 8GB laptop GPU to a single workstation GPU. More importantly, it changes what these machines can practically serve, from a 35B model on a laptop to a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. FreeToken turns open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence. We release the system at flashml.ai.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论gravypod · 2026-08

“The linked paper is a much more interesting read: https://arxiv.org/pdf/2608.16157 I think there are many other significant inference improvements which an be built out that are being ignored because the API between client and inference stack would be tricky to nail down. This is not a criticism of the research but instead the presentation but the repo …”

Run 290B+ frontier MoE models locally on your gaming PC原评论 ↗
被提交为 story · 5 次