推荐joaogui1 · 2023-01
“This initiative is getting some pretty good practical results Petals, taking a look at their citations should lead you to other examples in the field.”
推荐它,实际效果不错,可顺着其引用找到更多例子
多方共享显卡,在消费级 GPU 上协作运行并微调千亿级模型
论文针对 100B 以上参数的大语言模型对高端硬件的需求,提出 Petals:联合多方资源协作进行大模型推理和微调。RAM 卸载对交互式推理太慢,托管 API 又不够灵活,无法访问权重、注意力或 logits。作者展示在极大模型上这一策略优于卸载,可在消费级 GPU 上运行 BLOOM-176B 推理,速度约每秒 1 步,足够许多交互式应用。与多数推理 API 不同,Petals 原生暴露所服务模型的隐藏状态,可基于高效微调方法训练并共享自定义模型扩展。
HN 上的人把它当作分布式运行大模型的有实际结果的工作,也引用它测得的 RAM 卸载速度极限来说明单机卸载为何不够快。
“This initiative is getting some pretty good practical results Petals, taking a look at their citations should lead you to other examples in the field.”
“There was great work last year on distributed training and inference:”
“It's mentioned in their paper: Petals”
“See details in our paper: Petals”
“our measurements [1] show that even the theoretically-best RAM offloading setup”