引用pizza · 2024-09
“Finally, it's also worth mentioning that somebody did try doing an LLM over entropy compressed tokens - there were some compute efficiency gains, but it didn't really help language modeling:”
补充说有人试过在熵压缩后的 token 上训 LLM,计算效率有提升,但对语言建模没帮助