快讯
Quentin Gallouédec发布TRL v1.15：融合LM head默认开启，峰值显存最高降低82%
Quentin Gallouédec发布TRL v1.15，称其为迄今最大的优化。新版本默认启用融合LM head，峰值显存最高降低82%，序列长度提升至7倍；DPO从10k增至59k tokens，GRPO从29k增至115k tokens。
事件来源
查看原文
RT Quentin Gallouédec TRL v1.15 is out. Our biggest optimization ever. Fused LM head, on by default: → up to 82% less peak VRAM → 7x longer sequences → DPO 10k → 59k tokens, GRPO 29k → 115k And more... https://github.com/huggingface/trl/releases/tag/v1.15.0