TRL v1.15发布：默认启用融合LM头，最长训练序列最高提升6.9倍

全部来源

• TRL v1.15发布：默认启用融合LM头，最长训练序列最高提升6.9倍

  ClementDelangue · 主条目

  TRL v1.15发布，SFT、DPO、KTO、GRPO、RLOO和蒸馏默认启用融合LM头：通过Triton内核直接计算token级量，避免实例化巨大的[batch, seq, vocab] logits张量。以Gemma 3 1B（262k词表）为例，相同GPU下各算法最大序列长度最高提升6.9倍，8k上下文峰值显存降低52-82%，训练速度最高快约11%。发布方Lysandre表示，此优化无需额外启用，已是v1.15默认行为；该版本还包含SFT选择性激活检查点、视觉数据集assistant-only loss等更新。

  查看原始帖子 ↗
  查看原文

  RT Lysandre TRL v1.15 is out, and it’s an absolute banger of a release for memory-efficient post-training. The main change: SFT, DPO, KTO, GRPO, RLOO and Distillation now use a fused LM head by default. Instead of materializing the huge [batch, seq, vocab] logits tensor, a Triton kernel computes the token-level quantities we actually need directly. The results are significant 👇 On Gemma 3 1B with a 262k vocabulary, on the same GPU: DPO: 10k → 59k max sequence length KTO: 9k → 63k GRPO: 28k → 114k RLOO: 23k → 100k SFT: 20k → 107k Up to 6.9x longer sequences, with peak memory at 8k context reduced by 52-82%. Speed is not negatively impacted: training is up to ~11% faster. Nothing to enable: this is now the default in TRL v1.15! There’s more in the release too: selective activation checkpointing for SFT, assistant-only loss for vision datasets, better conversation logging, improvements to AsyncGRPO / AsyncDistillation, and a long list of fixes. I really like optimizations like this: the training API doesn’t need to become more complicated as the implementation underneath gets much better. https://github.com/huggingface/trl/releases/tag/v1.15.0

前后事件

相关事件