JiantaoJ称循环Transformer论文靠显式解码学会多跳推理

全部来源

• JiantaoJ称循环Transformer论文靠显式解码学会多跳推理

  JiantaoJ · 主条目

  JiantaoJ介绍最新循环Transformer论文，称循环可支持隐式推理；新技巧让模型在循环模块之间显式解码token，从而以可泛化方式学习多跳推理。引述中Hengyu Fu还提到前沿实验室传闻采用循环Transformer，以及研究用机制可解释性分析瓶颈，但正文在此处截断。

  查看原始帖子 ↗
  查看原文

  Check out our latest paper on looped transformers! We found looping enables implicit reasoning, and a new trick we introduced that forces the model to decode the tokens explicitly between looped modules enables the model to learn multi-hop reasoning in a generalizable way! Hengyu Fu: Frontier labs are rumored to be adopting looped Transformers for their potential to unlock implicit reasoning. But can looped Transformers really master it? We find a bottleneck that keeps looped Transformers from perfect implicit reasoning. Using mechanistic interpretability,

前后事件