Muyao Yuan, Liyan Xu, Weizhan Zhang et al. · 3 Nov 2026 · single gpu · code

LoCo shortens chain-of-thought by building DPO preference pairs at the exact points where a reasoning model starts overthinking, rather than rewarding shorter traces overall.

The pipeline is self-contained: sample math traces from a small open reasoning model, flag candidate overthinking states, freeze those prefixes, regenerate shorter continuations that still reach the verified answer, and train with LoRA DPO on the resulting state-conditioned pairs. This fits a single GPU if you use a 1.5B distilled model instead of the paper's 7B. The target is concrete: roughly 40% shorter completions on math benchmarks at equal or better accuracy, and you can check whether the savings carry over to code. Along the way you build a DPO training loop, a verifier-based data pipeline and a careful accuracy-versus-length evaluation, and you see how where a preference pair branches changes what the model learns.

discuss → · arXiv · pdf · hf · alphaXiv · code