Latent Reasoning in Continuous Space for Unified Multimodal Models
Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, Jinwoo Shin · KAIST AI
30 Oct 2026 · cs.CV, cs.CL, cs.LG · multi-gpu · code · reasoning & test-time compute, rl & post-training, pretraining, data & scaling, multimodal, evaluation & benchmarks
41 hf upvotes · 0 hn points, 0 comments · 9 stars · – citations
Unified Multimodal Models (UMMs) achieve remarkable success on diverse multimodal understanding and generation tasks, yet they still struggle to effectively leverage language-based chain-of-thought reasoning for image generation. As a result, existing reasoning-augmented UMMs often fail to inherit the test-time scaling and fine-grained controllability observed in large language models. We attribute this failure to a fundamental modality mismatch: reasoning is performed in a discrete token space, whereas image generation operates in a continuous space. To bridge this gap, we introduce a Latent Reasoning framework in Continuous space (LARC), which alleviates the token-level bottleneck by reasoning over continuous hidden states. Specifically, the training framework consists of two stages: (i) curriculum supervised fine-tuning (SFT), and (ii) information-gain reinforcement learning (RL). First, the curriculum SFT stage gradually converts token-level reasoning into latent reasoning. Subsequently, the information-gain RL stage self-evolves latent reasoning without ground truth supervision. Experiments across diverse text-to-image generation benchmarks show that LARC consistently improves generation quality, prompt following, and compositional fidelity. Notably, LARC achieves state-of-the-art performance on GenEval with a 7%p gain over BAGEL and demonstrates clear test-time scaling behavior, outperforming token-based generation under both parallel and sequential test-time scaling strategies.
Compute: multi-gpu. Fine-tunes BAGEL (14B) with 1M CoT samples; 8×A100, ~396 GPU-hours total.
Paper's setup: 8 × NVIDIA A100, ~396 GPU-hours. Smallest run that tests the claim: LARC fine-tuning of BAGEL on the 1M-sample CoT dataset; no smaller configuration reported.
Potential
signals behind the hidden gems rankingemerging topic “Unified Multimodal Models”: 6 papers in 2 weeks vs 2 in the 18 days before · early traction: 8 upvotes/day in its first 5 days · from kaist-ai, whose recent “Look Only When Needed: Certificate-Guided Visual Routing for…” drew 115 upvotes/points
Existing reproductions
Not checked yet. Looks for GitHub repos that cite this arXiv id and HF models tagged with it, then separates independent reproductions from code that only cites or uses the paper, and from paper lists.
Comments