Looped language models: how reusing layers lets a small model think longer

A looped language model is a neural network that runs the same block of layers more than once on each piece of text, instead of passing it through a long stack of different layers just once. The weights stay the same, but the amount of computation grows with every extra pass. The result is a model that can be small in memory yet spend more effort on a hard problem when asked to.

Reusing the same layers instead of stacking more

A standard transformer language model is a tall stack of layers, each with its own parameters. A token’s representation moves up the stack once, and the answer comes out at the top. Capability is usually bought by adding layers or widening them, which means more parameters to store.

A looped model changes one thing: part of the stack is shared and run repeatedly. An analogy is editing a draft. A standard model is a writer with a fixed number of reviewers, each seeing the text once. A looped model is one reviewer who can read the draft again, and then again, building on the previous pass. The reviewer is the same, so the staff cost does not grow, but the time spent per draft does.

The recent designs usually split the network into three parts: input layers that turn tokens into an internal representation, a recurrent block that is applied several times, and output layers that turn the final state into a prediction. The loop happens in the model’s internal numbers, not in words. Nothing extra is written out between passes. That is why the approach is often called latent reasoning or latent recurrence, in contrast with chain-of-thought, where a model reasons by generating visible text.

An idea older than modern chatbots

Sharing weights across depth is not new. In 2018, Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit and Lukasz Kaiser proposed the Universal Transformer, which applies one transformer block repeatedly and adds a per-position halting mechanism so that different tokens can receive different amounts of computation. The paper, presented at ICLR 2019, reported gains over standard transformers on several algorithmic and language tasks, and argued that standard transformers struggle with some problems such as copying strings or simple logical inference on inputs longer than those seen in training.

The idea returned in the era of large language models. In February 2025, Jonas Geiping and colleagues, including Tom Goldstein, published “Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach.” They built a proof-of-concept model with 3.5 billion parameters, trained on 800 billion tokens, that iterates a recurrent block to arbitrary depth at test time. According to the authors, performance on reasoning benchmarks improves, sometimes dramatically, with a computation load equivalent to a 50-billion-parameter model. They also point to advantages over chain-of-thought: no specialized training data is needed, it works with small context windows, and it can capture reasoning that is hard to put into words.

In October 2025, a team led by Rui-Jie Zhu and Jason Eshraghian introduced Ouro, a family of pre-trained looped language models (which they call LoopLM) at 1.4 billion and 2.6 billion parameters, trained on 7.7 trillion tokens. The authors report that these models match the results of language models of up to 12 billion parameters across a range of benchmarks. Their controlled experiments, they write, indicate that the advantage comes from better knowledge manipulation rather than greater knowledge capacity. Ouro also uses an entropy-regularized objective so the model learns how much depth to allocate.

Dory: a loop count chosen per request

The most visible recent release is Nvidia’s Dory. As covered in our report on Nvidia’s Dory 4B, the 4-billion-parameter model has 8 input layers, 20 recurrent layers that run repeatedly and 8 output layers, with the input reinjected at every loop. Users can set the loop count to 1, 2, 3 or 4 at inference time, with one set of weights. Nvidia recommends 2 as the default.

Nvidia’s own measurements show what the dial does. On SWE-bench Verified, the model scores 50.3 with one loop, 59.5 with two and 61.8 with four. On Terminal-Bench 2.1 the scores are 12.5, 18.1 and 22.2. Nvidia says looping also reduces the number of tokens the model needs to generate to solve tasks.

The last two rows show why the dial is not a free upgrade. On GPQA Diamond the two-loop result is slightly above the four-loop one, and on AA-Omniscience, which tests recall of facts, the score barely moves.

Why memory and speed do not scale the way intuition suggests

Looping saves parameters, not compute. Each extra pass costs roughly as much as running that block again, so a four-loop answer takes longer than a one-loop answer. What the model avoids is storing a larger set of distinct weights. That matters on devices where memory is the limit.

There is a second cost that is easy to miss: the attention cache. During generation, a transformer keeps a key-value cache for each attention layer. If every loop keeps its own cache, memory grows with the loop count. Nvidia’s model card says Dory uses last-loop KV caching, keeping one cache per recurrent attention layer, and states that with it the model carries no KV-cache overhead relative to an equivalent non-looped model. The card reports that accuracy stays close to the per-loop mode. This is a vendor claim from the model card, not an independent test.

Limits and what to watch

The evidence so far points to a pattern: looping helps most on tasks that need multi-step processing, such as coding agents and terminal work, and helps little on tasks that mostly need stored knowledge. That fits the Ouro authors’ reading that loops improve how knowledge is used, not how much a model holds. Nvidia’s comparison table also lists larger Qwen3.5 and Gemma 4 models that lead on several knowledge-heavy tests such as HLE and GPQA Diamond, so a looped 4B model does not replace a bigger one across the board.

Other open questions remain. The gains per extra loop shrink, with the largest step in Dory’s reported results coming from one loop to two. Training these models is harder than training a standard stack, and tooling is young: the Dory card notes that serving currently needs a vLLM pull-request build or a Dory-enabled llama.cpp build. Because the reasoning stays in internal numbers rather than text, it is also harder for a person to read than a chain-of-thought trace, although the Ouro authors report that LoopLM’s reasoning traces align more closely with its final outputs than explicit chain-of-thought traces do.

Worth watching: independent replications of the vendor benchmarks, support landing in mainstream inference engines, whether models learn to pick their own loop count per token rather than relying on a setting, and whether the approach scales from a few billion parameters to the frontier.