The Recurrent Looped Transformer (RLT) is a novel architecture that increases model depth dynamically as input sequences lengthen. Unlike traditional transformers with fixed depth per token, RLT's recurrent decoder extends the computation path proportionally to the number of tokens processed. After processing t tokens, the recurrent path traverses t times the number of decoder layers, allowing for deeper temporal reasoning.
RLT integrates a causal encoder that builds a global key-value memory and a recurrent decoder that carries forward its final hidden state along with a layerwise sliding-window attention (SWA) cache. This combination supports continuous latent computation as sequences grow, facilitating unbounded temporal depth without increasing per-token computational work.
The architecture is designed with model and hardware co-optimization in mind, enabling parallel encoder processing, sequence batching, memory reuse, and checkpointing around a recurrent core. This supports consistent training and inference workflows, including pretraining, supervised fine-tuning, sampling, and reinforcement learning replay, all sharing the same complete state transitions.
In practice, the model uses 48 layers in both encoder and decoder, with shared attention and feed-forward network weights across stages. Each token involves 96 logical blocks of computation, though decoder cross-attention causes variation in computational effort across blocks. Known tokens are encoded in batches causally, while decoder updates proceed sequentially, maintaining recurrent outputs and SWA caches.
The approach ensures forward consistency and complete gradient calculation through full backpropagation through time, accounting for recurrent outputs, decoder key-value caches, and encoder memory. Parameter updates invalidate old caches to maintain exact policy replay during reinforcement learning.
By maintaining behavior log-probabilities aligned with actual sampling distributions, RLT supports exact importance sampling and removes structural mismatches between prompts and responses. While the architecture offers theoretical advantages in reasoning depth and efficiency, empirical validation of hardware performance gains and reinforcement learning scalability remains forthcoming.
The Recurrent Looped Transformer represents a step toward models capable of deeper temporal reasoning with efficient computation, potentially impacting future developments in sequence modeling and reinforcement learning.