The Recurrent Looped Transformer (RLT) presents a novel architecture designed to increase the effective depth of transformer models as input sequences lengthen. Unlike traditional transformers with fixed-depth processing per token, RLT’s recurrent decoder extends the computational path proportionally with the number of tokens processed. After processing t tokens, the decoder path traverses t times the number of decoder layers, enabling deeper latent reasoning over longer sequences.
RLT integrates a causal encoder that builds a global key–value memory with a recurrent decoder that carries forward its final hidden state and a layerwise sliding-window attention (SWA) cache across all tokens in prompts and responses. This design supports continuous latent computation, allowing the model to maintain and update an extended temporal context.
The architecture emphasizes co-design between the model, hardware, and reinforcement learning algorithms. It supports parallel encoder operations, batching of known tokens, memory reuse, and checkpointing around the recurrent core. These features aim to optimize efficiency and scalability during training and inference.
A key aspect of RLT is its infinite temporal depth, referring to an extensible path through the decoder layers as sequences grow, rather than infinite computation per token. The model uses 48 layers each for the encoder and decoder, with shared weights for attention and feed-forward networks. This results in a temporal path that scales linearly with sequence length, traversing 48 decoder blocks per token.
The decoder updates proceed sequentially, constructing recurrent outputs and updating the SWA caches for attention. The model maintains forward consistency and complete gradients through full backpropagation through time (BPTT), including paths through recurrent outputs, decoder key-value caches, and encoder memory. This ensures accurate gradient updates and supports exact current-policy replay during reinforcement learning.
RLT also addresses challenges related to behavior log-probabilities and importance sampling, ensuring that sampling distributions accurately reflect the model’s behavior. The approach removes structural mismatches at prompt boundaries, although numerical kernel parity and off-policy estimation remain areas for further work.
Overall, the Recurrent Looped Transformer offers a promising framework for extending transformer depth dynamically with sequence length, potentially enhancing reasoning capabilities and hardware efficiency. Further evaluation is needed to establish realized gains in reasoning performance and reinforcement learning scalability.