Recursive Language Models
Resource: https://arxiv.org/html/2512.24601v1
Abstract:
We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy that treats long prompts as part of an external environment and allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt. We find that RLMs successfully handle inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of base LLMs and common long-context scaffolds across four diverse long-context tasks, while having comparable (or cheaper) cost per query.
Metodology
One known issue of LLM is called context rot: as the context get longer, the quality of the quality of the output decreased. 1
Thesis (of the authors): is it possible to scale the context size of a general purpose LLMs by orders of magnitude?
Authors take inspiration from out-of-core algorithms, in which data-processing systems with a small but fast main memory can process far larger datasets by cleverly managing how data is fetched into memory.
Recursive Language Models is a general-purpose inference paradigm. The idea is that, instead of fed long prompts into the neural network directly, they are treated as part of the environment that the LLM can symbolically interact with.
- REPL stands for Read-Eval-Print Loop
Also, RLMs encourage the LLM, in the code it produces, to programmatically construct sub-tasks on which they can invoke themselves recursively.
Benchmarks
LLM are tested on tasks represented by dataset benchmarks.
The following are tested by the authors:
- S-NIAH: needle-in-the-haystack tasks that require finding a specific phrase or number in a large set of unrelated text
- BrowseComp-Plus: A multi-hop question-answering benchmark for DeepResearch questions that requires reasoning over multiple different documents. The percentage of each correct answer is reported.
- OOLONG: A long reasoning benchmark that requires examining and transforming chunks of the input semantically, then aggregating these chunks to form a final answer
- OOLONG-Pairs: manually modified the trec_coarse split of OOLONG to include 20 new queries that specifically require aggregating pairs of chunks to construct the final answer.
- LongBench-v2 CodeQA: A multi-choice code repository understanding split from LongBench-v2 that is challenging for modern frontier models. We report the score as the percentage of correct answers. Each task requires reasoning over a fixed number of files in a codebase to find the right answer.
Experiments
Given the nature of LLMs, only empirical experiments can be done. Authors try to experiment GPT-5 (2025) and an open model (Qwen3-Coder-480B-A35B; 2025) across four diverse tasks with varying levels of complexity for deep research, information aggregation, code repository understanding and synthetic pairwise reasoning task where even frontier models fail catastrophically.
Experiments demonstrates that RLM have extremely strong performance even at the 10M+ token scale, while maintaining a comparable or lower cost.
For the GPT-5 experiments, we use GPT-5-mini for the recursive LMs and GPT-5 for the root LM, as we found this choice to strike a powerful tradeoff between the capabilities of RLMs and the cost of the recursive calls.
RLM with REPL: RLM that loads its context as a string in the memory of a Python REPL environment. The REPL environment also loads in a module that allows it to query a sub-LM inside the environment.
Observations
- RLMs can scale to the 10M+ token regime and can outperform base LMs and existing task-agnostic agent scaffolds on long context tasks
- The REPL environment is necessary for handling long inputs, while the recursive sub-calling of RLMs provides strong benefits on information-dense inputs
- LM performance degrades as a function of input length and problem complexity, while RLM performance scales better
- The inference cost of RLMs remain comparable to a base model call but are high variance due to differences in trajectory lengths.
- RLMs are a model-agnostic inference strategy, but different models exhibit different overall decisions on context management and sub-calling. In general GPT-5 and Qwen both exhibit strong performance compared to their base models, but GPT-5 is slightly better and solves all tasks in BrowseComp-Plus
Emergent Patterns in RLM Trajectories
Even without explicit training, RLMs exhibit interesting context management and problem decomposition behavior.
- Filtering input information using code execution based on model priors. The general models are able to filter input context without explicitly seeing it. Furthermore, model priors enable the RLM to narrow the search space and process fewer input tokens. For example in one case GPT-5 used regex to search for chunks containing key-words in the original prompt.
- Chunking and recursively sub-calling LMs. RLMs defer essentially unbounded-length reasoning chains to sub-(R)LM calls. The choice of decomposition can greatly affect task performance, especially for information-dense problems.
- Answer verification through sub-LM calls with small contexts. Some sub-LMs calls were made to verify the answers made by RLMs, either programmatically or not.
- Passing recursive LM outputs through variables for long output tasks. RLMs are able to produce essentially unbounded tokens well beyond the limit of the base LM by returning variables in the REPL as output. Through the REPL, the RLM can iteratively construct these variables as a mixture of programmatic and sub-(R)LM output calls
Related Work
Long Context LM Systems
There are two primarly orthogonald directions for long context management:
- Changing the architecture of and retraining the base LM to handle longer contexts
- Building a scaffold around the LM that implicitly handles the context
RLM focus on (2).
Task Decomposition through sub-LM calls.
Many LM-based agents use multiple well-placed LM calls to solve problems, however many of these calls are placed based on human-engineered workflows. Sever methods like: ViperGPT Surís et al. (2023), THREAD (Schroeder et al., 2025), DisCIPL (Grand et al., 2025), ReDel Zhu et al. (2024), Context Folding (Sun et al., 2025), and AgentFold (Ye et al., 2025) have explored deferring the choice of sub-LM calls to the LM.
These techniques emphasize task decomposition through recursive LM calls, but are unable to handle long context inputs beyond the length of the base LM. On the other hand, RLM can handle long context placing the prompt as part of the external environment.