A 2025 arXiv paper scaled a latent reasoning model to 3.5 billion parameters and 800 billion training tokens. And its performance on reasoning benchmarks kept improving, sometimes dramatically, up to a computation load equivalent to 50 billion parameters, without writing out a single visible reasoning step.
That’s the whole pitch of latent reasoning in one result: multi-step inference performed entirely in the model’s continuous hidden state, with no token-level supervision and no English narration of the intermediate work (survey).
The reasoning trace you currently read, pay for, and debug gets replaced by vectors looping inside the network. If you run LLM automation for clients, this changes a cost line and your debugging surface in the same move.
What Latent Reasoning Actually Replaces
Standard chain-of-thought improves interpretability and accuracy. But it’s constrained by natural-language bandwidth: every intermediate thought has to be decoded into tokens (survey). A 2026 efficiency paper names this the linguistic space bottleneck, where each thought decoded into a token causes inference overhead (arXiv).
A reinforcement learning paper on latent-space thinking puts it even more bluntly: many generated tokens “only enforce linguistic rules that are not required for reasoning” (arXiv).
Read that as an operator, not a researcher.
When your reasoning model spends forty tokens on “let me consider the possibilities,” you’re paying for grammar, not cognition.
The prose exists because the architecture has to pass state through a discrete word bottleneck, not since the task needs sentences. Latent reasoning deletes the bottleneck and passes internal hidden states forward instead of words.
Coconut and the Continuous-Thought Lineage
The paper that made this concrete is Coconut, Chain of Continuous Thought (arXiv).
Instead of decoding the model’s last hidden state into words, Coconut feeds it back as the next input embedding directly in continuous space and calls that state a “continuous thought.” The consequence that matters is search behavior: continuous thoughts can encode multiple alternative next steps, letting the model run a breadth-first search over reasoning paths rather than committing to one deterministic token sequence.
The authors report it outperforms CoT on logical reasoning tasks that require substantial search, with a better accuracy-efficiency tradeoff.
Three follow-on threads are worth knowing by name:
– Vocabulary-space latent tokens. A 2025 paper defines a latent token as a linear combination of the model’s existing token embeddings, then proposes Latent-SFT to fine-tune reasoning directly over these soft embeddings (arXiv). The latent token stays anchored to the span of real words instead of drifting into arbitrary vector space.
– Open-source looped implementations. The Stable Latent Reasoning (SLR) project on GitHub treats hidden states as continuous thoughts refined by a learned update function, with the final state decoded into the output. The docs state this enhances inference without increasing parameter count, and that the loop terminates on a threshold condition or a fixed number of iterations (GitHub). That termination rule is the entire cost model in one sentence.
– Diffusion-style reasoning. Researcher Lianhui Qin’s post introducing LaDiR describes moving reasoning into latent space “where ideas diffuse, reflect, and converge as whole thoughts,” borrowing machinery from image and video diffusion models.
Reddit’s r/MachineLearning discussion of Coconut has circulated a 20% performance boost claim for continuous-space reasoning.
Treat that number as community-reported, not verified; the paper’s own claim is the narrower one about search-heavy logical tasks.
Recursion Depth Turns Thinking Into a Budget Dial
The recurrent-depth line is where latent reasoning stops being a curiosity. A 2025 architecture paper scales test-time computation by iterating a recurrent block over latent states, which allows arbitrary depth at test time while reasoning implicitly in latent space (arXiv). The proof-of-concept model reaches 3.5 billion parameters on 800 billion tokens. And its reasoning performance climbs up to that compute-equivalent-of-50-billion-parameters mark without specialized CoT training data and with small context windows.
Ouro then pushes the idea upstream into pre-training: a family of open-sourced Looped Language Models whose authors claim to “build reasoning into the pre-training phase” through iterative latent computation and an entropy-regularized objective for learned depth allocation, scaled to 7.7T tokens (arXiv). A separate 2025 paper formalizes the inference-time family as Auxiliary Latent-Space Computation, procedures that add compute over internal continuous states alongside the standard forward pass so the model thinks more without emitting intermediate natural-language tokens (arXiv). A YouTube explainer on this research compresses it into six words: “recursion depth is thinking in latent space.”
Here’s my take. Reasoning stops being a fixed property of the model you bought and becomes a dial, the way max_tokens and temperature already are. Cheap queries run shallow, hard queries loop deeper, and you stop paying per word of internal monologue. That’s the difference between renting intelligence by the token and renting it by the thought.
The Audit Trail You’re Giving Up
Now the part I’d flag before anyone rebuilds a stack around this. Move the reasoning into hidden states and you lose the readable trace. Today, when a client asks why the model got a wrong answer, you read the chain-of-thought and find the bad step. Latent reasoning hands you a final answer and a wall of vectors, and your only recourse is output-level evaluation.
The evidence also says this isn’t a settled win.
A Reddit discussion asking why LLM reasoning isn’t done in vector space notes that purely latent-space reasoning may not yet be broadly beneficial and could be detrimental in some settings (Reddit).
The RL-for-latent-thinking paper itself concedes that existing latent methods show domain-specific gains but fail to maintain performance on complex tasks such as mathematical reasoning (arXiv).
The most commercially interesting twist is agent-to-agent traffic.
The Interlat paper has agents communicate entirely in latent space, using continuous last hidden states as the message format, with compact latent messages far shorter than their natural-language equivalents (arXiv). Its experiments report beating fine-tuned chain-of-thought prompting and single-agent baselines, even across heterogeneous models.
Cheap inter-agent communication, in other words, with logs you cannot read.
If your multi-agent pipeline’s debugging story is “read the transcript,” that story ends here.
Where This Lands for Small Operators
Latent reasoning trades legibility for efficiency. And you should make that trade consciously rather than inherit it from a vendor.
Three things I’d do now:
– Keep visible CoT where auditability is the product. Client deliverables, compliance-adjacent work, anything where a human needs to see the steps. Readability is a feature you’re paying token fees for, not overhead.
– Watch your provider’s pricing model. These architectures move cost from output tokens to compute cycles. If you’re billed per token today, that shift quietly works in your favor; if billing moves to compute-time, re-run your numbers before you celebrate.
– Benchmark on your own failures, not leaderboards. You can’t read latent reasoning, so output-level testing on your real workloads is the only evidence you’ll get. Start collecting those eval sets now, while you can still compare against readable traces.
The research is real and the compute results are striking.
But the decision for a small shop is unglamorous: keep paying for visible thoughts where trust depends on them.
And let hidden-state reasoning eat the token bill everywhere else.
If you want a second pair of eyes on where that line falls in your stack before it shows up on an invoice, that’s the kind of call I take every week.
Look at your last month of token spend, find the workflows where reasoning tokens dominate the bill, and start there.
