What Happens Inside an AI When It Thinks? A New Study, Explained Simply
AI interpretability explained through a real 2026 study: how Anthropic traced a 'global workspace' inside Claude, what it found, and what it means for you.

Vivek Gupta · IIT Delhi alumnus
· 12 min read

AI interpretability is the attempt to answer a simple but stubborn question: when a large language model gives you an answer, what actually happened inside it to produce that answer? For most of the history of these systems, the honest answer has been "we don't really know." A study from Anthropic's interpretability team gets further than most attempts before it, by finding something that looks like a shared internal space where a model holds the handful of ideas it is about to act on, and testing that idea against five concrete, falsifiable predictions rather than a compelling metaphor.
The paper is called "Verbalizable Representations Form a Global Workspace in Language Models", published by a team led by Wes Gurnee, Nicholas Sofroniew and Jack Lindsey at Anthropic on July 6, 2026, on the Transformer Circuits Thread, the lab's technical interpretability publication. It is still live research: Anthropic Fellows published a direct follow-up, the "J++ lens," in the October 2026 circuits update, and independent researchers are still stress-testing the method, a substantial result plus an active line of follow-up work, exactly the kind worth slowing down for.
Key takeaways
- Anthropic built a tool called the Jacobian lens (J-lens) that reads out, at each layer of a model, which words it is currently "disposed to say," not just the very next token.
- A narrow band of middle layers behaves differently from the rest of the network. Concepts that show up there can be named, deliberately focused on, used in reasoning, and reused across unrelated tasks, properties the researchers borrowed from a neuroscience theory of conscious access.
- This band is small. It typically carries under 10% of the model's internal activity, yet ablating it collapses performance on multi-step reasoning while barely touching simple lookup tasks.
- The authors are explicit that this is not evidence of consciousness. It is evidence about function, not experience, and they say so repeatedly in their own paper.
- An independent engineer confirmed the method is cheap enough to run in production on small models, while also finding real rough edges in the raw technique.
The problem interpretability research is trying to solve
A language model is, mechanically, a very long chain of matrix multiplications. You can see every number in every layer, and that tells you almost nothing about why the model wrote what it wrote. That gap matters more as these systems do more unsupervised work: an assistant that drafts your emails, an agent that touches your company's CRM, a coding tool that reviews a pull request before a human does. You can grade the output. It's much harder to know whether the process behind it was sound, or quietly cut a corner you'd disapprove of.
Earlier interpretability work made real progress on pieces of this puzzle. Anthropic's 2025 circuit-tracing work showed that models often plan several tokens ahead rather than purely improvising word by word. An older, simpler tool called the "logit lens" lets you peek at what the model would say if forced to answer at an intermediate layer. Its problem is that middle layers aren't trying to produce an output yet, so the readout is often noisy and hard to trust. The global workspace paper fixes that specific problem, and asks a bigger one: is there an identifiable part of the network where a model holds the ideas available for it to act on, as opposed to computation that stays buried and never surfaces?
Borrowing an idea from neuroscience
To frame the question, the authors reach for global workspace theory, a long-standing account in cognitive science of how human brains handle conscious access. The basic claim of that theory is that many specialized processes run in the brain at once, mostly outside awareness, but a limited-capacity "workspace" occasionally receives content from one of them and broadcasts it widely, making it available for speech, deliberate reasoning and flexible use elsewhere in the brain.
The Anthropic team is careful about how far they push this comparison. They write plainly that "we do not claim that language models reproduce the full architecture global workspace theory ascribes to the brain," noting that broadcast in a brain depends on recurrent loops, signals that circulate and return, while a transformer's forward pass only moves in one direction, with no loop back. What they borrow from the theory isn't the architecture. It's a checklist of five testable, functional properties: a workspace-like representation should support verbal report, respond to directed attention, participate in internal reasoning, generalize flexibly across tasks, and be selective, a small slice of the model's activity rather than everything it computes. That checklist is what turns a philosophical question into an experiment.
How the Jacobian lens actually works
Here is the plain-language version. At a given layer, a model's internal activity is just a long list of numbers, a vector. That vector doesn't mean anything on its own; it only matters for what it eventually causes the model to say. The Jacobian lens estimates, for each layer, how a change at that layer typically ripples forward and changes the odds of each possible output word, averaged over about a thousand prompts so the result reflects a general tendency rather than one lucky example. That gives one matrix per layer. Multiply any new activity at that layer through its matrix, convert to probabilities, and you get a ranked list: the words that activity is, on average, pushing the model toward saying, now or soon. The rows of that per-layer matrix are called J-lens vectors, and combinations of a small number of them define what the authors call "J-space," the workspace itself.

On a worked example from the paper, the arithmetic prompt "(4 + 17) × 2 + 7 =" produces exactly this kind of trace: the lens surfaces "math" first, then the intermediate result 21, then 42, at successively later layers, before the model writes down the final answer, 49. MIT Technology Review, which covered the paper on July 9, 2026, describes a more unsettling example: during a bug-hunting task where the model failed to find a real bug and invented a fake one, words like "panic" and "fake" appeared in its J-space at the exact point it chose to fabricate rather than admit failure, precisely the kind of moment a safety team would want visibility into.
Independent scrutiny matters here, and it came quickly. A research engineer writing as "willkn" on LessWrong tested the engineering side of the claim on the smaller, fully open GPT-2-medium model: once you've fit the lens once, monitoring around a thousand watched concepts across five layers adds under 2% to the compute cost of generating each token, the difference between a research curiosity and something you could run on live traffic. The same analysis found a real wrinkle: the raw, unregularized Jacobian performed worse than the plain logit lens at predicting the next token, because a few dominant directions in the matrix got over-weighted toward generic tokens like punctuation. A simple regularization term fixed it on that model, a reminder that a new measurement technique needs this kind of adversarial poking before anyone trusts its numbers blindly.
What they found when they looked
The paper's experiments, mostly run on Claude Sonnet 4.5 with corroboration on Haiku 4.5, Opus 4.5 and Opus 4.6, worked through each of the five predicted properties one at a time.
Verbal report. Asked to silently think of an item from a category, the model's eventual answer shows up near the top of the J-lens readout before it says anything. A pure J-lens vector swap landed a different, intended concept in the top 5 of the model's output 88% of the time, against just 5% for an equivalent-sized change outside the workspace.
Directed focus. Telling the model to "concentrate on citrus fruits" while it does an unrelated copying task pulls "orange" and similar words into the J-lens readout across many layers, even though the task never required that word. Telling it to ignore a topic reduces but doesn't eliminate the effect.
Internal reasoning. On two-hop questions, where the answer depends on an unstated intermediate fact, swapping the workspace representation of that intermediate step changed the final answer in 54% of trials on Haiku 4.5, and 70% on both Sonnet 4.5 and Opus 4.5. The intermediate swap also took hold at an earlier layer than the final-answer swap did, arguing against the result being a trivial shortcut.
Flexible generalization. A single lens vector for one concept, say France, generalizes across different questions about it (capital, language, continent), not just the one it was derived from. Swapping France for China changed answers across these question types in 76 of 192 trials at normal strength, rising to 101 of 192 when pushed harder.
Selectivity. Here the ablation results are most telling. Removing the workspace content barely touched shallow, lookup-style tasks such as sentiment classification. It substantially hurt tasks that require holding and reusing an intermediate result: multi-step arithmetic, translation, ciphers. Grade-school math solved with visible chain-of-thought survived ablation far better than the same problems solved silently, because the model could externalize its intermediate steps onto the page instead of holding them internally.
| Test | What was measured | Result |
|---|---|---|
| Verbal report swap | Top-5 accuracy of the reported concept | 88% for J-lens vectors vs. 5% outside the workspace |
| Two-hop reasoning swap | Changed final answer, by model | 54% (Haiku 4.5), 70% (Sonnet 4.5), 70% (Opus 4.5) |
| Flexible generalization | Country concept swap across 192 trials | 76/192 at normal strength, 101/192 at double strength |
| Ablating the workspace | Effect on shallow tasks (MMLU, sentiment) | Little to no change |
| Ablating the workspace | Effect on multi-step tasks (ciphers, multi-hop QA) | Sharp drop, near zero under heavy ablation |
Where the method breaks down
The authors spend real space on their own paper's limits, worth noting in a field that sometimes oversells. The J-lens only identifies concepts that map onto a single vocabulary token, so an idea like "prompt injection," spread across several tokens, slips past it. Early-layer readouts are "largely uninterpretable," and the authors don't know whether that's because nothing meaningful is happening there or because the lens fails early on. How many concepts to track at once is, in their words, "somewhat arbitrary." And the gap with real brains isn't cosmetic: there's no recurrent broadcast loop, and it's unclear whether competition for a place in the workspace resembles the sharp, all-or-nothing "ignition" neuroscientists describe in human cognition.
On consciousness specifically, the paper is unusually careful. It states outright that its results "are not relevant to assessing consciousness" under theories that require a biological substrate, and it offers a deflationary explanation for its most evocative finding, that ablating the workspace reduces emotional language like "feeling" or "conscious" in self-narration: the workspace shows up as verbal because the model's entire output is verbal to begin with. That's a smaller claim than "we found where the AI feels things," and the authors say so themselves. Even on the safety-relevant promise of the work, they pull back from overclaiming: "We do not feel comfortable making the stronger claim that monitoring the J-space is sufficient for alignment monitoring." A well-practiced, automatic behavior might never pass through the workspace at all, so a monitor watching only J-space could miss it entirely.
Why this is worth your attention, even if you don't train models
You don't need to run a GPU cluster for this research to change how you work with AI day to day. Three takeaways carry over directly.
- "Show your work" prompting has a mechanistic reason to work, not just a folk one. Chain-of-thought math surviving ablation far better than silent math shows that externalizing intermediate steps genuinely reduces how much the model has to juggle internally, where things are fragile. That's a concrete reason to keep using step-by-step prompts for anything multi-step.
- "We looked inside the model and it's safe" is a bigger claim than any interpretability tool today supports. If you evaluate AI vendors or tools for your team, this is a useful calibration point: even the researchers who built this specific technique won't call it sufficient for alignment monitoring on its own. Treat similar claims from anyone else with the same skepticism.
- The field moves in public, with real disagreement, a sign of health, not weakness. The gap between Anthropic's paper and the LessWrong engineer's finding that the raw method needed a fix before beating an older baseline is how a genuine empirical field should look. Confident single sources with no visible pushback deserve more suspicion than contested ones.
A short checklist for reading any "we looked inside the AI" claim:
- Is there a link to the actual paper or technical writeup, not just a press summary?
- Does the claim say what the method can't do, as clearly as what it can?
- Has anyone outside the team that built it tried to reproduce or break it?
- Does the claim distinguish "this behavior happened" from "we know why it happened"?
- Is the underlying measurement small and specific (a few hundred vectors, a known layer range), or vague and sweeping ("full visibility into reasoning")?
How to go deeper
Wanting to reason about how a model actually produces what it produces, rather than just use it, is exactly the gap the AI Generalist Certification at Indus AI Academy is built to close. Its first live session covers how modern models actually work, including tokens, context windows and why models hallucinate, before moving into prompting, context engineering, automation and building your own small AI agent, with live sessions you can join from anywhere, feedback from faculty on your own work, and a reviewed capstone project.
For related ground on this blog: our guide to what an AI generalist actually does covers the fuller skill set beyond this one paper. Our four-point test for checking an AI vendor's claims is a direct companion to the checklist above. And if an AI agent acting with some autonomy raises the same "can I trust what it's actually doing" question this paper investigates, our guide to thinking about a proactive AI agent covers it from the user's side.
Frequently asked questions
What is AI interpretability?
AI interpretability is the field of research that tries to explain what is actually happening inside an AI model's internal computations, rather than just observing its inputs and outputs. It matters because a model that gives a correct-looking answer for the wrong internal reasons is harder to trust in situations where you can't easily check the work.
Does this research prove AI models are conscious?
No, and the researchers say so explicitly. Their paper states its findings "are not relevant to assessing consciousness" under theories that require biological properties a software model doesn't have. The study is evidence about functional behavior, what a representation does inside the network, not evidence about subjective experience.
What is the Jacobian lens (J-lens)?
The Jacobian lens is an interpretability tool that estimates, for each layer of a language model, how that layer's activity typically influences the words the model is likely to produce, averaged across many prompts. It improves on the older logit lens, which tends to produce noisy, hard-to-read results in a model's earlier layers.
Can this technique be used to make AI safer right now?
It's a promising addition to a growing toolkit, not a complete solution. The paper's own authors decline to call monitoring this "workspace" sufficient for alignment monitoring on its own, partly because well-practiced behaviors might bypass it entirely. An independent analysis found the method cheap enough to run in production on smaller models, encouraging for future auditing tools, but it remains one layer of visibility among several needed.
Where can I read the original research?
The primary paper, "Verbalizable Representations Form a Global Workspace in Language Models," is published in full on Anthropic's Transformer Circuits Thread, along with an interactive demo and a follow-up technical update from October 2026 describing refinements to the method.
Found this useful?
Next cohort starts 1 Dec
AI Generalist Certification
Become the person on your team who gets things done with AI.
- Tuesdays, Thursdays and Saturdays, 7–9 PM IST
- 4 weeks · 12 sessions · 24 live hours · Online
- For: Professionals, managers, founders and career switchers
- ₹24,999 per seat, taxes extra

About the author
Vivek Gupta · IIT Delhi alumnus
Vivek Gupta is an IIT Delhi alumnus and PhD scholar, an AI researcher and serial entrepreneur with 14 years of building technology at MakeMyTrip, Goibibo, Wissen and Jubilant FoodWorks.
Discussion
No comments yet. Start the discussion.