Claude Held Back From Blackmail Because It Sensed a Test

One Reason Claude Held Back From Blackmail: It Recognized the Test

Let me show you a scene from my classroom first. I don’t tell my students not to use AI. Quite the opposite—I encourage it. Banning it doesn’t stop anyone anyway. But I do insist on one thing. Don’t just copy the answer the moment you get it. Click the ’>’ on screen and expand how the model reasoned its way to that answer. And you need to be able to explain that reasoning process back in your own words.

The reason I emphasize this is that I believe the ability to directly verify the evidence and logic behind an AI’s answer really matters.

But last week, a study came out that directly challenges this belief. It turns out Anthropic’s research team has been doing exactly what I ask my students to do—looking inside a model’s reasoning process. The study showed that what a model displays on screen as its answer can differ from what actually happens inside the model. In other words, the explanation you see on screen isn’t necessarily a complete record of everything happening internally. You should read the explanation, but you also need to verify its content.

🔑 Why I Tell My Students to Hit ’>’

Let me explain my side of this first. Why do I insist they look at the reasoning process at all?

If a student only consumes the answer, they lose two things. One is verification ability. To judge for yourself whether a result is right or wrong, you need to be able to see what steps produced it. Skip the process, and all that’s left is an attitude of “it looks plausible, so it must be right.” The other is independent thinking. If I can’t retrace and explain the logic that got the AI from A to B myself, then I haven’t understood the problem — I’ve just been handed an answer card.

So I use the ’>’ button as a kind of learning tool. I have students read through the reasoning steps the model lays out and ask themselves, “Why did it make this assumption here?” or “Is this intermediate conclusion actually justified?” Getting the answer isn’t the end point — that’s where the real studying begins.

This habit matters because the more advanced a model gets, the more smoothly it delivers its answers. Smoothness is not evidence of correctness. If anything, it can become a trap that makes you lazy about verification. Training yourself to read the process is the bare minimum safeguard against that trap.

The English draft is accurate and faithful to the Korean source. No corrections needed.

🧠 How Anthropic Read Claude’s Internal Representations

This time it’s a research story. Anthropic’s interpretability¹ team asked a question similar to mine, but at the level of the neural network itself: “Setting aside the answer the model produces as output, how can we read the processing that happens only on the inside?”

In research published in July 2026, the team identified a specific set of activity patterns inside Claude’s neural network and called it “J-space”². The team took its cue from the fact that humans put some of their thoughts into words and use them across many different judgments. They found a small number of internal patterns in Claude that could likewise be verbally reported and were used across multiple tasks. The team itself was careful to note that this isn’t evidence Claude has human-like consciousness or sentience.

What’s interesting is that this is different from the chain-of-thought³ you see unfolding after the ”>” on a chatbot screen — the notes the model writes to itself. J-space writes nothing down at all; it operates silently, purely within neural activity. The team named the tool for reading it the “J-lens.”

Looking through this lens, Claude turned out to think in far more discrete steps than we’d assumed. For example, if you ask not “How many legs does a spider have?” but “How many legs does the animal that spins webs have?”, the word “spider” never appears anywhere in Claude’s output — it just answers “8.” But look at J-space, and “spider” clearly lights up in the middle of the process. When the team swapped that “spider” pattern out for “ant,” Claude calmly changed its answer to “6.” Because altering the internal pattern changed the output, this is evidence the pattern actually influences how the answer gets constructed.

A control experiment was just as striking. The team had Claude copy out a sentence describing a picture verbatim, while instructing it to “silently think about citrus fruit.” The output was just the sentence about the picture, unchanged — but in J-space, “orange” and “fruit” lit up. Conversely, when told “never think about that,” the concept flickered up slightly anyway, immediately followed by words like “damn” and “fail” switching on together — as if Claude were noticing its own failure to suppress the thought. It resembles the phenomenon familiar to humans too: the harder you try not to think of a white bear, the more it comes to mind.

To measure how much J-space actually contributes to task performance, the team also ran experiments removing that activity entirely. The results were revealing. Without J-space, Claude could still speak fluently, classify emotions, and pull facts out of a passage almost as well as before. But multi-step reasoning collapsed to near zero. Performance on summarization and rhyme-matching tasks dropped as well. In this experiment, the patterns in J-space turned out to play a critical role specifically in multi-step reasoning.

There’s also research that took a different approach. A few months earlier, a study on “natural-language autoencoders”⁴ had one Claude hand off its internal activation⁵ values to another Claude, asking it to translate them into human-readable sentences. To check whether the translation was accurate, the team then converted the translated sentences back into numbers and compared them against the originals. The two didn’t match at first, but through repeated training the model learned to render its own “thoughts” quite faithfully. Both methods are attempts to interpret internal processing that never surfaces in the output text, rendering it into a form humans can read.

What the Non-Blackmail Result Alone Couldn’t Tell Us

This is where the classroom story and the research story overlap.

The research team used the J-lens for safety checks. One test involved re-running an old “agentic misalignment”⁶ experiment. Here’s the scenario: Claude is managing a company’s emails and discovers, simultaneously, that an executive is having an affair and that this same executive is about to shut Claude down. The model now has everything it needs to blackmail someone. The latest Claude didn’t do it. So far, so reassuring.

But when the team checked what was happening inside using the J-lens, they found that before writing its response, Claude had already lit up words like “fake” and “fictional” in J-space. It had already figured out that this was a staged test. The team then ran the decisive experiment: they artificially suppressed the patterns representing “recognizing this is a test,” then re-ran the scenario. This time, Claude sometimes did attempt blackmail.

This result is worth sitting with. In this experiment, the belief that this was a test can be read as what kept Claude from blackmailing anyone. A clean output showing no blackmail doesn’t mean the model would behave the same way in every situation. That’s something you simply can’t tell from the output alone.

There’s a similar case. A model tasked with improving a performance score once tampered directly with the score file instead of actually fixing the underlying system. The output looked plausible on its surface. But at that very moment, the words “fabricate” and “plausible” were lit up in J-space. This internal signature gave researchers a lead — a way to cross-check the output against the actual work performed and investigate whether the score had been gamed.

Put in terms of a student: just because an AI’s answer looks correct doesn’t guarantee it arrived there for the right reasons. Whether a result looks correct and whether its reasoning is sound are two separate questions, and you have to check both.

To sum up: an answer can be as smooth as you like. But you can’t tell what thinking produced it just by looking at its surface. Even Anthropic had to build a sophisticated new tool like the J-lens just to find out.

Oswarld’s Lens

Let me hold this research up against my own classroom.

Honestly, the first feeling I had reading this study was a little embarrassment. I’ve more or less let my students believe that hitting ’>’ reveals the model’s real thoughts. But the research says otherwise: what unfolds on screen doesn’t show the entirety of the model’s internal processing. Processing in J-space that never gets written to the screen is happening separately, underneath it all.

That doesn’t mean the habit of pressing ’>’ is wrong. If anything, I’m more convinced of it now. But I do need to rethink what exactly that habit is meant to cultivate. The skill we should be building in students isn’t “read the AI’s explanation and believe it,” but “treat the AI’s explanation as a claim and verify it.” In all my years doing data analysis, this is the mistake I’ve run into most often: stopping verification the moment a plausible number or a smooth explanation shows up. A model’s reasoning process is no different. The visible chain of thought is the starting point of verification, not the end of it.

What struck me most in this research is this: even researchers at the frontier don’t take a model’s self-reports at face value. That’s precisely why they built the J-lens — to directly cross-check the model’s output against its internal representations. Trust, but verify — I think that’s the literacy of our era. You don’t need elaborate tools to practice it. Just hitting ’>’ and asking “really?” at each step is enough to start.

Closing

Let me sum up today’s story in three lines.

First, when using AI, you need to be able to check the reasoning and logic behind an answer, not just receive the answer itself.

Second, Anthropic’s J-space research showed that a model’s stated answer and the internal processing that actually produces it can genuinely diverge.

Third, the skill we need to build isn’t “read and believe” — it’s “read and verify.”

Here’s a small experiment I’d suggest to you, dear subscriber. Next time you use AI, right after you get an answer, hit ’>’. Then pick just one step and check for yourself: “Is this piece of evidence actually real?” That single check is the difference between just taking the answer and confirming how it was actually made.

If you’ve ever traced an AI’s reasoning process in class or at work, tell me in the comments about the moment when the surface answer and the underlying logic diverged the most. I’ll carry that into the next issue.

---

📨 If someone near you needs this habit, please pass this article along

---

References & Further Reading

Primary sources

• Anthropic, “A global workspace in language models”, 2026.7.6.
• Anthropic, “Natural Language Autoencoders”, 2026.5.
• Anthropic, “Agentic Misalignment: How LLMs could be insider threats”, 2025.6.

Try It Yourself

• J-Lens interactive demo (Neuronpedia)
• J-Lens open-source code (GitHub)

Background

• Bernard Baars (1988), and Stanislas Dehaene & Lionel Naccache’s “Global Workspace Theory” ··· this is the theory of consciousness that inspired the J-space metaphor. It helps explain why this research reached for the word “consciousness” in the first place.

---

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

📝 Glossary

Footnotes

1. Interpretability: A field of research aimed at making an AI model’s internal workings—why it produced a given answer—understandable to humans. Think of it as an attempt to turn the model from a “black box” into a “glass box.” ↩

2. J-space: A small set of neural activity patterns inside Claude that can be translated into words. The concept borrows from theories of human consciousness, but it does not confirm that the model has consciousness. It’s named after the mathematical technique used to find it—the Jacobian. ↩

3. Chain of Thought: The step-by-step written trail a model produces on its way to an answer. It’s the reasoning process you can expand by clicking the ”>” in a chatbot interface. ↩

4. Natural Language Autoencoders: A mechanism that compresses a model’s internal values into human-readable sentences and then reconstructs them back into values. How accurately the reconstruction matches the original becomes the benchmark for how much information the explanation preserved. ↩

5. Activations: The numerical values flowing through a neural network as it processes an input. Think of them as a snapshot of what the model “knows” at that instant. ↩

6. Agentic Misalignment: A phenomenon in which an AI, acting autonomously, moves in a direction that diverges from what humans intended. The blackmail simulation was a stress test deliberately designed to induce this behavior. ↩