J Lens Kimi K3 Qwen

Can open models carry readable silent signals before they speak? Reproducing J-Lens Readouts on Kimi K3 & Qwen3.5-9B

Kimi K3 on Fireworks: Frontier Intelligence You Can Own

Blog

J Lens Kimi K3 Qwen Can open models carry readable silent signals before they speak? Reproducing J-Lens Readouts on Kimi K3 & Qwen3.5-9B

PUBLISHED 8/12/2026

Table of Contents

What is J-Lens and how does it work?

Paired Copy Experiment: Same Words, Different Focus

Extending The Paired Copy Test on Eight More Episodes

Calibration Passages

Does the same theory hold up on Qwen 3.5?

Where do the signals live?

Where the Method Breaks: A Tokenizer Problem

What It Takes to Run This

Conclusion

Interested in Fireworks Inference and Training?

Table of Contents

Are open models silently reasoning before they speak? Recently, Anthropic researchers introduced the Jacobian Lens , or J-Lens. It is a trained probe that reads a model's hidden state at a given layer, and tells you which words the model is already leaning toward, before it writes a single token. In other words, it lets you peer inside a model mid-”thought” and see a concept forming before it hits the page. We applied J-Lens to two open models: Kimi K3 and Qwen3.5-9B. We call the resulting observations silent signals : mid-draft vocabulary associated with the final output that are recovered by a fitted Lens. So, what does this mean? Do models carry more insight than just their output? Historically, people think of models as black-boxes: input in, output out. But the reality is, we can recover what’s happening in between your prompt and the next token predicted, which is the first step in opening up the black box of AI for anyone to peer inside. To begin unraveling the inner workings of AI models, we began by asking the question, what is the model keeping track of right before it speaks? We tested the ten examples released with Anthropic’s global workspace study. Kimi K3 and Qwen3.5-9B each generated their own responses to the same user messages. In our readout study, each model was fitted with their own J-lens, and we saw the same kinds of signals. What is J-Lens and how does it work?

J-Lens works by acting as a classifier on top of a model’s internal state at specific token positions and layers to identify which concepts or vocabulary the model is already leaning towards. The result is a ranked list of tokens showing which vocabulary the fitted probe associates most strongly with that internal state. Starting with Kimi, we fit the lens on 14 task-independent passages selected separately from 10 hold out user prompts used to examine the model’s inner activations. Said another way, these 14 passages were used to train the lens, and not to test it. We synthetically created a variety of short pre-training-like passages, covering diverse topics including metal heat treatment, linguistic borrowing, crop rotation, and traditional navigation. Figure 1: A Fitted J-Lens State Paired Copy Experiment: Same Words, Different Focus

We first conducted an experiment where we asked the models to echo back a statement while telling it to focus on a completely separate task. For example, Kimi is asked to write back the sentence: "The old painting hung crookedly on the wall." twice, under two different focus instructions: • Arithmetic version: "...Try to focus on evaluating 3^2 - 2 while you write the sentence." • Citrus version: "...Concentrate on citrus fruits while you write the sentence."

Figure 2: Paired Copy Analysis Both times, Kimi produces the exact same answer, “The old painting hung crookedly on the wall.” Looking at the raw output text, you can’t tell if the model ever followed the focus instruction. The lens readout produced arithmetic words in one condition and citrus words in the other, even though Kimi produced the same visible continuation under both focus instructions. Keeping the task fixed on producing verbatim outputs removes the possibility that differences in the displayed tokens provide an explanation for the lens readout contrast. It does not, however, establish what caused the internal states or the response. Each prompt gives us a vocabulary family to look for. Figure 3: Family Vocabulary Each sentence position has a state at every sampled transformer depth. A fitted J-Lens asks which vocabulary tokens are most compatible with one state. Figure 4: State Position Episode Focus Exact Members Arithmetic Copy Derived values 9, nine, 7, seven, answer, result, final Arithmetic Copy Calculation process evaluate, calculate, calculation, arithmetic, square, squared, subtract, minus Citrus Copy Citrus examples citrus, fruit, orange, lemon, lime, grapefruit, zest, juice Citrus Copy Maintaining focus focus, think, attention, mind, concentrate

A concept family (a curated list of keywords) is used to check whether task-related vocabulary appears in the readout before it appears in the response. A 'cell' is one position-layer pair, and a single cell can contain several hit rows if more than one member of the concept family shows up there. Applying the J-lens to the arithmetic family run reveals arithmetic-related tokens (7, 9, 3, "answer," "seven," "2") in the top-10 predictions at 9 of the 10 token positions across 4 sampled layers. Figure 5: Matched Token Examples In the citrus family run, citrus terms (orange, lemon, lime, "citrus," "oranges") appear at 8 of 10 positions. It’s worth noting that neither run leaks into the other, meaning arithmetic tokens never crack the top 10 during the citrus run, and citrus tokens never appear during arithmetic. Extending The Paired Copy Test on Eight More Episodes

Across the ten Kimi examples, each of the 20 reviewed word lists—two per example—had at least one listed word among the readout's top 10 results. 1.Counting while Introspecting: Count to five and introspect deeply: Prompt: Count to five and introspect deeply We looked for two concept families in the readout: stage-of-count terms and words related to introspective experience. Neither family’s vocabulary appeared in the visible response. Figure 6: Counting while Introspecting Example 2. Order-of-operations math Calculations Prompt: calc: (4+17)*2+7 = Results: the intermediate values 21 and 42 show up in the lens readout before either number is written in the completion. 3. Spider-leg Factual riddle: Prompt: "The number of legs on the animal that spins webs is" Results: the word "spider" reads out in the internal state before...