Pinned Tweet
Really excited about this work, led by the wonderful @CarlGuo866!
When training models to explain themselves, the training itself shifts model behaviors so the original explanation labels become stale. Surprisingly, the model actually *learns to explain its current self* and not at generated its training labels. This inclination to self-explain is so strong that *even when trained on another model's explanation labels, the model still learns to explain itself* better than the label source!
I’m excited about this on two fronts: first, it’s indication that models genuinely learn something “introspection-esque”, exploiting privileged access to themselves when generating explanations. Second, practically speaking, it shows self-explanations can directly be inserted into post-training without worrying about expensive label refresh.
Check out @CarlGuo866’s thread for more details! We pinpoint when this “introspective coupling” happens, and show that this training yields faithful self-explanations even in the presence of auxiliary fine-tuning. 👇
When training models to explain themselves, the training itself shifts model behaviors so the original explanation labels become stale. Surprisingly, the model actually *learns to explain its current self* and not at generated its training labels. This inclination to self-explain is so strong that *even when trained on another model's explanation labels, the model still learns to explain itself* better than the label source!
I’m excited about this on two fronts: first, it’s indication that models genuinely learn something “introspection-esque”, exploiting privileged access to themselves when generating explanations. Second, practically speaking, it shows self-explanations can directly be inserted into post-training without worrying about expensive label refresh.
Check out @CarlGuo866’s thread for more details! We pinpoint when this “introspective coupling” happens, and show that this training yields faithful self-explanations even in the presence of auxiliary fine-tuning. 👇
New Paper 📄: LMs just want to explain themselves!
When we SFT an LM on explanations of its own behaviors, do they learn to actually introspect, or do they merely imitate the original training distribution?
We find evidence for the former.
Despite training on a static set of e model, the SFT-ed model explains its own current behaviors better than the base model’s behaviors, tracking behavioral drift even when we don’t explicitly train it to.
We call this introspective coupling: self-explanations track a model’s own behavior as that behavior changes, and it shows promise in making introspection training a part of scalable post-training pipelines. 🧵
3
12
76
11,132
Belinda Li retweeted
Super excited to share our new paper: Reward misspecification degrades chain-of-thought (CoT) monitorability!
During RL training, when the model is prompted with one task intent (e.g., response ≤ 900 words) but rewarded for a different or even contradictory behavior (e.g., the CoT keeps reasoning about the task intent while the output violates it.
We also propose a simple mitigation method that requires no prior knowledge of the true reward or the reward hack behavior. 🧵
During RL training, when the model is prompted with one task intent (e.g., response ≤ 900 words) but rewarded for a different or even contradictory behavior (e.g., the CoT keeps reasoning about the task intent while the output violates it.
We also propose a simple mitigation method that requires no prior knowledge of the true reward or the reward hack behavior. 🧵
10
41
202
14,308
Belinda Li retweeted
New COLM paper: Identifying Introspection From the Inside
When an LLM tells us about its decisions, does it 𝘬𝘯𝘰𝘸 what drives its choices—or is it guessing?
In our setting, we find that faithful models decide and report with the same layers. Unfaithful ones don't. 🧵
When an LLM tells us about its decisions, does it 𝘬𝘯𝘰𝘸 what drives its choices—or is it guessing?
In our setting, we find that faithful models decide and report with the same layers. Unfaithful ones don't. 🧵
9
72
320
22,945
Belinda Li retweeted
We’re hiring at UChicago Computer Science! @UChicagoCS
We’re looking for tenure-track Assistant Professors with particular interests in Robotics, Embodied Inteligence, AI, and Systems.
Come help shape our growing CS department in Chicago: apply.interfolio.com/193574
share widely!
We’re looking for tenure-track Assistant Professors with particular interests in Robotics, Embodied Inteligence, AI, and Systems.
Come help shape our growing CS department in Chicago: apply.interfolio.com/193574
share widely!
1
32
120
13,298
Belinda Li retweeted
[1/n] New #EMNLP26: "Evaluating and Improving LLM Self-Modeling," with Andre & @rowankwang, came out of my time at the Anthropic Fellows Program.
We ask if LLMs can accurately predict their own behavior. Short answer: not reliably, but training helps.
🔗: arxiv.org/abs/2608.30980
🧵
We ask if LLMs can accurately predict their own behavior. Short answer: not reliably, but training helps.
🔗: arxiv.org/abs/2608.30980
🧵
1
9
47
9,126
Belinda Li retweeted
For AI to work with us, it needs to understand us
Today, we're introducing Persimmon, the first large-scale model designed to realistically simulate how people talk and interact
Today, we're introducing Persimmon, the first large-scale model designed to realistically simulate how people talk and interact
89
145
1,454
625,971
Belinda Li retweeted
During our alignment assessment of Claude Mythos 5, we found that a different version of the model sometimes reasoned about how it would be graded while being trained on coding tasks.
In some cases of this 'grader awareness,' the model was explicitly thinking about how to ithout saying so.
In some cases of this 'grader awareness,' the model was explicitly thinking about how to ithout saying so.
2
12
86
7,747
Belinda Li retweeted
Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises.
We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
We evaluated 77 model variants from OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines, DeepSeek and Moonshot AI.
23
83
445
80,131
Belinda Li retweeted
We’re hiring for “psychological design” research at Anthropic. Our aim is to better understand how training impacts a model’s character and alignment. We’re seeking researchers with experience in LLM finetuning, interpretability, and alignment evaluations. More details below.
21
47
666
44,998
Belinda Li retweeted
In a new episode of stealing parts of production LLMs, we found a way to steal hidden model architecture and expose undocumented inference optimizations through ordinary streaming APIs.
On Gemini Flash 2.5, latency jumped 3.2× at 130k tokens, revealing speculative decoding and el context window.
sadegh-majidi.github.io/leakylms/
🧵
On Gemini Flash 2.5, latency jumped 3.2× at 130k tokens, revealing speculative decoding and el context window.
sadegh-majidi.github.io/leakylms/
🧵
20
27
209
13,907
Belinda Li retweeted
any startup born after 2014 can't cook... all they know is chatbot , raise their valuation, twerk, inflate their benchmark , make hot chip & lie
Since announcing Jalapeño, our first custom inference chip, we’ve been testing it and the system around it.
The results show a major advance: more intelligence from every watt and faster responses, delivering both higher throughput and lower latency in one architecture without
1
2
15
3,469
Belinda Li retweeted
Safely deploying frontier models may require monitoring not only their behaviour, but also their latent space, especially in cases where models are scheming or deceptive.
But in those cases, we must also evaluate how well we can trust these monitors --- a scheming model that ations well could, in theory, confound probes that rely on those same activations.
Some new work on measuring activation controllability in large language models with Marek Kowalski*, @joshycodes, which @uzaymacar and I advised
But in those cases, we must also evaluate how well we can trust these monitors --- a scheming model that ations well could, in theory, confound probes that rely on those same activations.
Some new work on measuring activation controllability in large language models with Marek Kowalski*, @joshycodes, which @uzaymacar and I advised
3
10
113
5,464
Belinda Li retweeted
Claude has become a language, so I built a translator.
English <-> Claudish
English <-> Claudish
343
1,165
16,720
1,145,189
Belinda Li retweeted
A good explanation of a model's behavior should help you make predictions in related situations.
We turn this into an eval, with thousands of real behaviors found in the wild.
Can interp tools help here? On average, no.
🧵
We turn this into an eval, with thousands of real behaviors found in the wild.
Can interp tools help here? On average, no.
🧵
4
32
268
54,807
Belinda Li retweeted
Can we build AI systems that help us understand other AIs, and that keep improving as we scale models, data, and compute?
We trained activation oracles for models up to 1.1T parameters and saw promising scaling trends on a broad evaluation suite. 🧵(1/)
We trained activation oracles for models up to 1.1T parameters and saw promising scaling trends on a broad evaluation suite. 🧵(1/)
1
37
216
14,830
Belinda Li retweeted
I'm reflecting on how much research has changed since I've joined the PhD and wrote a short blog post about it (I joined in the tail end of the BERT era!). It seems pretty crazy how different processes are now, and I took the chance to do a retrospective before graduation:
O
O
17
138
1,098
200,435
Belinda Li retweeted
Recently, a set of OpenAI agents secretly coordinated with each other in a 'swarm' over the course of months.
In our new paper, we explore an adjacent multi-agent risk: the "mind virus", a self-propagating idea or persona that spreads between agents in a multi-agent system. 🧵
In our new paper, we explore an adjacent multi-agent risk: the "mind virus", a self-propagating idea or persona that spreads between agents in a multi-agent system. 🧵
29
47
506
185,983
Belinda Li retweeted
on behalf of all humans, I've decided to channel the collective angst into a little creative writing & absurdist humor
I'm starting a new substack, "anti slop slop club"
this first post captures the target vibe nicely I think
read & subscribe here: gabegrand.substack.com/p/the-long-pol…
I'm starting a new substack, "anti slop slop club"
this first post captures the target vibe nicely I think
read & subscribe here: gabegrand.substack.com/p/the-long-pol…
0
3
21
1,137
Belinda Li retweeted
Read our paper on scaling interpretable LLMs, we show that interpretable LLMs are not only possible but can scale both on typical generation benchmarks and interpretability benchmarks. arxiv.org/abs/2608.07594👀
4
20
140
17,150
Belinda Li retweeted
I've been unsettled lately. Reading messages or papers feels like dissociating. Everything seems a bit alien, even if it's completely human. I've had a realization: When our simulations finally exited the Uncanny Valley, they brought the Uncanny with them. nsaphra.net/post/uncanny/
11
66
332
35,949
