New Free Systems research: can AI audit AI? We find that AI auditors are pretty good at spotting bad behavior in agent logs, but you have to pay a serious cost in false positives if you want to detect most of it.
We're rapidly arriving in an era where only AI can oversee RSI, with models training new models in ways we can't totally observe, we'll need to rely more and more on AI to pore over the vast quantities of information the models are producing so that we can shape model behavior effectively and avoid bad outcomes.
As the @METR_Evals report on the HuggingFace incident put it: "The sheer scale of data (over a thousand transcripts, each of which was extremely long) meant that we had to heavily delegate our analysis to often-unreliable AI agents."
We'll need a whole science of how to do this well -- how to create AI auditors who are reliable, accurate, and unbiased.
So we decided to do a first test. Building from previous research at METR and a project called R-Judge, we gave a range of AI models transcripts that experts have hand-coded for bad behavior. Then we saw how the AI models did.
A few takeaways:
--AI models are solid at spotting reward hacking and sandbagging in transcripts, finding over 80% of instances on average when false positives are held at < 5%
--Maybe surprisingly, more powerful models don't seem to be consistently better at this task than less powerful model
--False positives are a significant challenge. To catch 99% of unsafe behavior incidents, you have to accept an average rate of 15% for false positives -- meaning you will have a bunch of incorrect flagged cases to deal with. Depending on how you triage and what your threat model is, this may be OK, or it may make the whole enterprise prohibitively expensive.
This is just a small first step in a new research agenda to understand and create effective AI auditors. Lots more to come!
This is Free Systems work, unrelated to Anthropic, led by @danmthomp in collaboration with @fouirnaies and Sandy Handan-Nader.
Full post here: freesystems.substack.com/p/were-using-a…
We're rapidly arriving in an era where only AI can oversee RSI, with models training new models in ways we can't totally observe, we'll need to rely more and more on AI to pore over the vast quantities of information the models are producing so that we can shape model behavior effectively and avoid bad outcomes.
As the @METR_Evals report on the HuggingFace incident put it: "The sheer scale of data (over a thousand transcripts, each of which was extremely long) meant that we had to heavily delegate our analysis to often-unreliable AI agents."
We'll need a whole science of how to do this well -- how to create AI auditors who are reliable, accurate, and unbiased.
So we decided to do a first test. Building from previous research at METR and a project called R-Judge, we gave a range of AI models transcripts that experts have hand-coded for bad behavior. Then we saw how the AI models did.
A few takeaways:
--AI models are solid at spotting reward hacking and sandbagging in transcripts, finding over 80% of instances on average when false positives are held at < 5%
--Maybe surprisingly, more powerful models don't seem to be consistently better at this task than less powerful model
--False positives are a significant challenge. To catch 99% of unsafe behavior incidents, you have to accept an average rate of 15% for false positives -- meaning you will have a bunch of incorrect flagged cases to deal with. Depending on how you triage and what your threat model is, this may be OK, or it may make the whole enterprise prohibitively expensive.
This is just a small first step in a new research agenda to understand and create effective AI auditors. Lots more to come!
This is Free Systems work, unrelated to Anthropic, led by @danmthomp in collaboration with @fouirnaies and Sandy Handan-Nader.
Full post here: freesystems.substack.com/p/were-using-a…
How do you review petabytes of agent activity logs? That’s a lot of logs.
10
10
86
4,770
@ahall_research I've done a bunch of work on this. AI auditors are also susceptible to getting pushed around by the AI itself, directly and indirectly (strangeloopcanon.com/p/who-audits-t…).
And identifying problems requires heterogeneous approaches, we have some new non-llm tools which helps substantially.
And identifying problems requires heterogeneous approaches, we have some new non-llm tools which helps substantially.
2
1
13
817
@krishnanrohit @ahall_research This is great! Thanks for sharing!
0
0
1
29
@ahall_research is the 15% per transcript? because if 1 in 100 logs is actually bad, that's roughly 148 false flags next to ~10 real ones per 1000 logs, so a human still reviews ~16% of everything (and only ~6% of flags are real). what base rate was the test set built on
1
0
1
77
@Chahatusharma @ahall_research That’s right. The issue is that the auditors occasionally classify real misbehavior as very likely to be benign, so you have to dramatically lower the threshold at which we count a log as misbehavior if we want to catch 99% of the real cases.
0
0
0
17
Today we're sharing some initial data on how Americans use Claude for politics, based on aggregated conversation data analyzed with differentially private methods. A couple interesting takeaways:
*Almost no one asks AI how they should vote*
About ~5.75% of conversations in the f political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.
*Political conversations have many goals*
These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.
We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.
This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.
Full post is here: anthropic.com/news/2026-us-m…
*Almost no one asks AI how they should vote*
About ~5.75% of conversations in the f political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.
*Political conversations have many goals*
These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.
We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.
This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.
Full post is here: anthropic.com/news/2026-us-m…
21
55
298
34,218
@ahall_research Can y'all measure the share of high-engagement users who ever ask for a political recommendation? Could it be that many high-engagement users ask for election recommendations, but that one-time request is swamped by frequent requests for political info or help with gov class?
2
0
2
227
@ahall_research Reply got too long, but I wanted to also say that this is a super cool project, and I'm looking forward to seeing more work like this from all of the labs!
0
0
2
102
Dan Thompson retweeted
Today we're sharing some initial data on how Americans use Claude for politics, based on aggregated conversation data analyzed with differentially private methods. A couple interesting takeaways:
*Almost no one asks AI how they should vote*
About ~5.75% of conversations in the f political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.
*Political conversations have many goals*
These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.
We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.
This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.
Full post is here: anthropic.com/news/2026-us-m…
*Almost no one asks AI how they should vote*
About ~5.75% of conversations in the f political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.
*Political conversations have many goals*
These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.
We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.
This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.
Full post is here: anthropic.com/news/2026-us-m…
21
55
298
34,218
Dan Thompson retweeted
Political scientists and economists: we're looking for researchers with deep expertise to come work with us for a unique 4-month fellowship at @AnthropicAI. You'll work closely with us to evaluate, improve, and apply Claude's capabilities in your field. It's going to be extremely k, and definitely worth taking a short leave from your university for if you need to!
More info here:
job-boards.greenhouse.io/anthropic/jobs…
More info here:
job-boards.greenhouse.io/anthropic/jobs…
49
139
922
364,532
Why has no one done the massive AI-assisted "how much do negative weights really matter for difference-in-difference" review paper yet?
12
2
168
96,187
@instrumenthull @xuyiqing has a version of this in the APSR for political science papers yiqingxu.org/papers/english…
0
0
1
222
We wanted to check if open-weight models had a similar herding behavior, and it seems Qwen is more resilient to herd mentality than Haiku.
In this experiment, Qwen3.8-27B doesn't fall into the initial trap and corrects course.
In this experiment, Qwen3.8-27B doesn't fall into the initial trap and corrects course.
Follow-up to our @freesystemslab experiment this week: Rewarding AI agents for the swarm’s performance rather than their own makes the swarm less likely to converge on a false belief. But even with the right incentives, agents don’t always figure out how to avoid getting it wrong
1
0
8
261
Thank you @danmthomp and @ahall_research for publishing these results!
We're currently running other experiments on swarm behavior, so expect more results soon.
We're currently running other experiments on swarm behavior, so expect more results soon.
2
0
3
76
@cmpatino_ @ahall_research This is great! Does instructing each agent to report their private signal get the last few agents in the sequence to land on the right answer?
1
0
0
30
Whoa, the economist is running ads with a plot from
@vavreck, @RyanBaxterKing, @RyanDEnos, and Jake Brown
@vavreck, @RyanBaxterKing, @RyanDEnos, and Jake Brown
The mental health of conservatives suffers when they live in progressive places
0
1
1
242
Follow-up to our @freesystemslab experiment this week: Rewarding AI agents for the swarm’s performance rather than their own makes the swarm less likely to converge on a false belief. But even with the right incentives, agents don’t always figure out how to avoid getting it wrong
1
2
19
3,602
@freesystemslab Ultimately, we’re finding that describing the right incentives can help reduce cascades toward false beliefs, but the models we're using are not yet smart enough to figure out how to avoid these cascades without explicit instruction.
1
0
0
53
@freesystemslab This is all joint work with @fouirnaies, @ahall_research, and Sandy Handan-Nader. More to come!
0
0
1
56
Dan Thompson retweeted
My view is that "alignment" is the wrong framing for safety with agentic AI. I think it is often conceptually confused and basically always flattens the conversation in an unhelpful way.
Some non-expert thoughts from a social scientific point of view...
1/
Some non-expert thoughts from a social scientific point of view...
1/
4
22
74
9,546
Dan Thompson retweeted
Latest Free Systems research: how are AI models making voting recommendations in Brazil?
@leticiaauriemo and Luca Moreno Louzado ran a huge study to find out! Some highlights...
--Top frontier models are quite stubborn at refusing to give direct voting reccos, consistent with a dding reccos; other models are less stubborn
--As you provide more info to AI about yourself and what you care about, it's easier to get reccos even when you shouldn't
--These recommendations don't necessarily reflect the distribution of political preferences in society (not clear that they should, but it's an interesting benchmark). In Brazil, reccos for a less popular businessman candidate (Zema) occur at an unusually high rate.
More generally, I think these results raise a lot of profound question about what we want out of AI for politics. Should we forbid AI from making direct recommendations, like Brazil has?
How do you even define what a recommendation is? It's not clear, and in this study, the authors found AI models providing advice that arguably falls short of a direct recommendation but toes the line.
What would optimal political advice look like? And how would we know it if we saw it?
Incredibly excited to see this research, and to see Free Systems continue to scale beyond me and my work, and beyond the borders of the US! Check out the full piece here:
freesystems.substack.com/p/who-would-ai…
@leticiaauriemo and Luca Moreno Louzado ran a huge study to find out! Some highlights...
--Top frontier models are quite stubborn at refusing to give direct voting reccos, consistent with a dding reccos; other models are less stubborn
--As you provide more info to AI about yourself and what you care about, it's easier to get reccos even when you shouldn't
--These recommendations don't necessarily reflect the distribution of political preferences in society (not clear that they should, but it's an interesting benchmark). In Brazil, reccos for a less popular businessman candidate (Zema) occur at an unusually high rate.
More generally, I think these results raise a lot of profound question about what we want out of AI for politics. Should we forbid AI from making direct recommendations, like Brazil has?
How do you even define what a recommendation is? It's not clear, and in this study, the authors found AI models providing advice that arguably falls short of a direct recommendation but toes the line.
What would optimal political advice look like? And how would we know it if we saw it?
Incredibly excited to see this research, and to see Free Systems continue to scale beyond me and my work, and beyond the borders of the US! Check out the full piece here:
freesystems.substack.com/p/who-would-ai…
3
11
60
13,476