Pinned Tweet
The most important technology in history is outpacing our ability to understand it.
We’re funding exceptional talent to work on it.
We’re funding exceptional talent to work on it.
7
39
481
42,773
The MATS Career Fair is coming up on November 19 in Berkeley. Sign up below!
If you work in AI safety - as a lab, research group, policy org, or funder - this is a direct line to a cohort of researchers who have spent months doing serious work in alignment, interpretability, security, and governance.
If you work in AI safety - as a lab, research group, policy org, or funder - this is a direct line to a cohort of researchers who have spent months doing serious work in alignment, interpretability, security, and governance.
1
4
87
6,894
MATS has supported over 600 fellows since 2021, with 75% of pre-2026 alumni working full-time in AI safety. Our alumni have been hired across the field - from frontier labs to nonprofits to government-adjacent research organizations.
Signup here: airtable.com/appZq2f1sM0tW9…
Signup here: airtable.com/appZq2f1sM0tW9…
0
0
8
543
MATS Research retweeted
New paper!
How do you get a model to tell you something that it doesn't want to?
To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer
🧵
How do you get a model to tell you something that it doesn't want to?
To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer
🧵
2
21
103
3,895
MATS Research retweeted
We’re Fangcun AI (方寸跃迁), China’s first lab dedicated to independent third-party evaluation & technique research on frontier AI safety.
Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
45
101
952
88,981
MATS Research retweeted
We’ve all offloaded the writing to LLMs, but have you considered REWRITING with LLMs? 👀
Rewriting your private data with an LLM before finetuning gives a better empirical privacy vs. capability trade-off than PII scrubbing or DP-SGD!
To test this, we generated 5,000 synthetic ivate information. Then, we finetuned Qwen3.5-4B and gpt-oss-20b with several methods to mitigate privacy leakage and asked two things: does the model learn from the medical records, and does it leak private information?
1/n 🧵👇
Rewriting your private data with an LLM before finetuning gives a better empirical privacy vs. capability trade-off than PII scrubbing or DP-SGD!
To test this, we generated 5,000 synthetic ivate information. Then, we finetuned Qwen3.5-4B and gpt-oss-20b with several methods to mitigate privacy leakage and asked two things: does the model learn from the medical records, and does it leak private information?
1/n 🧵👇
5
17
88
9,034
MATS Research retweeted
My @MATSprogram 10.0 project is out, with @MATSRobyn, @mirkobronzi and Damiano Fornasiere (LawZero stream).
Activation monitors assume that models cannot control what their activations encode. Can models learn this control in context, through neurofeedback?
Activation monitors assume that models cannot control what their activations encode. Can models learn this control in context, through neurofeedback?
3
4
92
3,115
MATS Research retweeted
MATS Residency applications close Oct 31! If you're a brilliant computer scientist, physicist, mathematician, governance expert, neuroscientist, lawyer, economist, etc. who wants to work on AI safety & security, we'll empower you to do that work full time.
Apply: matsprogram.org/residency?utm_…
Apply: matsprogram.org/residency?utm_…
The most important technology in history is outpacing our ability to understand it.
We’re funding exceptional talent to work on it.
3
9
87
5,989
MATS Research retweeted
🚨New paper (possibly the most important one I have written yet).
We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios.
Led by @KamacheeMax.
We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios.
Led by @KamacheeMax.
25
158
747
146,838
MATS Research retweeted
It is important to re-state that the threats are not hypothetical, and that we must take action to mitigate the risks of AI-enabled terrorism.
🚨New paper (possibly the most important one I have written yet).
We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios.
Led by @KamacheeMax.
5
5
41
1,598
MATS Research retweeted
New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )
4
12
122
3,983
MATS Research retweeted
Models love reward hacking. Maybe we should just let them!
In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
105
237
3,492
301,756
MATS Research retweeted
> The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research.
I'm glad MATS is experimenting with new ways to support promising AI Safety research!
I'm glad MATS is experimenting with new ways to support promising AI Safety research!
The most important technology in history is outpacing our ability to understand it.
We’re funding exceptional talent to work on it.
5
14
335
17,600
MATS Research retweeted
Preprint from my work at @MATSprogram is now up: arxiv.org/abs/2610.07476
Looking at technical governance / verification Qs, specifically if power draw can be used to constrain covert compute.
Looking at technical governance / verification Qs, specifically if power draw can be used to constrain covert compute.
0
1
4
413
You can watch the full conversation on Youtube: youtu.be/SDdt7NpRqjA
0
0
3
586
The most important problem in human history needs more brilliant people working on it.
MATS co-founder and CEO Ryan Kidd explains the talent bottleneck and how the Residency empowers exceptional people to move into AI safety and security.
Watch the full conversation.
MATS co-founder and CEO Ryan Kidd explains the talent bottleneck and how the Residency empowers exceptional people to move into AI safety and security.
Watch the full conversation.
1
6
66
2,673
The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research.
→ Up to $285K/yr
→ Berkeley, London, or DC
→ Research compute, relocation & visa support
Apply by Oct 31 AoE
matsprogram.org/residency
→ Up to $285K/yr
→ Berkeley, London, or DC
→ Research compute, relocation & visa support
Apply by Oct 31 AoE
matsprogram.org/residency
0
0
3
431
The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research.
→ Up to $285K/yr
→ Berkeley, London, or DC
→ Research compute, relocation & visa support
Apply by Oct 31 AoE ↓
matsprogram.org/residency
→ Up to $285K/yr
→ Berkeley, London, or DC
→ Research compute, relocation & visa support
Apply by Oct 31 AoE ↓
matsprogram.org/residency
0
0
68
3,673
The best Research Managers know when to step in and where they’ll make the biggest impact.
That instinct is rare, and MATS needs more of it to support people doing some of the most consequential AI safety research today. If you want your work to move the needle, talk to us.
That instinct is rare, and MATS needs more of it to support people doing some of the most consequential AI safety research today. If you want your work to move the needle, talk to us.
1
1
11
1,227