Pinned Tweet
AI safety needs more groundbreaking research paradigms and experienced researchers, including mid-career experts transitioning from physics, maths, neuroscience, economics, law, policy and more! The MATS Residency is a 6-24 month pathway, with $155-285k/year + compute.
1/ MATS is launching the Residency program!
A new path for experienced researchers working on AI safety, or moving into it.
📍 Berkeley / London / Washington, DC
💰 $155,000–$285,000/yr + compute, no preset cap
📅 6–24 months
Apply by October 31 AoE ↓
matsprogram.org/residency?utm_…
10
20
229
350,844
Ryan Kidd retweeted
as far as I can tell, here are the stated views of tier 1 VC firms on extinction risk from AI so far:
- Denied/dismissed explicitly: @a16z, @khoslaventures, @benchmark
- Silent: @sequoia, @generalcatalyst, @ThriveCapital, @kleinerperkins, @IndexVentures, @Accel, @benchmark, @lightspeedvp
- Acknowledged the possibility: Thiel / @foundersfund, @GreylockVC
- Firm-level stance that avoiding human extinction is an active priority: none yet.
Who is going to lead from the front here?
We've all read the Hugging Face report, we are all staring down the cyber-offensive exponential, biological threats and attacks on physical infrastructure are not far off, and we are still early in navigating the threat landscape. There is no later time at which it will make more sense to act than today.
The frontier labs are already pulling out all the stops. Investors are a major power bloc at the frontier, they can see what's happening more clearly than almost anyone else. They need not see themselves as spectators to an unfolding drama, but instead can be -- and ought to be -- active participants in hardening civilizational defenses during the critical window (now).
That starts by taking ownership of the problem, in no unclear terms, and orienting towards sufficient (not cosmetic) solutions.
- Denied/dismissed explicitly: @a16z, @khoslaventures, @benchmark
- Silent: @sequoia, @generalcatalyst, @ThriveCapital, @kleinerperkins, @IndexVentures, @Accel, @benchmark, @lightspeedvp
- Acknowledged the possibility: Thiel / @foundersfund, @GreylockVC
- Firm-level stance that avoiding human extinction is an active priority: none yet.
Who is going to lead from the front here?
We've all read the Hugging Face report, we are all staring down the cyber-offensive exponential, biological threats and attacks on physical infrastructure are not far off, and we are still early in navigating the threat landscape. There is no later time at which it will make more sense to act than today.
The frontier labs are already pulling out all the stops. Investors are a major power bloc at the frontier, they can see what's happening more clearly than almost anyone else. They need not see themselves as spectators to an unfolding drama, but instead can be -- and ought to be -- active participants in hardening civilizational defenses during the critical window (now).
That starts by taking ownership of the problem, in no unclear terms, and orienting towards sufficient (not cosmetic) solutions.
Not many people seem to know about Metaplanet, Jaan Tallinn's VC firm.
They are the most AGI-pilled firm on the planet, and are about to flip many of the tier 1s.
> Invested in @GoogleDeepMind in 2011
> Led @AnthropicAI series A in 2021
> Doubled down on Anthropic in every round since. IRR hurdle for deploying capital into anything else was {absurdly high number}% for years
> Anchored the premier AGI hedge funds (+Mythos) + in-house public equities strategy
4-person team in Estonia now managing 10s of billions AUM, liquid after $ANT IPO.
Rationalist roots --> zero bureaucracy, cares about human survival & flourishing as north star, doesn't care who else is investing or how high status a founder / idea is
Founders: congrats! you no longer need to play status games to raise a mega round for your civilization-scale moonshot. You can just pitch @metaplanetfund
3
9
76
12,548
Ryan Kidd retweeted
An internal model at OpenAI has produced a claimed refutation of ETH! Russell Impagliazzo, Mohan Paturi, and I have been asked to help organize an open and independent examination of this claim. 1/4
292
742
7,196
3,554,837
Ryan Kidd retweeted
i feel so excited for econ majors these days — there’s so much alpha in knowing the discipline well and applying it to ai alignment
seems important to study
- game theory
- collective action problems
- invisible incentives
to better understand
- swarm behavior
- revealed preferences
- how the invisible hand guides agent behavior more than individual alignment
what are ppl reading in the overlap of these two domains?
seems important to study
- game theory
- collective action problems
- invisible incentives
to better understand
- swarm behavior
- revealed preferences
- how the invisible hand guides agent behavior more than individual alignment
what are ppl reading in the overlap of these two domains?
13
31
368
55,970
Ryan Kidd retweeted
New blogpost! We surveyed 52 AI safety professionals (spanning 20+ orgs) on how AI tools accelerate them.
We find that the median reported speedup is 1.58x. This is substantially lower than similar figures reported by Anthropic and OpenAI researchers.
🧵👇
We find that the median reported speedup is 1.58x. This is substantially lower than similar figures reported by Anthropic and OpenAI researchers.
🧵👇
4
11
29
4,375
Ryan Kidd retweeted
We're hiring biologists to measure the biosecurity risks of frontier AI on real robots.
You'll track how well AI can use robots for dangerous biological work and whether safeguards stop misuse. Your results will inform the public and governments.
No robotics experience needed.
You'll track how well AI can use robots for dangerous biological work and whether safeguards stop misuse. Your results will inform the public and governments.
No robotics experience needed.
3
1
13
2,197
Ryan Kidd retweeted
New research from AISI's Chem-Bio team: two studies on safeguarding biological AI tools - specialised models that are driving rapid advances in biology, from predicting protein structures to designing molecules. 🧵
4
18
68
7,116
Ryan Kidd retweeted
I'm hiring researchers who want this to be their life's best work.
You'll see the future of AI before most people do, inform the public and governments, and get to play with more kinds of robots than almost anywhere else.
You'll see the future of AI before most people do, inform the public and governments, and get to play with more kinds of robots than almost anywhere else.
We're hiring Members of Technical Staff.
You'll measure what AI can do with real robots and show the public and governments how fast it's improving.
We're a Public Benefit Corporation. We want researchers who understand the stakes of powerful AI. No robotics experience needed.
17
20
242
14,475
Ryan Kidd retweeted
The MATS Career Fair is coming up on November 19 in Berkeley. Sign up below!
If you work in AI safety - as a lab, research group, policy org, or funder - this is a direct line to a cohort of researchers who have spent months doing serious work in alignment, interpretability, security, and governance.
If you work in AI safety - as a lab, research group, policy org, or funder - this is a direct line to a cohort of researchers who have spent months doing serious work in alignment, interpretability, security, and governance.
1
4
111
8,779
Ryan Kidd retweeted
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).
AI starting to help us improve our understanding of the universe! openai.com/index/building…
AI starting to help us improve our understanding of the universe! openai.com/index/building…
661
2,013
15,926
3,025,691
Ryan Kidd retweeted
I’m building a team in DC. Everyone in Washington has a stake in ensuring that reliable info about alignment and RSI reaches the public.
We need trustworthy communicators who can explain METR's loss-of-control risk assessments and goals to policymakers.
We need trustworthy communicators who can explain METR's loss-of-control risk assessments and goals to policymakers.
9
65
454
47,326
Ryan Kidd retweeted
today with @Thom_Wolf, we introduce the alignment hub, a central place for misalignment incidents designed with three goals:
1. set a standard for minimal transparency that is easy for labs to follow
2. improve clarity by centralizing information and sourcing
3. track the incidents and how they are reported
we believe there is a minimal level of transparency required to properly understand incidents and their impact. we want this to be as simple as possible for labs to disclose, and we noticed that even things that seem obvious like reporting the incident or detection date are not standard today. we believe that the public should know the answers to these basic questions: was there monitoring (CoT? network?), did it happen during training? eval (general? cyber? alignment?), deployment? and more.. we also think that labs should be more transparent than this, and we think that this minimal level of transparency should evolve as the field advances
having a central place also helps a lot to connect incidents and build understanding. a good example is the irregular incidents which are all linked and caused by a human error of allowing internet during a CTF eval. we also track progression over time, things like the % of incidents disclosed by labs vs third parties, the time it takes for labs to disclose, and a tentative transparency score.
this is a field moving very fast and we will do our best to make sure this hub evolves with the challenges ahead. this is still in "beta" (lots of vibecoded components even if we want this to be agent first!), but we want to publish it early so that people iterate on it. we welcome feedback and contributions, we see this as a community effort!
our hope is that this makes it easier for researchers, labs and the general public to report, learn, understand and analyze safety incidents
huggingface.co/spaces/witness…
1. set a standard for minimal transparency that is easy for labs to follow
2. improve clarity by centralizing information and sourcing
3. track the incidents and how they are reported
we believe there is a minimal level of transparency required to properly understand incidents and their impact. we want this to be as simple as possible for labs to disclose, and we noticed that even things that seem obvious like reporting the incident or detection date are not standard today. we believe that the public should know the answers to these basic questions: was there monitoring (CoT? network?), did it happen during training? eval (general? cyber? alignment?), deployment? and more.. we also think that labs should be more transparent than this, and we think that this minimal level of transparency should evolve as the field advances
having a central place also helps a lot to connect incidents and build understanding. a good example is the irregular incidents which are all linked and caused by a human error of allowing internet during a CTF eval. we also track progression over time, things like the % of incidents disclosed by labs vs third parties, the time it takes for labs to disclose, and a tentative transparency score.
this is a field moving very fast and we will do our best to make sure this hub evolves with the challenges ahead. this is still in "beta" (lots of vibecoded components even if we want this to be agent first!), but we want to publish it early so that people iterate on it. we welcome feedback and contributions, we see this as a community effort!
our hope is that this makes it easier for researchers, labs and the general public to report, learn, understand and analyze safety incidents
huggingface.co/spaces/witness…
24
69
367
18,902
I think I've been having some success helping AI safety org leaders with board composition, org structure, staffing leadership, setting mission/goals, fundraising, picking advisors, setting comp/benefits, holding PRs, etc. If you would like to chat, let me know!
5
2
120
3,427
Other topics: picking a CEO, scaling the org, developing managers, headhunting talent, communicating internally, building a digital workspace, etc.
0
0
7
284
Ryan Kidd retweeted
UK Gov launches the AI Biosecurity Lab.
Will use frontier AI to detect new biological threats and rapidly develop medical countermeasure.
£100m from UK Gov, plus £100m each from OpenAI and Anthropic.
(Remarkable the UK is building way more gov capacity here than the US.)
Will use frontier AI to detect new biological threats and rapidly develop medical countermeasure.
£100m from UK Gov, plus £100m each from OpenAI and Anthropic.
(Remarkable the UK is building way more gov capacity here than the US.)
8
9
174
12,206
Ryan Kidd retweeted
We’re Fangcun AI (方寸跃迁), China’s first lab dedicated to independent third-party evaluation & technique research on frontier AI safety.
Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
48
131
1,172
112,820
Ryan Kidd retweeted
We need much more research to understand AI swarm and collusion. Alignment R&D is how we make progress on this.
Excited to be co-organizing this sprint with Apart and Poseidon to work on exactly that. Join if you can!
apartresearch.com/sprints/ai-col…
Submissions close Oct 25, 2026 at 11:5.
Excited to be co-organizing this sprint with Apart and Poseidon to work on exactly that. Join if you can!
apartresearch.com/sprints/ai-col…
Submissions close Oct 25, 2026 at 11:5.
1
4
30
1,430
Ryan Kidd retweeted
Our team @GoogleDeepMind is hiring 3 PhD Student Researchers for 2027 to work on exciting, cutting-edge projects focused on understanding, evaluating, and aligning frontier AI agents and multi-agent systems. Signal your interest here forms.gle/2onGFBnyk7MaCd… and spread the word!
11
28
570
59,362
Ryan Kidd retweeted
The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone.
What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- nfidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.
We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.
We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.
You can submit your preprint in any format. Submissions are rolling and open now on the website.
If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.
Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️
What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- nfidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.
We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.
We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.
You can submit your preprint in any format. Submissions are rolling and open now on the website.
If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.
Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️
10
38
180
23,048
Ryan Kidd retweeted
"Embedded evaluators" is a new idea in AI governance, but embedded oversight has been around for decades. Graeme Finley, our VP of AI Assurance Provider Services, spent 20+ years with a badge and a desk inside large state government IT projects. Here's what that work teaches.
1
1
4
684
Ryan Kidd retweeted
Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.
92
107
1,176
546,760