Formal methods are now on the AI exponential.
We are funding 7 world class teams (and 1 red team) to show that we can bring these capabilities into the real world to help us rapidly secure our digital infrastructure.
Very proud of the quality of talent & work we're funding!
We are funding 7 world class teams (and 1 red team) to show that we can bring these capabilities into the real world to help us rapidly secure our digital infrastructure.
Very proud of the quality of talent & work we're funding!
Could AI help prove software is secure?
We’re backing eight teams to test AI-enabled formal methods in real systems. Plus: a new £60m Robot Locomotion programme, Creator stories and opportunities across ARIA.
Read October’s update: ariaresearch.substack.com/p/proof-pulse-…
2
6
58
2,189
Nora Ammann retweeted
We are out of stealth! As AI writes more code, we're building tools to prove programs meet precise requirements, with Lean proofs anyone can independently check. Let's make verified software the norm in every language and application.
Come take a look: andtruth.com
Come take a look: andtruth.com
4
31
207
22,908
Nora Ammann retweeted
Understanding how governments use AI is important, so how are UK public servants using AI at work today?
To start getting a sense of this, we polled ~200 UK public servants from our network - in collaboration between @GovAIOrg, @B_I_Team, and @apoliticalco.
Results suggest day use AI daily for routine tasks, and that broader adoption might be held back by lack of trust in accuracy and security.
🧵below on key charts. A blog summary of our results can be found on the GovAI website: governance.ai/analysis/how-u…
1) The vast majority (95%) of our respondents report using AI tools for work in the last month, and 6 out of 10 use it at least once per day.
To start getting a sense of this, we polled ~200 UK public servants from our network - in collaboration between @GovAIOrg, @B_I_Team, and @apoliticalco.
Results suggest day use AI daily for routine tasks, and that broader adoption might be held back by lack of trust in accuracy and security.
🧵below on key charts. A blog summary of our results can be found on the GovAI website: governance.ai/analysis/how-u…
1) The vast majority (95%) of our respondents report using AI tools for work in the last month, and 6 out of 10 use it at least once per day.
1
2
7
1,133
Nora Ammann retweeted
Anthropic: “We cannot underscore enough the impact that UK @AISecurityInst has had, and will continue to have, on global AI safety and security.”
1
55
474
23,011
Nora Ammann retweeted
I’m hiring a Chief of Staff to be my right hand across everything I do at Cosmos.
That starts with Cosmos Institute (@cosmos_inst), the nonprofit I founded after selling two AI companies, having two kids, and thinking hard about the good life and the good society. Through the Institute, we do research internally and with fellows from @GoogleDeepMind, @AnthropicAI, @OpenAI, @Microsoft, and academia, have given out 200+ grants to build fast prototypes, and we’re launching a big new in-person bet for higher education in 2027. This side of my work also includes @UniofOxford, where I co-teach a graduate seminar on philosophy and AI and support the Human-Centered AI Lab (HAI Lab), as well as a book I’m writing on human autonomy in the age of AI.
On the venture side, you’ll work with me across Cosmos Ventures (@cosmos_vc), our new $77.6M fund backing founders building new institutions for the AI age. Our early portfolio includes companies like AIUC (@aiunderwriting), @PrimeIntellect, and Workshop Labs (acq. by @thinkymachines), with several more to be announced soon. We’re also going to be incubating new companies in 2027.
You’ll come in, do this well for two years, and if you want to build a thesis-aligned company or a nonprofit next, we will back you, or place you at a top company in our community. The role is based in Austin, with relocation provided.
To apply, read our essays and tell us where we’re wrong: docs.google.com/document/d/18_…
That starts with Cosmos Institute (@cosmos_inst), the nonprofit I founded after selling two AI companies, having two kids, and thinking hard about the good life and the good society. Through the Institute, we do research internally and with fellows from @GoogleDeepMind, @AnthropicAI, @OpenAI, @Microsoft, and academia, have given out 200+ grants to build fast prototypes, and we’re launching a big new in-person bet for higher education in 2027. This side of my work also includes @UniofOxford, where I co-teach a graduate seminar on philosophy and AI and support the Human-Centered AI Lab (HAI Lab), as well as a book I’m writing on human autonomy in the age of AI.
On the venture side, you’ll work with me across Cosmos Ventures (@cosmos_vc), our new $77.6M fund backing founders building new institutions for the AI age. Our early portfolio includes companies like AIUC (@aiunderwriting), @PrimeIntellect, and Workshop Labs (acq. by @thinkymachines), with several more to be announced soon. We’re also going to be incubating new companies in 2027.
You’ll come in, do this well for two years, and if you want to build a thesis-aligned company or a nonprofit next, we will back you, or place you at a top company in our community. The role is based in Austin, with relocation provided.
To apply, read our essays and tell us where we’re wrong: docs.google.com/document/d/18_…
18
56
472
62,518
Nora Ammann retweeted
We’re looking for the next generation of UK FRO founders, and bringing the conversation to Scotland this week!
5
2
9
1,418
Cool! Hopefully soon we'll learn more about the details (e.g. what formal model of C), and that the proofs become independently reproducible....
But very excited about Harmonic's direction of travel!
But very excited about Harmonic's direction of travel!
We’re bringing Aristotle to code.
AI is writing more code than anyone can review. We're building the AI that verifies it.
Starting today with C and Verilog.
0
1
9
630
Super cool, and big kudos for making this happen @maxvonhippel & co!
We just sent out the last batch of invites for Vibecheck (fmxai.org/vibecheck). We're now at capacity (for the building, for two floors). The event is going to slap and I'm really, really excited about it!
- HUGE thanks to our sponsors, including @datadoghq , @theoremlabs SafeWithAtlas, @astriolabs, For-All R&D (@qd_forall), and @thegp. This would be totally impossible without you!
- I'm also really stoked about several companies that are providing compute and platform tokens for competitors, including: @anthropicai, @PrimeIntellect, party.build, @lanyon_ai , @wearerandomlabs .
- And finally, we have some amazing experts who've helped plan the event and/or will be in attendance helping to run the event and judge entries, including folks from several of the above-mentioned companies, and from several academic labs, as well as from @mathematics_inc , @axiommathai , and @HarmonicMath .
I'll be sending rejection emails for anyone who didn't make it in shortly, and will close down the application form. If any currently RSVPd people cancel, we will send some last-minute invites accordingly. If you wanted to participate in this event but didn't make it in, please note two things: first, this does not reflect poorly on you -- we specifically aimed to accept people from a wide variety of backgrounds, not just a bunch of our friends or of FM experts. And second, we are going to run more of these events! We are loosely planning to run a London event, colocated with FMxAI London in 2027 (and co-organized with @galois, see fmxai.org for more ..); a NYC event, at some TBD date; potentially also an online/remote one; and a defense-oriented event co-organized with @mathematics_inc. If you're interested in one of these let me know.
This event is absolutely a labor of love, and we hope you'll follow along. We'll be posting updates, repos, podcast episodes, video interviews, and more throughout, and it's going to be a lot of fun. Also, if you didn't make it in but want to experiment in your free time, we will be posting all the rules and judging criteria in the coming weeks on fmxai.org/vibecheck, and we encourage you to play around and post your work!
And for those of you who got in, see you in SF!!
1
0
5
394
Nora Ammann retweeted
Huge. British Govt announces:
• £165m: National AI Biosecurity Initiative: Includes £100m for a new AI Biosecurity Lab, using AI to detect emerging biological threats.
• £120m: AI Security Institute (AISI): Additional funding to strengthen AI risk research and evaluation.
• £70m: National Exposure and Real-Time Vulnerability (NERV) Centre: Identifying vulnerabilities in critical infrastructure.
• £60m: National Cyber Shield: Strengthening Britain’s cyber defences.
• £60m: UK Sovereign AI: Expanding investment into physical and industrial AI.
• £60m: R&D Procurement Scheme: Supporting companies developing AI security and NHS innovations.
• £30m: Mayoral AI Partnerships: Helping local businesses adopt AI, create jobs and improve public services.
• £23m : AI Economics Institute: Research into AI’s effects on productivity, employment and living standards.
• £18m: UK Advanced Chip Design Academy: Developing domestic semiconductor design expertise and reducing overseas dependencies.
• £165m: National AI Biosecurity Initiative: Includes £100m for a new AI Biosecurity Lab, using AI to detect emerging biological threats.
• £120m: AI Security Institute (AISI): Additional funding to strengthen AI risk research and evaluation.
• £70m: National Exposure and Real-Time Vulnerability (NERV) Centre: Identifying vulnerabilities in critical infrastructure.
• £60m: National Cyber Shield: Strengthening Britain’s cyber defences.
• £60m: UK Sovereign AI: Expanding investment into physical and industrial AI.
• £60m: R&D Procurement Scheme: Supporting companies developing AI security and NHS innovations.
• £30m: Mayoral AI Partnerships: Helping local businesses adopt AI, create jobs and improve public services.
• £23m : AI Economics Institute: Research into AI’s effects on productivity, employment and living standards.
• £18m: UK Advanced Chip Design Academy: Developing domestic semiconductor design expertise and reducing overseas dependencies.
1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? 🇬🇧
Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.
18
41
279
30,384
Nora Ammann retweeted
I interviewed @tsmilton_amodo about the tech we'd need to verify a potential US-China deal on AI.
We discuss Amodo's prototype for inference verification, as well as the myriad ways you could use power monitoring, zero knowledge proofs and/or unilateral surveillance to sary is using compute.
Though solving some kind of verification is central to many people's theories of victory, there are precious few people working on it currently. Tom is one such person!
Listen to the episode on Youtube, Spotify or Apple Podcasts.
You can also find more about Tom and Amodo's work at
ai-verification.com
00:00:00 Using compute verification for an AI slowdown
00:06:08 Proving a datacentre is only running inference
00:13:22 Reading a GPU's workload from its power draw
00:23:35 What would it take to get verification deal-ready?
00:37:49 How realistic is verification?
00:46:07 Preparing for a rushed US-China deal
01:07:20 How good is AI at hardware design?
We discuss Amodo's prototype for inference verification, as well as the myriad ways you could use power monitoring, zero knowledge proofs and/or unilateral surveillance to sary is using compute.
Though solving some kind of verification is central to many people's theories of victory, there are precious few people working on it currently. Tom is one such person!
Listen to the episode on Youtube, Spotify or Apple Podcasts.
You can also find more about Tom and Amodo's work at
ai-verification.com
00:00:00 Using compute verification for an AI slowdown
00:06:08 Proving a datacentre is only running inference
00:13:22 Reading a GPU's workload from its power draw
00:23:35 What would it take to get verification deal-ready?
00:37:49 How realistic is verification?
00:46:07 Preparing for a rushed US-China deal
01:07:20 How good is AI at hardware design?
2
10
58
5,347
Nora Ammann retweeted
Making sure I state it clearly:
We can literally patch and prove bug-freeness of the entire software stack running in production. With AI that exists right now for ~$100m.
This means no more software-level hacks. I know it sounds too good to be true, but it isn't. AMA.
We can literally patch and prove bug-freeness of the entire software stack running in production. With AI that exists right now for ~$100m.
This means no more software-level hacks. I know it sounds too good to be true, but it isn't. AMA.
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here!
Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
50
52
822
91,658
Nora Ammann retweeted
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here!
Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
38
118
1,066
134,666
Nora Ammann retweeted
How to find and fix ≈all exploitable bugs:
1. State good-enough theorems about production binaries
2. Prove them
theorem.dev/oss-verify
1. State good-enough theorems about production binaries
2. Prove them
theorem.dev/oss-verify
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here!
Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
3
15
116
12,425
Nora Ammann retweeted
@maxvonhippel @rajashree (not so) secretly, I don't want a world where we stop at assembly, I want verification down to Maxwell's equations. Assembly is just a nice nearly-universal IR on the way there.
5
15
106
4,339
Nora Ammann retweeted
Some stats on #php2shell. In the 10 hours before it was publicly announced, across the @Cloudflare network we saw thousands of attempts to exploit it. That jumped to tens of thousands in the 45 minutes post announcement, hundreds of thousands within 2 hours, and millions now.
288
3,269
22,651
2,273,164
Nora Ammann retweeted
The NCSC is encouraging UK organisations to prepare for immediate updating of systems using PHP. Where immediate remediation is not feasible, organisations should consider taking the system offline until it can be fully mitigated.
For more⬇️
ncsc.gov.uk/news/critical-…
For more⬇️
ncsc.gov.uk/news/critical-…
81
902
3,443
1,038,537
Nora Ammann retweeted
Excited to announce that I’m moving to Beijing and starting a professorship in AI safety at Tsinghua University’s College of AI!
Here I explain why: sorenmindermann.substack.com/p/why-im-becom…
Here I explain why: sorenmindermann.substack.com/p/why-im-becom…
167
76
1,648
161,460
Nora Ammann retweeted
🚨 Wow. WOW. I just witnessed the most involved and sophisticated phishing attempt I've ever seen and y'all need to know about this.
The TL;DR is that @BeaconLayer reached out invite me onto their show, spent days going over the run-of-show in detail with clearly AI-assisted my work, booked the recording, and then tried to get me to install malware in order to join the call.
Some of you have probably already heard about the similar attempt made on @0xzak but this one was news to me. The difference is that this isn't just someone posing as a producer from a popular show, it's the actual show, which has episodes out and a sizeable audience on @X which makes them look legit. I suspect all of those other guests have malware installed already and I caution everyone to do a security sweep.
I've been cautioning people against deepfake scams for a decade so you'd think I'd have smelled something sooner — it takes real balls to go after cypherpunks specifically — but at least I caught it before they convinced another rube to commit a bash command and give up all of their passwords.
Here's the full story — I have the receipts, and now you do too:
1/13
The TL;DR is that @BeaconLayer reached out invite me onto their show, spent days going over the run-of-show in detail with clearly AI-assisted my work, booked the recording, and then tried to get me to install malware in order to join the call.
Some of you have probably already heard about the similar attempt made on @0xzak but this one was news to me. The difference is that this isn't just someone posing as a producer from a popular show, it's the actual show, which has episodes out and a sizeable audience on @X which makes them look legit. I suspect all of those other guests have malware installed already and I caution everyone to do a security sweep.
I've been cautioning people against deepfake scams for a decade so you'd think I'd have smelled something sooner — it takes real balls to go after cypherpunks specifically — but at least I caught it before they convinced another rube to commit a bash command and give up all of their passwords.
Here's the full story — I have the receipts, and now you do too:
1/13
4
12
68
12,197
Nora Ammann retweeted
A horrific Russian strike in the Kharkiv region. A “shahed” struck people directly – in a park near an ordinary café. Pure atrocity. Six people were killed, including children. A little girl was killed in a stroller – her name was Angelina, she was just over a year old, born in ’s name was Inna. She died in the hospital… Another girl who was killed was 11 years old. My condolences to their families and loved ones. Another 14 people were injured, including six more children.
I am grateful to everyone who supports Ukraine under these circumstances and helps us shoot down “shaheds,” shoot down missiles, and stop Russian assaults. These are horrific photos, but behind them are shattered lives and the truth about this war. Lives must be protected. Ukraine needs air defense. You can't think about making money with Russia as long as it continues its asinine war. We ask that sanctions on Russia stay in place as long as the killings continue. We ask America to be strong, and we ask it to apply the Lindsey Graham law. We ask Europe to continue its support. We ask that everything possible be done to bring peace and guarantee security. Ukraine wants peace. It is Russia that has insisted on continuing the killing all this time. The world has enough strength to stop the war. That strength must finally be used – against the murderers.
I am grateful to everyone who supports Ukraine under these circumstances and helps us shoot down “shaheds,” shoot down missiles, and stop Russian assaults. These are horrific photos, but behind them are shattered lives and the truth about this war. Lives must be protected. Ukraine needs air defense. You can't think about making money with Russia as long as it continues its asinine war. We ask that sanctions on Russia stay in place as long as the killings continue. We ask America to be strong, and we ask it to apply the Lindsey Graham law. We ask Europe to continue its support. We ask that everything possible be done to bring peace and guarantee security. Ukraine wants peace. It is Russia that has insisted on continuing the killing all this time. The world has enough strength to stop the war. That strength must finally be used – against the murderers.
5,690
28,433
94,202
4,432,141