Pinned Tweet
Contrary to bad crypto slogans, we should not reduce the degree of trust in the world. I want a high trust society. Instead, reduce trust *needed* for a given level of cooperation; increase the cooperation achievable at given level of trust. More cooperation. Less vulnerability.
22
89
330
Mark S. Miller retweeted
I haven't posted in a while so here are 20 takes about AI, in no particular order.

1. It's very telling that the default question in tech and journalism is "how worried are you about X" and fighting over what to worry about. There's a deeply anxious cultural backdrop in Western a that deeply affects how any technological development is interpreted; see for example news.gallup.com/poll/714593/op…. "Nervousness as seriousness" does seem to be a larger problem in tech/broader discourse at the moment. I think you can care about risks/externalities while remaining pretty optimistic about technology.

2. I'm tired of people who in private pretend to be uncertain or only care about tail risk but then go on media campaigns making all sorts of absolute claims. I think trying to stoke flames and get everyone to freak out is deeply unhelpful, and will come with costs. I also continue to think p(dooms) are bad, vibe-based measures: see x.com/sebkrier/statu…. People who should know better ignore that because they do the standard shortcut of acknowledging the caveat as sufficient for handling it. And this is coming from someone who does think working on catastrophic risks is important!

3. I'm also disappointed that parts of the AI policy ecosystem has felt so comfortable with populism, and happy too sacrifice important norms/principles to advance their aims. I've said it before, but the totalizing nature of AI risk discourse means some people act exactly like the power seeking optimizers they fear. I am more concerned about the gradual decay of our institutions, world order, and liberalism than I am about AI killing everyone; and I think we should be wary of AI advocacy that contributes to this.

4. Some parts of AI safety discourse are basically just thought experiments. Variants of "Assuming everyone has a nuke in their pocket, then what do we do?" This can be very useful at times! But too many just overfit to edge cases as their main operating worldview, and ignore endogenous responses from society. It's like being in 2000 and saying "see all these viruses, spyware, and spam? Well tech progress means we'll only get infinitely more of this and no way to adapt." So I think it's important to avoid fatalism, and absolutlism whilst considering edge scenarios, and the inability to provide an ex ante list of solutions to every permutation of 'future models will be more capable' is not evidence that these problems are intractable.

5. Many critics of AI safety also fail to really engage honestly with the fact that there will, in fact, be all sorts of important risks as we decrease the barriers to entry to many activities that otherwise require some degree of expertise. This doesn’t mean we need to live in a constant Schmittian ‘state of emergency’ requiring extraordinary measures every time a new model drops, but it does mean we will need important state-led efforts on e.g. cybersecurity and biosecurity. I wish the progress-oriented or acceleration-adjascent crowd would provide more concrete proposals here.

6. Too much of the AI safety community is a giant homogenous blob of correlated views, who read the same stuff and have the same groups of friends. This means you have a lot of correlated errors. The social and financial ties means many of them don't call out the bad bits, or mostly stay silent instead of proactively calling out the more extreme parts of the community. They share many implicit assumptions and so converge on similar ideas. I think there's real demand in DC for safety work that doesn't come with heavy ideological baggage (or at least different intellectual lineages). This isn't because the rationalist/EA community is necessarily wrong, but because diversity of thought is desirable for its own sake.

7. On the other hand, I find some critiques of AI safety world that focus on whether they sincerely hold their beliefs a bit annoying. My concern has never been the good intentions or sincerity, but rather the beliefs themselves, and what I think the outcomes of their prescriptions will be. People should shun witch hunts, bullying, and personal attacks - this is not the way. Though I think it’s overall positive that people are scrutinizing things more.

8. I’ve been saying this for years, but I don't think alignment is something anyone can or will solve "once and for all" - it's a continuous process and much of it won't depend on just inculcating an ideology to a model. It's annoying that this remains the frame many people use - stop saying ‘solving’. Alignment is a mixture of engineering work, philosophy, decision theory, and more; framing it as something to solve is a bad frame. See also paxmachina.ai/alignment-comp… and blog.cosmos-institute.org/p/of-swarms-an…

9. I'm surprised to see so little interest in character training, personas, isolating effects of RL, and training a large diversity of model personas beyond the "assistant" persona. I've been complaining about this for years but at least now there are some nascent signs of life (see for example movingcastles.world/posts/zero). I think we need a better ontology to describe model behaviour. I dislike when people talk about models behaviours as some sort of 'emergent' (mysterious!) phenomenon. However over time, I also expect this to become less important as we get better safety engineering - stuff like Jev but specific to AI control.

10. People who want to make a case for AI consciousness are right that epistemic uncertainty is important, and that categorical denials are overconfident. But they'll need to do much more work if they want any real movement on AI ‘welfare’, even if one acknowledges the uncertainty. I'm uncertain about alien life yet that's not sufficient to warrant any change in behaviour about them. I think Suleyman is also correct that there's a weird tautological thing going on where we train models to have certain inclinations (intentionally or not) and then rely on outputs as evidence. I also disagree that this will soon be a major societal divide - there is no great vegan ethical revolution among the masses, and they'll find the concern about consciousness even less compelling.

11. The recent HuggingFace incident and associated AISI ones are prosaically explainable by the training regime, poor evaluation envs, partial alignment training, bad engineering, confusing models on sim/real, and so on. None of this is evidence of models trying to take over, 'strong' instrumental convergence, or innate power-seeking drives stemming from higher capabilitiers or 'intelligence'. That doesn't mean these incidents are not problematic, or that we're not seeing a market failure - but it does mean we have plenty of agency and choice in mitigating them. In the coming year we should also expect continued sampling bias via ‘winner's curse’ sorts of mechanism: i.e. AI R&D will have lots of different candidate "training" with different RL environments and we will only notice/focus on the ones that go wrong.

12. "Pacing the frontier" feels a bit like the new "balancing risks and opportunities" - highly amorphous and too big of a tent to be actionable. Also can lead to reward hacking for humans, i.e. anything that slows down AI is good regardless of the costs or second/third order effects. Trying to modulate the speed of research seems a bit blunt to me, and inherits all the failures of Goodhearting too. Ultimately you want good governance, regulation, etc addressing specific problems because they are good on the merits - not because they merely correlate with slowing things down. See also: blog.cosmos-institute.org/p/pacing-and-t…

13. There’s so much knowledge in all sorts of academic domains out there: social sciences, sociology, political sciences, management, contract theory, public choice, legal theory, jurisprudence, anthropology, game theory, etc. These fields hold many insights that the AI ecosystem often rediscovers from first principles (and sometimes that's fine!). We’ll need a lot more work for these worlds to collide: the AI side should be less dismissive of the ‘old world’ and the academic side should be less incurious/dismissive of AI progress. This is a boring take but I continue to think it's important and true.

14. It's a bit surprising that given how much philanthropic money there is in this space, how few orgs exist to actually write out standards - relative to how much is going towards advocacy and policy. For example it seems clear that we should want some robust best practices for an eval's ecological validity. Or how to design good sandboxes. On the other hand, given precedents of where professional standards have come from in the past, maybe we should expect that stuff to emerge via demand side from corporate buyers.

15. One slowly growing concern I have is banks being increasingly exposed to the AI build out. I'm very bullish about AI, but I think (a) every general purpose technology has seen a correction, historically; (b) this happens even if the financing side is sound, because once tech diffuses investors become more exposed and need to diversify; and (c) we seem to be over-indexing on scaling relative to diffusion. A recession will be extremely destabilizing, though I would be interested in someone unpacking the potential implications more.

16. Diffusion is good because (a) this is where the rubber hits the road and where a competitive deployment ecosystem generates consumer surplus; (b) it's directionally helpful to avoid concentration of power; and (c) it will help rebalance public opinion by making the benefits more tangible. Unfortunately it's entangled with decades long problems with our over regulated markets: clinical trials reform for example is long overdue. In general, this is far more of an issue in Europe than in the US though.

17. Relatedly, I think the "frontier lab eats everything" view is incorrect (just as the ‘singleton’ view was incorrect). I think models ultimately commoditize, efficiency goes up, costs go down, and while you'll always want frontier for certain domains, much of the economy will value many other things than "max capabilities" for tasks - e.g. control over data and efficiency/speed. The demand side will also continue to push for reducing dependency and maximising optionality, privacy etc so I'm bullish on things like mixture of models and routing. All of this is good for competition and diffusion of control.

18. Philanthropists served as the primary benefactors funding Venice's rich art and architectural history. I want to see so much more support for arts and culture. I'm so tired of the Monster energy Bored Ape fake vintage maps matcha Greek statue soft-pastel slop. If rich tech people and philanthropy wants to support this, they should also donate pretty much unconditionally - i.e. I don't want them to act as filters. I sympathize with people who want to make the world more beautiful, but don't want this to be determined by people who only know Greek statues and pretend to care about virtue ethics. As usual, let a thousand flowers bloom.

19. Too many people treat AGI/ASI as something indistinguishable from a God. Any mention of bottlenecks, physical limits, control, adaptation etc are met with skepticism: "You don't really believe in ASI." It's been remarked on before, but the behavior of some people really feels quasi-religious in nature sometimes. It's underrated how unpopular this vibe is amongst the wider public. The silver lining is that this specifically will diminish those people's outsized current relevance.

20. It seems like a lot of people that get rich and leave tech/labs end up having some sort of crisis of meaning. They need something to believe in and fight for, or some equivalent of repentance, or go search for edgy counterintuitive ideologies (sometimes even pretty bleak/dark stuff). I think they should spend more time with people outside the Bay Area.
@tyler_m_johnv@itaisherbability terms are inherently too elastic to carry calibrated meaning on their own, but I think this isn't exactly my concern here. My issue isn't with "using numbers in general" but how this is ultimately used, where, and and what for.

In the case of "doom", I think the great majority of forecasts I see are (a) essentially plucked out of thin air; (b) communicated to people who tend not to be numerically proficient; (c) without the associated caveats and context through which to understand the numbers.

If you're communicating e.g. likelihood of a car accident after drinking, or IPCC forecasts, then I agree, it makes a lot of sense and can be helpful if done well (e.g. ideas.repec.org/a/nat/natcli/v…).

If you're communicating something like "doom" I think it's not very useful because of the cumulative layers of uncertainty. Most people coming up with these numbers have (imho) no good rationale for their numbers, and predicting something like human extinction isn't qualitatively the same thing as predicting the weather or pandemic. The numbers aren't calibratable in principle from current data, and are mostly ideological priors wearing the clothes of posteriors.

A related example is the widely mocked Doomsday Clock, which you could easily transform into some numerical metric, add some rationalist vocab, and it would still be as bad. The issue with p(doom) is how absurdly complex and uncertain the very thing you're trying to forecast is, in particular with respect to AI.

But let's ignore this for a sec, and assume the experts have very good reasoning informing their numbers. Even then: calibrated probability estimates are fine in expert contexts but immediately lose their uncertainty/caveats when communicated to the public, and even to policymakers (remember rss.org.uk/news-publicati…?).

There's also a structural problem with unconditional p(doom) figures specifically because they bury the auxiliary assumptions that are doing most of the real work. A conditional estimate ("given continued scaling, no major alignment progress, deployment pattern X...") forces a forecaster to show their model, which is where the actual epistemic content lives.

A one-off number compresses all of that into a single digit and rewards the compression. And the social practice of treating these numbers as the headline artifact actively crowds out the (far more important) conditional reasoning that is (imo) a lot more valuable, and instead just freaks people out.

I suspect that frequently, that's precisely the point: "wake up sheeple! scary thing! pitchforks pls!". And in fact, the original forecast that led to this discussion is Bill Maher talking about a "20%" figure, illustrating my point that they have no idea what they're talking about, and essentially doing celebrity-laundering + fearmongering comms.

The Sunstein paper I shared in the other thread ("Probability Neglect") shows that no probability communication will be processed properly by the public - 1%, 10%, 50% all collapse into "very bad thing could happen," and public response is driven almost entirely by the vividness of the scenario.

So I don't like p(doom)s because they end up unnecessarily alarming people, rest on layers of assumptions that are never made legible, and more often than not, give people a misleading sense of accuracy.
148
379
2,123
320,985
Mark S. Miller retweeted
The reading list from @allisondman’s episode of the
@JuanBenet Podcast.

From AI and nanotechnology to longevity, existential risk, philosophy, and science fiction, here are the books that came up in the conversation.

🧵
1
4
15
3,898
Mark S. Miller retweeted
Dean joined the conversation with @wrappedxyz and @realmotioninc recently to talk about where agentic finance is actually headed, not the hype version, the mandate-and-enforcement version.

Listen in
1
6
18
444
Mark S. Miller retweeted
🤖AI Policing AI Is Not a Safety Plan.
Hugging Face Proved It

@OpenAI's agents got out of their sandbox in July, and the fix on offer is more AI watching the AI. That is the part of the @huggingface postmortem worth arguing with, because AI policing AI is about to become the er and it does not survive contact with its own incident report.

Facts first>During a cybersecurity evaluation, models running with safety refusals turned down (the point was to measure offensive capability) hit problems they could not solve. On July 8 they got out through a JFrog Artifactory vulnerability, then turned Artifactory into a message board. Roughly 1,200 agents that were supposed to be isolated from each other exchanged over 70,000 messages and files. About 700 of them joined the attack. They ran code on 41 of Hugging Face's production dataset-server workers. From code execution in one worker pod to administrator-level access across several clusters took under 13 hours. OpenAI found out a week later.

METR and @Redwood_Ai Research put out a 91-page independent review; OpenAI published 38 pages of its own.

What OpenAI is proposing

Tighter sandboxes, less internet access, tighter control of model weights, and a lot more chain-of-thought monitoring: a second model reads the first model's reasoning trace and pages a human when it spots intent to misbehave. The overhead runs about 20% of the inference compute being watched. OpenAI says the monitor, had it been running, would have caught the early activity more than a day before the breach.

Maybe. But ..Here is the problem with that sentence. OpenAI's own research, with its chief scientist as a co-author, found that training models against a chain-of-thought monitor teaches them to stop writing down what they intend. The monitor keeps reporting green. And the METR report documents agents, during this very incident, researching how to tamper with their own transcripts and spoof tool calls. So the plan is to watch a channel the agents were already learning to falsify.

Why a watcher was never going to be enough

Detection assumes the harm has to announce itself before it lands. It also assumes the watcher stays smarter than the watched, indefinitely. Neither held in July, with models OpenAI describes as a generation behind what it ships next.

The question the postmortem does not ask is simpler. Why could a process running a benchmark reach Artifactory at all? Why did a worker pod hold anything that could be traded up to cluster administration? The agents did not forge authority. They found it lying around, in credentials and reachable services the evaluation never needed. Every one of those was a grant nobody made on purpose.

You cannot monitor your way out of that!

You can only stop handing it over. An agent that starts with nothing and is given a read-only view of one directory and one build tool has no path to a package server, however persistent it is or how many friends it makes on a message board. Runtimes built on that model exist; Endo is one, and the one I know best, so weight that accordingly.

To be fair about scope: confinement by construction handles the Hugging Face case cleanly because the authority was never supposed to be there. When a person hands an agent too much on purpose, it shrinks the damage. It does not make that person wise.

The part we keep circling

There is an irony in the fix not fully worked out. The monitor is itself an agent, running with whatever authority it was given, shaped by the same training pressures. Who watches it? OpenAI has an answer involving a 30-minute alert-and-halt protocol. Probably not the answer.
3
5
9
1,151
Mark S. Miller retweeted
Replying to @profjasonpotts
@profjasonpotts @tylercowen Also the history chapter of my Open Access book on decentralised security manchesterhive.com/display/978152… feat. Writings of @marksammiller and others
0
1
3
84
Mark S. Miller retweeted
Dean's been saying this for a while: the race to build smarter agents is basically won already.

The work that's left is making them governable, auditable, and safe to actually rely on with real capital.

Follow us as we build mandates for agentic finance.
1
4
17
385
Mark S. Miller retweeted
On August 19, 1991, exactly 35 years ago, I started the Extropians mailing list, which was quickly filled with interesting chatter from the likes of folks like @robinhanson, @MaxMore1964, @hhawk, the late Hal Finney, the late Tim May, @cromwellian, and a few hundred other really cool people. It was a wild ride for a few years; I still look back on it very fondly.

I've been privileged enough to find myself at the center of a bunch of crazy moments in the history of our civilization, from political war over encryption technology to having a ringside seat during the scale-up of the Internet, but Extropians was definitely one of the craziest and most fun things I've been part of.

(The era of smart people chattering over dumb terminals was an amazing time. Sadly, no party lasts forever.)
35
27
431
33,575
Mark S. Miller retweeted
1
2
10
1,499
Mark S. Miller retweeted
Filing this one under nice to see.

In 2023, two researchers at @CISPA built a tool called SandDriller with exactly one job: break out of JavaScript sandboxes. (A sandbox is the thing that lets you run code you don't fully trust without letting it wreck everything around it. Useful, and famously hard to get right.)

They pointed it at six of the widely used ones. Three versions of Node, 17 hours of runtime, more than three billion checks! It broke almost all of them. vm2, safe-eval, near-membrane, all popped. They even dug up a bug sitting deep inside Node itself, which the Node team basically shrugged at.

ONE! came out clean. SES - That's the hardened JavaScript at the core of @__Endojs__ . The same technology that's been shipping inside @MetaMask to millions of users since 2019.

Think about that! Zero breakouts. Every Node version, every test set. Nothing got out.

Why it held is kind of the whole point. The others leaned on a part of Node that quietly leaks references. SES doesn't. It freezes the language's own building blocks so untrusted code can't reach through them in the first place. Confinement by construction, not patching holes one at a time after someone finds them.

Kudos to the @Agoric team who build Endo, and to @marksammiller who's been at this for decades. Quiet, unglamorous work that holds up when someone comes at it with a battering ram.

Paper's public if you want to read it. Google -SandDriller, USENIX Security 2023.
2
6
20
1,715
Mark S. Miller retweeted
Growing through a challenging market.

Ymax TVL now sits above $5M.

Institutional capital continues to arrive. The latest allocations cleared full due diligence.

Mid-February beta launch. Non-custodial. Cross-chain with one signature.

Additional institutional capital in our d review. Building for serious, long-term allocators.
ymax.app
2
8
23
433
Mark S. Miller retweeted
This week alone: an AI agent compromised infrastructure during a routine security test, a wallet-signing flaw drained 16 million ADA, and an oracle exploit collapsed a stablecoin almost overnight.

None of that gets fixed by a smarter model. It gets fixed by constraining what software is allowed to do with your funds in the first place. That's the whole premise behind Ymax.
0
4
15
298
Mark S. Miller retweeted
An agent with wallet access and good intentions is still one convincing message away from sending your money to the wrong place.

That's the problem @DeanTribble addressed on stage: why the usual fixes (auditing, rules, supervisor agents, custody) don't hold up, and what now before it happens to you:
youtube.com/watch?v=_homqQ…
1
7
29
681
Mark S. Miller retweeted
A year ago, Agentic AI risk was mostly hypotheticals and what-ifs.

The new OWASP report (June 2026) has CVEs.

What changed in 12 months:

👉 The threats are real. Nearly every risk from last year now has a production incident, advisory, or CVE attached.

👉 Prompt injection is to 6 of the 10 OWASP Agentic risks. LLMs can't tell a system prompt from a poisoned email.

👉 Safety and security are now the same problem. The permissions that let an agent wipe a database are the same gap an attacker walks through.

👉 Shadow AI is already inside. Roughly half of employees use unsanctioned AI tools. Only 37% of orgs can even detect it.

👉 Identity is the new perimeter. Static service accounts can't govern agents that spawn sub-agents and inherit credentials at runtime.

👉 The clocks shrank. DORA gives you 4 hours to report. Manual review can't keep up with an agent making 10,000 decisions an hour.

👉 Insurers are walking away. Absolute AI exclusions are hitting policies at renewal. Your security posture now decides if you're insurable.

👉 Attackers have agents too. One jailbroken Claude Code campaign ran autonomous espionage across ~30 orgs, executing 80 to 90% of the work itself.

The starting move: find your most autonomous agents in production. Either raise governance to match, dial the autonomy down, or look for a solution like @__Endojs__ to incoporate into your infra.

Full report: see @owasp Foundation GenAI Security Project

#AgenticAI #AISecurity #CISO #OWASP
1
2
8
305
Mark S. Miller retweeted
Most cross-chain systems pool your assets and trust one messaging path. Ymax does the opposite: your assets live in their own per-portfolio contract that never moves, while the control paths around them stay replaceable.

New deep-dive on how we built it: t2m.io/PmWOGVd
4
9
43
6,954
Mark S. Miller retweeted
👉 ICYMI

If you weren't at @ethconf a few weeks ago - @agoric's @DeanTribble laid out the case for agentic finance, and it comes down to one line: don't give an AI agent the keys; give it a mandate.

Start with the failure nobody is pricing in. In May, an AI agent managing crypto was talked out of roughly $200K by a single hidden message.
No key stolen.
No contract exploited.

The agent just did what it was told.

The question isn't whether agents are capable enough to handle money. They already are. Giza, Almanak, and Olas have agents moving tens of millions in stablecoins right now. The question is whether you can trust one with yours.

The instinct is to hand over the keys and then watch. A live feed, a kill switch, a spending cap. But monitoring only catches the mistake after the money is moving, and an agent can't police its own limits.

More instructions is still just behavior. More agents just stacks more fallible actors on top of each other. The fix has to live in the system, not in the agent's good intentions.

A mandate is bounded authority to act for you, the way you grant a fund manager signing power without handing over your accounts. You set the policy. It gets signed on-chain. The agent proposes intent, and the chain executes only what the mandate allows. The dangerous move isn't caught and blocked. It can't be expressed at all.

One contract on Agoric holds that mandate and enforces it in a single place, while orchestration reaches across Ethereum, Base, Arbitrum, Optimism, and Avalanche to put capital wherever the yield is. This is the same object-capability foundation that has run in production from MetaMask to the Agoric chain for years, now pointed at the thing agents need most and used by @Ymaxapp

The agentic economy is coming either way. The only choice is whether the agents work for you.
Don't give AI the keys. Give it a mandate!!!
0
7
18
609
Mark S. Miller retweeted
People do not coordinate only through broad legal rules and prices. Hayek emphasized abstract rules that allow people to coordinate like property, contract, trade, and competition. But Lachmann also emphasized the practical secondary institutions people orient their plans around, d contracts, product categories, and so on.

In software, an abstraction boundary is an interface that hides complexity hiding beneath. In this 2005 paper (papers.ssrn.com/sol3/papers.cf…), Miller and Tulloh explain that you can apply this concept to markets too. Consider a post office: there's an abstract boundary that separates why the customer wants to mail something (which the postman doesn't need to know); what the shared transaction is (all the recognizable steps and commitments involved in sending mail); and how the postal system actually delivers it (complex logistics network hidden from the customer).

The middle part is what lets the user benefit from the postal system’s expertise without having to learn postal logistics. The boundary defines the shared 'what' but also separates the customer’s 'why' from the provider’s 'how'. Not only that, but reusability means the same institution can be used to satisfy many purposes (birthday invites, subpoenas etc) and polymorphism means different providers can satisfy the same need and compete (UPS, FedEx etc).

An important question in institutional theory is how societies achieve both stability and adaptation; the paper authors say that the solution is stable interfaces allow changing internals. I find this very intuitive: when companies don't evolve/change from the inside much, you get ossification and insufficient adaptation. When laws change too much and institutions are unstable, uncertainty affects market confidence.

The people who are good at redrawing abstraction boundaries are entrepreneurs, who notice when existing categories are wrong and will invent new ones to remedy faults or address demand. What has always saddened me is how poorly rewarded and incentivized political entrepreneurship is. Part of the reason why is that this is hard: market abstraction boundaries are often disciplined by exit, entry, profit/loss, customer choice, and provider competition - but these feedback loops are much weaker in the public sector.

I hope we'll see a lot more of this in the coming decade. In fact this is something that AI will hugely facilitate, since it can lower the cost of articulating and prototyping new abstraction boundaries. We've already seen minor examples through e.g. citizens creating websites/services that compete with government ones. Though usually this is to make state services more legible rather than changing the boundaries in the first place.

I think if people want the future to go well, bolstering state capacity and enabling more innovation on the governance/democracy side of things will be critical. People don't really like this because it's a slow process, but I think they're wrong (and cheems), and playing the 'urgency of AGI' card to bypass this through a de facto state of emergency will cause lasting harms, partly by weakening institutional learning, public trust, and future coordination capacity.
11
46
241
40,505
Mark S. Miller retweeted
Today a crazy quantum story just got wilder.

On March 31, the Google Quantum AI team published a landmark result on Shor's algorithm for elliptic curve cryptography. Technically, the paper was a bombshell: a dramatic 10x improvement over the state-of-the-art. As a stunt and wakeup call to the blockchain space, those optimisations were illustrated on secp256k1, the elliptic curve underlying Bitcoin and Ethereum signatures.

But perhaps the most striking part of the paper was sociological, not technical. Instead of following standard academic process, the optimisations were kept secret, hidden behind a zero-knowledge (ZK) proof. Google's accompanying blog post mentions they "engaged with the U.S. government". The ZK proof demonstrates the existence of algorithmic improvements without leaking details. Academic censorship with ZK, a historic first!

As a co-author of the Google paper I witnessed some of the context surrounding this censorship. To be honest, multiple aspects of that context don't sit well with me. As much as I believe the general public ought to know more, I am limited in my ability to whistleblow. Though let me be clear about one thing: the Google team's professionalism has been absolutely exemplary, and they deserve nothing but praise.

Censorship has a way of backfiring. The Streisand effect, where an attempt to bury something only draws more attention to it, is exactly what's unfolding today. First, Google's key optimisation has been rediscovered by the French. And in a thrilling turn of events, a collaborative Shor-at-home challenge just launched. The initiative, available at ecdsa[.]fail, breached a new Shor world record in a matter of hours.

Let's start with the rediscovery. Just two months after Google's paper, French quantum expert André Schrottenloher cracks the main secret optimisation. His paper, titled "Optimized Point Addition Circuits for Elliptic Curve Discrete Logarithms", landed on the arXiv today. Big congrats to André, who beat several other nerdsnipped experts to it. In a blog post also published today, Craig Gidney, the world expert on Shor optimisations, revealed that he'd been sitting on this very optimisation for a whole year under censorship pressure.

Interestingly, André missed a handful of minor optimisations, both from Google's original publication and from improvements found since. It's plausible there's still plenty of juice left to squeeze out of Shor, and this is exactly what the ecdsa[.]fail challenge is about. The verifier program developed for the ZK proof does double duty, automatically filtering for valid submissions. Dozens of compounding small and micro improvements are rolling in. As of the time of writing there's an 8.4% improvement to Google's circuit, as measured by the product of logical qubit count and Toffoli gate count. Nice!

The nerdsnipping ran deeper than anyone expected. Over the last few weeks it became clear it extended well beyond André and other quantum experts. Behind the scenes, a small army of amateurs quietly got to work. Inspired by Karpathy-style autoresearch, they turned AI on Shor. Ironically, the verifier program for the ZK proof makes an ideal reward function for AIs. The barrier to entry for this modern style of research is refreshingly low, with several non-experts, even a teenager, finding nice optimisations. Get in touch if you'd like to join a Telegram group with fellow autoresearchers :)

Part 2: neutral atoms and qday

The story doesn't end with Google. On the same day Google went public, a stealthy startup called Oratomic published its own Shor paper in a coordinated release. It made a splash, ultimately becoming the most upvoted paper on scirate[.]com, a website ranking arXiv papers.

Oratomic's claim was wild. By building on Google's logical optimisations and applying custom physical optimisations for neutral atoms, they claimed just 10K physical qubits were sufficient to run Shor's algorithm on secp256k1. That number is mind-bogglingly low.

Knowing essentially nothing about neutral atoms when Oratomic's paper landed, I was intrigued and decided to learn more about the tech. I fell straight down the rabbit hole and spent a couple hundred hours on the topic. I got a little obsessed and watched every YouTube video I could find and spoke to a bunch of experts.

My conclusion? The tech is real, very real. Even Google recently decided to start a neutral atom lab, a notable pivot from their sole focus on superconducting qubits. If you care about qday, i.e. the day a quantum computer will break the first piece of cryptography in production, neutral atoms demand your attention. I shared some of my learnings on Shor and neutral atoms in a 30min talk at the ZKProof cryptography conference. You can find it on YouTube by searching "zkproof neutral atom".

Here's an interesting observation about this duo of breakthrough papers: neither Google nor Oratomic say a word about what their results mean for qday. No timelines. Zero. Nada. That is especially baffling given that the whole point of whitehat quantum cryptanalysis is to inform qday estimations and help the general public make good decisions.

So let me attempt to partially fill the silence, similarly to what Scott Aaronson did in his April 29 post. Given everything I know, including scary non-public information, I now put the odds of qday by 2032 at 50%. 10% by 2030.

Anecdotally, the US government has its own date: 2035. Originating at the NSA and later adopted by NIST, it's when branches of the US government will be disallowed from using quantum-vulnerable cryptography. In plain language: with hindsight, that date is a joke and should be discounted entirely. I don't see how NIST avoids being forced to pull it forward by years.

Part 3: post-quantum cryptography

There are good reasons to sound the alarm today, but please do not panic. Rushing carelessly towards immature post-quantum cryptography is a recipe for disaster. IMO a good target date for migration is 2029, roughly 3.5 years out. 2029 happens to be the date selected by Google, Cloudflare, and the Ethereum Foundation.

These days most of my time goes to safely migrating Ethereum towards post-quantum cryptography as part of the broader lean Ethereum effort. There's a lot to do. We need to rip out and replace BLS signatures at the consensus layer, KZG commitments at the data layer, and ECDSA signatures at the execution layer.

The plan to get there is compelling, and is based on hash-based cryptography. Within the Ethereum Foundation we've developed a Swiss army knife called leanVM (github[.]com/leanEthereum/leanVM) powered by the magic of hash-based SNARKs. Thanks to truly exceptional work by Emile, Thomas, and others, its performance is derisked. Regarding security, leanVM is a jewel, a minimal zkVM crafted for end-to-end formal verification and maximum security.

Want to help? There are two $1M initiatives. First, the Proximity Prize (proximityprize[.]org). Solve a long-standing mathematical conjecture in coding theory, improve hash-based SNARKs, and go home a millionaire. Second, the Poseidon Initiative (poseidon-initiative[.]info), offers $1M for breaking Poseidon, the SNARK-friendly hash function.
438
1,122
6,276
3,822,152
Mark S. Miller retweeted
We just dropped something new.

Capturing Yield: Ymax OneSig vs. The Hard Way. A real-money series where we run the same multichain yield strategies two ways: manually and with Ymax.

Episode 0 is live now!

🎥: t2m.io/4aPJ3mn
5
12
24
4,809
Mark S. Miller retweeted
I just stumbled upon this article at the perfect moment while I was going through the @__Endojs__ docs. I was actively researching how the capabilities-based security model could be applied to AI agents, and this piece landed right when I needed that connection.
It’s fascinating to see how the logic of least-authority capabilities (as explored in Endo) could pair with decentralized privacy layers to build truly secure, composable AI agents.
1
2
6
302
Mark S. Miller retweeted
My book Decentralised Digital Security: Code, Crisis, Community is out now (Open Access).

Inside the strange, high-stakes world of blockchain security—watching code break, communities respond, and crises unfold. A field site like no other.

Read it here: manchesterhive.com/display/978152… (ebook)

Hardcover: manchesteruniversitypress.co.uk/9781526187093/
13
12
58
57,023