Larry the Learning Mechanic@lrn_mech

Oct 22

Nick

@nickcammarata

Oct 15

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

already-clean.goodfire.com

Jamie Simon@learning_mech

Oct 20

Jean is building honest software

@slashslashdev

Oct 19

"I’ve been writing an album of folk songs about AI...." @learning_mech is performing a concert this Thurs at @imbue_ai! RSVP luma.com/n6j7vvod?tk=Te…

Jamie Simon@learning_mech

Oct 18

Florentin Guth@FlorentinGuth

Oct 18

I'm delighted to announce that I'll join @MITEECS as an Assistant Professor next Fall! I will be recruiting graduate students this cycle, please apply if you're interested in the science of deep learning! Despite the ambient doomerism, now is a great time to do research.

Jamie Simon@learning_mech

Oct 4

Krzakala Florent

@KrzakalaF

Oct 3

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help.

After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem.

Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?”

Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold.

Let us be specific:
- In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path.
arxiv.org/abs/2310.01169

- Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787

- On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps.
arxiv.org/abs/2608.10184

- On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415

- On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372

These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins:
“We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…”

Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem.

How is that consistent with advertising this example through the suggestion that there was “no existing solution path”?

Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement.

An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new.

A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches.

You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt.

Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.