Pinned Tweet
"Evolution is anything but random."
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
8
42
331
33,596
Machine Learning Street Talk retweeted
Interesting interviews by Dr. Tim Scarfe. Math has relied on GOFAI for some time that was painstakingly built. Agents can now use these tools to do amazing things.
youtu.be/IRJDwtAreB4?is…
youtu.be/IRJDwtAreB4?is…
0
1
6
1,041
With Kevin Hartnett (@KSHartnett), Leonardo de Moura (@Leonard41111588), Kevin Buzzard, Jeremy Avigad and Paul Lessard.
Kevin's new book, The Proof in the Code: How a Truth Machine Is Transforming Math and AI, tells the whole story: quantabooks.org/books/the-proo…
Kevin's new book, The Proof in the Code: How a Truth Machine Is Transforming Math and AI, tells the whole story: quantabooks.org/books/the-proo…
0
1
1
825
Machine Learning Street Talk retweeted
Interesting that automated hill climbing might end up encouraging fundamental innovation.
As the cost of hill climbing goes down, exploring a new idea can become much cheaper.
Historically, exploring a new direction could require a prohibitively large investment in optimizing everything to match the performance of a mature method, especially when other layers have been co-optimized for that mature method.
As the cost of hill climbing goes down, exploring a new idea can become much cheaper.
Historically, exploring a new direction could require a prohibitively large investment in optimizing everything to match the performance of a mature method, especially when other layers have been co-optimized for that mature method.
I think the last month might have been the biggest month in (zeroth-order) optimization history.
In the 10 days since we released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup. Dust itself was up to 10,000x than the previous state-of-the-art.
Not only can we pretrain a transformer without backprop, it's much more efficient than anyone would've expected. Search fking works!
Now, scaling is the biggest open question. Some of the tricks that brought the time down to 1.6 seconds will simply not scale. To discard those, we'll be scaling the dataset every time it gets overfit, starting with 30M tokens instead of 10M.
Huge thanks to @jayden_teoh_, @spicey_lemonade, @RobertTLange, @EmanueleRodola, @ChunyuanDeng, @LuochengHuman, @Alaxarr, @pavleculafic, @IsaacYong13, Euraba and areu01or00 (GitHub), for the contributions. And @cs_serdar for gracefully maintaining the speedrun and @bishmdl76 for helping out.
Expect a lot more from us over the coming weeks.
2
2
50
3,184
Machine Learning Street Talk retweeted
I was wondering if OPAL could help the optimizer for the 0-order noise val track.
So I took the (old 10M token) Noise benchmark's best validation-loss record (#14, 3.676) and wrapped its optimizer with OPAL. Every 50 updates, gpt-6.1-sol reads the loss and its own diagnostics, e in Python, and tests it before it goes live.
Results (seed 0):
- record: 3.6758
- no agent (rerun): 3.6773
- agent, neutral prompt: 3.6791
- agent, prompt informed by the record: 3.6695
- same, xhigh reasoning: 3.6656 (−0.010 vs the record)
The OPAL system prompt mattered: telling the agent what its gradients are, how the record's optimizer came about, and how the end-of-run weight merge works turned no gain into a clear one.
There is an interesting moment at update 243: The record switches its gradient estimator (Boost → Cut), and the gradients change scale. All three agents had tuned their optimizers on Boost's gradients, and they lost ground right after the switch, up to 0.03 in training loss. OPAL noticed, rescaled the LM head's Adam statistics to the new gradients, and kept adjusting. By update ~620, every run was ahead of the record's optimizer again, and the informed runs stayed ahead to the end.
What the best agent changed, keeping the record's ANVIL II update for the weight matrices:
- per-layer rates for the residual mixing scalars, which it halved at the start
- matrix rate −10%, +20% on two query matrices, −15% on the value matrices
- longer momentum averaging (higher β1) for the scalars, head and gates
- the head's Adam rescale at the estimator switch
This is, of course, only anecdotal, and a minor improvement. Still, demonstrates that OPAL doesn't diverge and can mine small gains.
Intuitively, online agent-guided adaptation makes the most sense when hyperparameters/configurations haven't been tuned to death.
So I took the (old 10M token) Noise benchmark's best validation-loss record (#14, 3.676) and wrapped its optimizer with OPAL. Every 50 updates, gpt-6.1-sol reads the loss and its own diagnostics, e in Python, and tests it before it goes live.
Results (seed 0):
- record: 3.6758
- no agent (rerun): 3.6773
- agent, neutral prompt: 3.6791
- agent, prompt informed by the record: 3.6695
- same, xhigh reasoning: 3.6656 (−0.010 vs the record)
The OPAL system prompt mattered: telling the agent what its gradients are, how the record's optimizer came about, and how the end-of-run weight merge works turned no gain into a clear one.
There is an interesting moment at update 243: The record switches its gradient estimator (Boost → Cut), and the gradients change scale. All three agents had tuned their optimizers on Boost's gradients, and they lost ground right after the switch, up to 0.03 in training loss. OPAL noticed, rescaled the LM head's Adam statistics to the new gradients, and kept adjusting. By update ~620, every run was ahead of the record's optimizer again, and the informed runs stayed ahead to the end.
What the best agent changed, keeping the record's ANVIL II update for the weight matrices:
- per-layer rates for the residual mixing scalars, which it halved at the start
- matrix rate −10%, +20% on two query matrices, −15% on the value matrices
- longer momentum averaging (higher β1) for the scalars, head and gates
- the head's Adam rescale at the estimator switch
This is, of course, only anecdotal, and a minor improvement. Still, demonstrates that OPAL doesn't diverge and can mine small gains.
Intuitively, online agent-guided adaptation makes the most sense when hyperparameters/configurations haven't been tuned to death.
What if optimizers could reason adaptively?
OPAL (OPtimizer with an Agent in the Loop) by @SimulatedAnneal et al. puts an LLM agent inside the training loop. It's not an outer-loop hyperparameter tuner. It watches the training dynamics and rewrites the optimizer's source code nterface is two files, both written by the agent:
1. analysis: probes the agent writes itself to pull out whatever stats it wants (per-group grad norms, update/weight ratios, cosine similarity between consecutive gradients, spike detectors). Raw weights would flood its context, so it decides what to look at.
2. updater: the update rule, run on every step. Since the interface is code, anything can change: LR schedules per param group (including raising the LR mid-run), AdamW→Muon on specific tensors, bf16 vs fp32 per tensor, momentum resets, clipping, gating bad batches, even A/B tests on itself.
analysis → reasoning → update rule → training → analysis. The algorithm is no longer a fixed object. Part of it is an agent. It's fully zero-shot: no meta-training runs.
This native embedding of agents within established algorithms is super interesting: it enables online meta-learning, with the meta-learner swapped out. It reminds me a lot of @Luke_Metz 's work on learned optimizers. VeLO amortized optimizer learning into network weights through huge meta-training across thousands of tasks. OPAL gets its "meta-knowledge" for free from LLM pretraining (papers, folklore, debugging intuition) and adapts within a single run, through reasoning in context instead of gradients. That partly avoids the problems of meta-optimization: truncated unrolls, short-horizon bias, generalizing to unseen tasks.
The interactive blog post is great with many takeaways re agent implemented strategies: per-tensor adaptive clipping, extend SWA, momentum adaptation based on inferred flatness.
Blog: xingjianbai.com/reactive-optim…
Code: github.com/xingjian-bai/o…
6
5
44
6,646
Machine Learning Street Talk retweeted
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
654
1,069
9,748
2,069,008
Machine Learning Street Talk retweeted
🚨🚨ALERT: ANTHROPIC IS A/B TESTING USAGE LIMITS
Many users on X are reporting LOWER usage limits right now. THEY ARE NOT WRONG. We detected an A/B test going on that DECREASES your monthly API-value by ~20% while INCREASING your 5h usage limits by ~20%. This only affects SOME accounts. (1/2)🧵
Many users on X are reporting LOWER usage limits right now. THEY ARE NOT WRONG. We detected an A/B test going on that DECREASES your monthly API-value by ~20% while INCREASING your 5h usage limits by ~20%. This only affects SOME accounts. (1/2)🧵
203
197
3,228
510,692
Machine Learning Street Talk retweeted
Yet more evidence that reinforcement learning is basically useless.
Turns out that simply extracting policies with supervised learning conditioned on the advantage can get state of the art results in RL, as long as we get a few important details right!
15
17
252
45,247
This is massive - open weights, best in class coding model at nearly 130 tokens/sec on a MacBook Pro (!!!)
@Recursive_SI cooked hard on this one
@Recursive_SI cooked hard on this one
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
0
0
57
4,455
Machine Learning Street Talk retweeted
The next generation of renormalizing generative models could become useful for hierarchical planning.
Currently, its an abstract predictive hierarchy. It could become an abstract control hierarchy. Read the blog:
blog.lancelotdacosta.com/posts/2026-10-…
Currently, its an abstract predictive hierarchy. It could become an abstract control hierarchy. Read the blog:
blog.lancelotdacosta.com/posts/2026-10-…
0
6
31
1,993
Great post on dealing with conference slop tsunami
Some thoughts on reviewing: spylab.ai/blog/reviewing/
First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%.
Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
0
2
22
2,885
Machine Learning Street Talk retweeted
New paper: AI evaluation is a burgeoning field. How can researchers entering the field understand best practices and open problems?
In our paper, led by @flxcn, we develop a concrete list of open questions and best practices for agent evaluation. We also share recommendations anies, policymakers, and more.
In our paper, led by @flxcn, we develop a concrete list of open questions and best practices for agent evaluation. We also share recommendations anies, policymakers, and more.
22
44
315
24,214
Machine Learning Street Talk retweeted
my iclr 27 papers to review are some of the strongest evidence against rsi I've seen to date. just the most dreadful claudeslop.
12
20
547
28,854
Machine Learning Street Talk retweeted
First, congrats to @nickcammarata on questioning assumptions in his field and reinvigorating its discussion. In the same spirit, I want to interrogate interpretability as well: why does it always start from the assumption that the system works? If NNs face challenges with AI neralization, continual learning, and creativity, then are not their internal mechanism the place to explain (and ultimately rectify!) such problems? If you look inside a system and really understand how it works, then shouldn’t you also gain some insight into how it could work better?
Mech interp focuses on mind reading and mind control, but seems strangely ambivalent to mind improvement. Our 2025 paper on fractured entangled representations (FER) showed evidence of a striking chasm between neural networks with good representation (from open-ended search) and bad representation (from conventional gradient descent) in small networks that output Picbreeder images. I would expect precisely the kind of meticulous sleuthing @nickcammarata showcases to help to illuminate where FER might be hiding and how much of it there really is in large networks and LLMs, and ultimately what kinds of interventions mitigate it.
Let me give an example image from Nick’s own blog, which I pasted into this post. The image is presented in the blog to support the idea that there is a “clean” interpretation of each neuron. But what about thinking about the story these neurons are telling about the adaptability of the circuit itself?
The story about cleanliness says that a number of clean feature detectors feed into a “pointy” V-detector, which is then used to help identify a bunch of pointed objects.
But think about the entire circuit more critically, and it starts to look like the entanglement (the E) in FER: we have a “van front” detector feeding into our pointy V-detector, which then feeds into (is read by) a coyote face detector. Does it not follow that if a gradient update later alters our van front detector, then our downstream V-detector will consequently be altered, and hence our ability to detect coyote faces? If so, then changes in van detection will impact our ability to perceive coyotes - coyotes and vans are entangled in this circuit!
Is this normal in large networks? Should we even be using the word “clean” to describe a situation like that? How pervasive is this kind of entanglement? Similar evidence for and questions about fracture (as opposed to entanglement in this example) could easily be raised.
To be clear, I’m not trying to claim that the FER hypothesis has been confirmed by a single (or even a few) anecdotal examples in a piece creating controversy with its own set of provocative claims. Rather, what I’m saying is, if you have (even imperfect) microscopes looking inside these systems then there are a lot really interesting questions you could be asking about the structure of their representations beyond just whether we can read their minds - mechanistic interpretability could widen its scope!
Mech interp focuses on mind reading and mind control, but seems strangely ambivalent to mind improvement. Our 2025 paper on fractured entangled representations (FER) showed evidence of a striking chasm between neural networks with good representation (from open-ended search) and bad representation (from conventional gradient descent) in small networks that output Picbreeder images. I would expect precisely the kind of meticulous sleuthing @nickcammarata showcases to help to illuminate where FER might be hiding and how much of it there really is in large networks and LLMs, and ultimately what kinds of interventions mitigate it.
Let me give an example image from Nick’s own blog, which I pasted into this post. The image is presented in the blog to support the idea that there is a “clean” interpretation of each neuron. But what about thinking about the story these neurons are telling about the adaptability of the circuit itself?
The story about cleanliness says that a number of clean feature detectors feed into a “pointy” V-detector, which is then used to help identify a bunch of pointed objects.
But think about the entire circuit more critically, and it starts to look like the entanglement (the E) in FER: we have a “van front” detector feeding into our pointy V-detector, which then feeds into (is read by) a coyote face detector. Does it not follow that if a gradient update later alters our van front detector, then our downstream V-detector will consequently be altered, and hence our ability to detect coyote faces? If so, then changes in van detection will impact our ability to perceive coyotes - coyotes and vans are entangled in this circuit!
Is this normal in large networks? Should we even be using the word “clean” to describe a situation like that? How pervasive is this kind of entanglement? Similar evidence for and questions about fracture (as opposed to entanglement in this example) could easily be raised.
To be clear, I’m not trying to claim that the FER hypothesis has been confirmed by a single (or even a few) anecdotal examples in a piece creating controversy with its own set of provocative claims. Rather, what I’m saying is, if you have (even imperfect) microscopes looking inside these systems then there are a lot really interesting questions you could be asking about the structure of their representations beyond just whether we can read their minds - mechanistic interpretability could widen its scope!
13
14
147
8,770
Machine Learning Street Talk retweeted
Some observations about the "total" automation of software engineering:
1. There are more software engineers now than in 2025, and demand is growing.
2. Software engineers have never worked harder than they do now.
3. They largely don't write code anymore, agents write ~100% of the code.
4. The bottom line of the companies that employs these engineers and agents (i.e. the terminal purpose of all this code) is typically not budging as a result of AI adoption.
This is why you have to think in terms of systems. Automating one aspect of a system creates its own friction and moves the bottlenecks elsewhere.
1. There are more software engineers now than in 2025, and demand is growing.
2. Software engineers have never worked harder than they do now.
3. They largely don't write code anymore, agents write ~100% of the code.
4. The bottom line of the companies that employs these engineers and agents (i.e. the terminal purpose of all this code) is typically not budging as a result of AI adoption.
This is why you have to think in terms of systems. Automating one aspect of a system creates its own friction and moves the bottlenecks elsewhere.
326
251
2,796
287,103
Machine Learning Street Talk retweeted
There has never been a better time to go into math: youtube.com/watch?v=LKrwhj…
18
23
244
19,009
Machine Learning Street Talk retweeted
Natural evolution's open-endedness leads to beautiful, complex emergent structures and self-organizing behavior 🌱✨. Replicating this in silico is famously hard 💻. Our paper points to a promising direction by evolving populations of competing neural cellular automata with sambard
⚠️⚠️flashing lights, rapid cuts, or strobe effects in this thread! 🚨🚨
1/n
⚠️⚠️flashing lights, rapid cuts, or strobe effects in this thread! 🚨🚨
1/n
3
36
160
28,897
Interesting work from @goodfire!
> The leading explanation for polysemanticity was superposition. This proposed a new learning principle: when a neural network runs out of neurons, it stops grouping related things together and instead begins folding unrelated things together, simply because they don’t interfere. The result is a kind of knot: the network could have folded in many almost equally good ways, so which things a neuron groups together tells you almost nothing. You have to unfold the knot before you can study anything. In practice that means training a second model – a decomposition such as a sparse autoencoder – to approximate the network as it would look unfolded and studying that instead. If the network really is knotted, there may be no alternative.
So they are basically arguing that modern neural networks produce a much cleaner and interpretable factorisation, and the idea of "convergent natural modularity" is very exciting (to the extent to which it may be possible).
Also check out our recent interview with Tom their Chief Scientist.
> The leading explanation for polysemanticity was superposition. This proposed a new learning principle: when a neural network runs out of neurons, it stops grouping related things together and instead begins folding unrelated things together, simply because they don’t interfere. The result is a kind of knot: the network could have folded in many almost equally good ways, so which things a neuron groups together tells you almost nothing. You have to unfold the knot before you can study anything. In practice that means training a second model – a decomposition such as a sparse autoencoder – to approximate the network as it would look unfolded and studying that instead. If the network really is knotted, there may be no alternative.
So they are basically arguing that modern neural networks produce a much cleaner and interpretable factorisation, and the idea of "convergent natural modularity" is very exciting (to the extent to which it may be possible).
Also check out our recent interview with Tom their Chief Scientist.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com
2
5
49
3,369
> I used to think of neurons as human-shaped nouns: one is a car, another is a curve. Now I think of them more as control flow. A neuron is less about what it sees than about what it does to the flow coming in, and often its job is to gate a region of the network. Seen this way, monosemanticity is almost the wrong question. A better one is what a neuron lets through and what it keeps out.
1
0
5
755
This is also quite an interesting example of what we as humans would (intuitively) think is a bad factorisation; why start with pose and not objects? Or maybe our intuitions are just bad and pose is better to start with even though it might fractionate downstream objects.
There on analogy here though that these "forks" in the road of representations get "locked in" earlier on and can't be changed. Maybe in the future we could design training algorithms to produce obviously better paths.
There on analogy here though that these "forks" in the road of representations get "locked in" earlier on and can't be changed. Maybe in the future we could design training algorithms to produce obviously better paths.
0
0
2
512