Pinned Tweet
1/ As compute continues to grow and simulators continue to improve, it is becoming feasible to train RL agents for billions or trillions of timesteps. However, this is only useful if agents can continue learning over such long training horizons, which is far from given 👇
8
70
667
135,063
This is so cool!
I’ve had so much fun making bricky, and playing with it: a compiler for LEGO® models, built for AI agents.
The agent writes a Python build script. bricky places every part, reports each mistake, and renders the model.
Thanks to @omar_aflak for the idea of a LEGO® compiler.
0
0
7
733
Really excited to be on a panel with such great researchers!
Finally, here are the participants of our AutoRL debate!
We’re switching things up from a traditional panel: three controversial positions, with participants randomly assigned to argue for or against each one.
Expect opposing views, lively discussions, and surprising insights!
2
0
17
1,727
Michael Beukman retweeted
🇨🇦 I'll be heading to Montreal this weekend to present Hierarchical Behaviour Spaces at RLC. If you're interested in RL in hard exploration environments (especially the NLE) get in contact and let's have a chat!
4
6
34
2,603
I'll be at RLC in Montreal this weekend to present this paper. Would love to chat about all things open-ended learning, curriculum learning and RL applications, please reach out :)
1/ As compute continues to grow and simulators continue to improve, it is becoming feasible to train RL agents for billions or trillions of timesteps. However, this is only useful if agents can continue learning over such long training horizons, which is far from given 👇
0
2
13
1,631
1/ We are happy to announce the largest known dataset of expert trajectories for physics-based tasks, containing over 11M unique levels in Kinetix.
3
14
147
17,561
3/ While there are many different ways you can use this data, as a proof of concept, we show that standard behaviour cloning serves as a strong starting point for further PPO fine-tuning, leading to strong agents using much less compute than training from scratch using RL.
2
0
11
742
4/
Dataset (326 GB total): huggingface.co/datasets/mbeuk…
Code + examples: github.com/FLAIROx/Kinetix
Blog: kinetix-env.github.io/dataset/
Thanks to everyone from FLAIR that helped with this, in particular @mitrma, @uljadb99 and @j_foerst!
Dataset (326 GB total): huggingface.co/datasets/mbeuk…
Code + examples: github.com/FLAIROx/Kinetix
Blog: kinetix-env.github.io/dataset/
Thanks to everyone from FLAIR that helped with this, in particular @mitrma, @uljadb99 and @j_foerst!
1
2
24
1,994
Really exciting work from @mitrma!
Over this week, we'll be posting about papers from BOLD on show at the upcoming #ICML2026. Up first: LEO, led by @mitrma!
Typical approaches in goal-conditioned RL select a single goal that is completed later in the episode. This paper instead asks a simple question: what if r labelling (and how could we do that efficiently)?
Check it out: arxiv.org/pdf/2605.23551
0
0
6
433
So excited for BOLD!
Hello world :)
We are BOLD — the British Open-ended Learning and Discovery Lab!
BOLD is a new academic research lab fully focussed on paradigm breaking discoveries in fundamental AI. We work towards more efficient & open AI that is built around human needs and capabilities.
To ghs, we pioneer new modes of collaboration in academia that are more focussed, resourced, agile, and collaborative. Rather than fragmenting resources, today we are sunsetting 5 of the UKs leading AI labs to join forces under our joined scientific vision.
Our vision is centered around three pillars:
⚡ Beyond backpropagation – questioning the foundations of the field.
🤝 Human-centric learning & discovery – treating humans as core to our algorithms
🤖 Embodied learning – fast learning and adapting methods that deal with the messy real world
BOLD is backed by @UKRI_News and @EPSRC with £30M – and this is just the beginning. We are urgently looking for partners and sponsors to 10x this.
👉 ox.ac.uk/news/2026-06-2…
👉 bold-lab.ai
@j_foerst, @CULLYAntoine, @tonizza82, @shimon8282, @tonizza82, Ani Calinescu & @_rockt
0
0
16
955
Michael Beukman retweeted
⛳ Introducing purejaxgcrl: goal-conditioned reinforcement learning in end-to-end JAX for discrete action environments!
Train generalist, goal-conditioned agents in minutes on a single GPU.
1/
Train generalist, goal-conditioned agents in minutes on a single GPU.
1/
1
22
86
5,735
Michael Beukman retweeted
After interviewing for Research Scientist roles at DeepMind, Isomorphic, Meta, Cohere and more, I wrote up everything I learned. Technical prep, logistics, negotiation, and emotional breakdowns. Check out my guide: silviasapora.github.io/blog/ml-interv…
47
293
3,633
247,012
It was really cool to see the results of trying to soak up as much information from every transition as possible!
Hindsight Experience Replay has become the ubiquitous method for goal-conditioned reinforcement learning, but leaves open the question of which goal to relabel with.
In this work, accepted at ICML, we propose instead simply Learning Everything All at Once (LEO).
1/
0
0
6
372
Michael Beukman retweeted
CV has CNNs, NLP has transformers - what inductive bias does RL have? How can policies generalise to regions of the dataset suffering from poor transitions?
We motivate hierarchy by enabling distinct state-representations at different levels of the hierarchy @FLAIR_Ox @j_foerst
We motivate hierarchy by enabling distinct state-representations at different levels of the hierarchy @FLAIR_Ox @j_foerst
2
17
88
24,369
Check out this really cool population-based Alife work :)
Natural evolution's open-endedness leads to beautiful, complex emergent structures and self-organizing behavior 🌱✨. Replicating this in silico is famously hard 💻. Our paper points to a promising direction by evolving populations of competing neural cellular automata with sambard
⚠️⚠️flashing lights, rapid cuts, or strobe effects in this thread! 🚨🚨
1/n
0
0
6
1,393
Michael Beukman retweeted
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it ion: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it ion: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos).
157
641
3,574
505,437
Michael Beukman retweeted
1/ 🪩 Automating the discovery of new algorithms could unlock significant breakthroughs in ML research. But optimising agents for this research has been limited by too few tasks to learn from!
Introducing DiscoGen, a procedural generator of algorithm discovery tasks 🧵
Introducing DiscoGen, a procedural generator of algorithm discovery tasks 🧵
4
44
194
65,521
10/ Armed with these insights, we turn to the Kinetix benchmark—an open-ended universe of 2D physics based tasks. We scale to 1 million parallel environments, and show that this leads to monotonic performance improvement for more than 1 trillion timesteps without stagnating 🚀
2
4
41
13,869
11/ Thanks to my great co-authors @khimya, Zeyu Zheng, @wwdabney, @j_foerst, @MichaelD1729 and @clarelyle!
Check out the paper here for more details: arxiv.org/abs/2603.06009.
Check out the paper here for more details: arxiv.org/abs/2603.06009.
1
9
92
5,773