Pinned Tweet
Highly recommended read.
It's an absolute masterclass in building a modern harness and enabling more autonomous agent systems.
This is the best write-up I have read on how I have been working with my agent teams powered by my custom harness.
In simple terms, you want a design that lets the agent operate continually and stay productive across tasks.
This requires a coordination-execution architecture. This is possible now, as LLM agents have gotten better at coordination and maintaining persistent context.
Once you get the core architecture right, you can explore proactive components (event triggering, acting ahead, etc.) and interesting ways to communicate with agents seamlessly (e.g., an asynchronous ReAct Loop).
On top of that, the design scales through state management and sandboxes. Target a design that separates state, compute, and environment. And build something that can persist and is always available across devices.
Btw, you can do this with other existing harnesses too. It doesn't have to be DeepSeek Harness.
As agent capabilities improve, you should find new ways to interact with them and avoid getting in the agents' way.
It's an absolute masterclass in building a modern harness and enabling more autonomous agent systems.
This is the best write-up I have read on how I have been working with my agent teams powered by my custom harness.
In simple terms, you want a design that lets the agent operate continually and stay productive across tasks.
This requires a coordination-execution architecture. This is possible now, as LLM agents have gotten better at coordination and maintaining persistent context.
Once you get the core architecture right, you can explore proactive components (event triggering, acting ahead, etc.) and interesting ways to communicate with agents seamlessly (e.g., an asynchronous ReAct Loop).
On top of that, the design scales through state management and sandboxes. Target a design that separates state, compute, and environment. And build something that can persist and is always available across devices.
Btw, you can do this with other existing harnesses too. It doesn't have to be DeepSeek Harness.
As agent capabilities improve, you should find new ways to interact with them and avoid getting in the agents' way.
61
67
923
130,039
Introducing Hacker Papers!
I absolutely love this HackerTube idea from @karpathy.
So I took his prompt and similar data sources and asked my agent to build the arXiv papers version of it.
Absolute bangers there.
Find all 1K+ papers here: academy.dair.ai/papers/collect…
I absolutely love this HackerTube idea from @karpathy.
So I took his prompt and similar data sources and asked my agent to build the arXiv papers version of it.
Absolute bangers there.
Find all 1K+ papers here: academy.dair.ai/papers/collect…
Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
hackertube.org
- Excellent (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊
A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.
Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.
66
123
1,231
50,910
elvis retweeted
Highly recommended read.
It's an absolute masterclass in building a modern harness and enabling more autonomous agent systems.
This is the best write-up I have read on how I have been working with my agent teams powered by my custom harness.
In simple terms, you want a design that lets the agent operate continually and stay productive across tasks.
This requires a coordination-execution architecture. This is possible now, as LLM agents have gotten better at coordination and maintaining persistent context.
Once you get the core architecture right, you can explore proactive components (event triggering, acting ahead, etc.) and interesting ways to communicate with agents seamlessly (e.g., an asynchronous ReAct Loop).
On top of that, the design scales through state management and sandboxes. Target a design that separates state, compute, and environment. And build something that can persist and is always available across devices.
Btw, you can do this with other existing harnesses too. It doesn't have to be DeepSeek Harness.
As agent capabilities improve, you should find new ways to interact with them and avoid getting in the agents' way.
It's an absolute masterclass in building a modern harness and enabling more autonomous agent systems.
This is the best write-up I have read on how I have been working with my agent teams powered by my custom harness.
In simple terms, you want a design that lets the agent operate continually and stay productive across tasks.
This requires a coordination-execution architecture. This is possible now, as LLM agents have gotten better at coordination and maintaining persistent context.
Once you get the core architecture right, you can explore proactive components (event triggering, acting ahead, etc.) and interesting ways to communicate with agents seamlessly (e.g., an asynchronous ReAct Loop).
On top of that, the design scales through state management and sandboxes. Target a design that separates state, compute, and environment. And build something that can persist and is always available across devices.
Btw, you can do this with other existing harnesses too. It doesn't have to be DeepSeek Harness.
As agent capabilities improve, you should find new ways to interact with them and avoid getting in the agents' way.
61
67
923
130,039
Highly recommended read.
I've been bullish on the idea of personal malleable software.
Patrick calls this pattern “liquid software”. This is a new way to build software, especially personalized software.
I have been building my own harness this way, but this also has implications in a world of recursive self-improvement (RSI).
In simple terms, every request in the improvements flow could go to an agent, and the agent changes the application.
In some sense, the user and the application can co-evolve as needs change. This is working for me.
I use the same co-evolution pattern at @dair_ai to build our internal custom agent harness, which we have been developing for over a year.
We implement, experiment with, and evaluate research ideas and our own ideas, so the harness keeps evolving. In my experience, this lets us build novel agent interfaces and functionality, and scale with agents much faster than more rigid harnesses allow.
I agree that scaling and security are the main challenges. Maintainability and collaboration are the open questions once this pattern moves beyond personal tools.
For builders, this means you can ship a foundation and let real usage shape the product. Small improvements could come directly from what users need. Experimentation also gets easier, because you can try new workflows and behaviors without a big release cycle.
I believe some version of this will be important for RSI to deliver significant business value. That is why I have been pushing to build evals that measure economic outcomes in the real world.
This idea also connects to Jack’s earlier “From Hierarchy to Intelligence” x.com/jack/status/20…, which describes rebuilding Block as a mini-AGI. Customers and people inside the company interact directly with the company's intelligence layer, and the software changes in response.
As agents improve, you should start designing applications that have that adaptiveness built in. Today, this works well in personal software, but I expect organizations to slowly start adopting the idea to build production-grade software that serves real-world users.
I've been bullish on the idea of personal malleable software.
Patrick calls this pattern “liquid software”. This is a new way to build software, especially personalized software.
I have been building my own harness this way, but this also has implications in a world of recursive self-improvement (RSI).
In simple terms, every request in the improvements flow could go to an agent, and the agent changes the application.
In some sense, the user and the application can co-evolve as needs change. This is working for me.
I use the same co-evolution pattern at @dair_ai to build our internal custom agent harness, which we have been developing for over a year.
We implement, experiment with, and evaluate research ideas and our own ideas, so the harness keeps evolving. In my experience, this lets us build novel agent interfaces and functionality, and scale with agents much faster than more rigid harnesses allow.
I agree that scaling and security are the main challenges. Maintainability and collaboration are the open questions once this pattern moves beyond personal tools.
For builders, this means you can ship a foundation and let real usage shape the product. Small improvements could come directly from what users need. Experimentation also gets easier, because you can try new workflows and behaviors without a big release cycle.
I believe some version of this will be important for RSI to deliver significant business value. That is why I have been pushing to build evals that measure economic outcomes in the real world.
This idea also connects to Jack’s earlier “From Hierarchy to Intelligence” x.com/jack/status/20…, which describes rebuilding Block as a mini-AGI. Customers and people inside the company interact directly with the company's intelligence layer, and the software changes in response.
As agents improve, you should start designing applications that have that adaptiveness built in. Today, this works well in personal software, but I expect organizations to slowly start adopting the idea to build production-grade software that serves real-world users.
I'm now building an "improvements" feature into all of my personal vibecoded applications: a freeform text input that's hooked up to an agent that yolo implements whatever is submitted.
It's an interesting experience -- the activation energy barrier to sanding any rough edge or ature becomes extremely low. Someone else who uses one of the applications I created has already implemented 150 improvements over the course of a few days. This pattern feels like a new kind of tool -- liquid software? -- and every application that doesn't have this feature now has a slightly rigid and anachronistic flavor. Why can't I just tell it how I want it to be different?
Now, it's not obvious to me how this pattern can or should scale. Does it become unmaintainable after a while? (Maybe model intelligence improvement outstrips my feature appetite such that it's fine?) How does collaborative software work in this model? Can software designed this way be secure and reliable?
Whatever the larger questions about broader deployment, it definitely works great in a personal context. It reminds me a little of the visions of Lisp/Lisp machines, Smalltalk, and HyperCard -- systems that the user can take control of and modify. Those visions never quite worked, though: the vast majority of software today is not end-user modifiable in any useful way. So, while this "liquid" affordance is a very small touch, the way it changes one's relation to the software feels a bit transgressive and empowering in a Lickliderian and Engelbartian way.
32
5
85
10,695
Highly recommended paper from CMU and others on how foundation models get trained.
It fundamentally asks whether pretraining, midtraining, SFT, and RL really need to be separate stages.
They show that this separation isn't fundamental.
All of these stages can run under a algorithm, which they call unified training.
The key step treats suitable RL objectives as maximum-likelihood fitting to model-generated targets. RL generations and static data then share one loss and one trainer.
This changes the design problem. You now choose which data to train on, in what proportions and when. Staged training becomes one special case.
They first trained a 124M model from scratch on reasoning puzzles. The best unified runs beat staged training on the RL tasks. They also beat it on larger puzzles the model never saw.
Those runs reused the midtraining corpus up to 4 times. Staged training lost accuracy when midtraining ran past its best budget.
Under a fixed compute budget, unified training had the best accuracy at the smallest and largest budgets. Staged training led at intermediate budgets.
The real-world test used a 0.76B model on math RL, with 0% to 5% midtraining data mixed in. Math accuracy stayed similar while retention improved. Coding accuracy dropped as the mix grew.
Continual learning gave the clearest result. After math training, all unified models beat staged training on final coding accuracy, despite starting lower. The 5% mix did best.
The experiments are small-scale. The authors have not found a mixture or schedule that works best across the board.
arxiv.org/abs/2610.40564
Chat with Paper: academy.dair.ai/papers/trainin…
It fundamentally asks whether pretraining, midtraining, SFT, and RL really need to be separate stages.
They show that this separation isn't fundamental.
All of these stages can run under a algorithm, which they call unified training.
The key step treats suitable RL objectives as maximum-likelihood fitting to model-generated targets. RL generations and static data then share one loss and one trainer.
This changes the design problem. You now choose which data to train on, in what proportions and when. Staged training becomes one special case.
They first trained a 124M model from scratch on reasoning puzzles. The best unified runs beat staged training on the RL tasks. They also beat it on larger puzzles the model never saw.
Those runs reused the midtraining corpus up to 4 times. Staged training lost accuracy when midtraining ran past its best budget.
Under a fixed compute budget, unified training had the best accuracy at the smallest and largest budgets. Staged training led at intermediate budgets.
The real-world test used a 0.76B model on math RL, with 0% to 5% midtraining data mixed in. Math accuracy stayed similar while retention improved. Coding accuracy dropped as the mix grew.
Continual learning gave the clearest result. After math training, all unified models beat staged training on final coding accuracy, despite starting lower. The 5% mix did best.
The experiments are small-scale. The authors have not found a mixture or schedule that works best across the board.
arxiv.org/abs/2610.40564
Chat with Paper: academy.dair.ai/papers/trainin…
45
27
222
24,152
elvis retweeted
Learn to build evals, folks!
You are not RSI-pilled enough if you don't know how to properly measure the thing you want to improve.
It doesn't matter if it's a model, complex codebase, or a business function. This is extremely hard in the real world.
You ever wonder why most of what AI is great at today is around verifiable problems?
The open-ended world requires a completely different approach to improving and measuring models.
We need to go back to hardcore science and evals. Enough of this short-term benchmaxxing.
This space offers so much opportunity. We are doing research and building evals in this space @dair_ai. I am currently chatting with folks building in this space too. Feel free to reach out for potential collab opportunities or just share perspectives.
You are not RSI-pilled enough if you don't know how to properly measure the thing you want to improve.
It doesn't matter if it's a model, complex codebase, or a business function. This is extremely hard in the real world.
You ever wonder why most of what AI is great at today is around verifiable problems?
The open-ended world requires a completely different approach to improving and measuring models.
We need to go back to hardcore science and evals. Enough of this short-term benchmaxxing.
This space offers so much opportunity. We are doing research and building evals in this space @dair_ai. I am currently chatting with folks building in this space too. Feel free to reach out for potential collab opportunities or just share perspectives.
89
14
237
16,646
elvis retweeted
Brilliant paper on agent memory from FAIR at Meta.
(bookmark it)
A single global history is the usual way to let agents learn from past runs. When tasks are interleaved, it gains almost nothing over no memory at all, an average of +0.015 in normalized reward.
This work al Learning. Past trajectories are grouped into clusters, and each cluster keeps its state in one memory system from a fixed library. Before each run, an LLM router reuses a cluster or creates a new one.
They tested it on a stream mixing 301 CL-Bench instances with 97 banking instances from τ³-Bench, across four language models.
Routed CL raised reward over the global history by 15% to 30% of the maximum attainable reward. Estimated API cost fell by 65% to 72%.
The runs were sequential, so parallel agents updating one cluster remain untested.
If your agents handle mixed work, it might help to split their memory into clusters before adding a bigger memory system.
arxiv.org/abs/2610.38796
Chat with Paper: academy.dair.ai/papers/routed-…
(bookmark it)
A single global history is the usual way to let agents learn from past runs. When tasks are interleaved, it gains almost nothing over no memory at all, an average of +0.015 in normalized reward.
This work al Learning. Past trajectories are grouped into clusters, and each cluster keeps its state in one memory system from a fixed library. Before each run, an LLM router reuses a cluster or creates a new one.
They tested it on a stream mixing 301 CL-Bench instances with 97 banking instances from τ³-Bench, across four language models.
Routed CL raised reward over the global history by 15% to 30% of the maximum attainable reward. Estimated API cost fell by 65% to 72%.
The runs were sequential, so parallel agents updating one cluster remain untested.
If your agents handle mixed work, it might help to split their memory into clusters before adding a bigger memory system.
arxiv.org/abs/2610.38796
Chat with Paper: academy.dair.ai/papers/routed-…
40
40
257
17,311
Learn to build evals, folks!
You are not RSI-pilled enough if you don't know how to properly measure the thing you want to improve.
It doesn't matter if it's a model, complex codebase, or a business function. This is extremely hard in the real world.
You ever wonder why most of what AI is great at today is around verifiable problems?
The open-ended world requires a completely different approach to improving and measuring models.
We need to go back to hardcore science and evals. Enough of this short-term benchmaxxing.
This space offers so much opportunity. We are doing research and building evals in this space @dair_ai. I am currently chatting with folks building in this space too. Feel free to reach out for potential collab opportunities or just share perspectives.
You are not RSI-pilled enough if you don't know how to properly measure the thing you want to improve.
It doesn't matter if it's a model, complex codebase, or a business function. This is extremely hard in the real world.
You ever wonder why most of what AI is great at today is around verifiable problems?
The open-ended world requires a completely different approach to improving and measuring models.
We need to go back to hardcore science and evals. Enough of this short-term benchmaxxing.
This space offers so much opportunity. We are doing research and building evals in this space @dair_ai. I am currently chatting with folks building in this space too. Feel free to reach out for potential collab opportunities or just share perspectives.
89
14
237
16,646
Banger paper from IBM on decision gates for agents.
They add a runtime gate to tool-using agents and keep the models and tools the same.
JevAct is a check that sits between the agent and its tools. An LLM first turns the domain policy into code checks and yes/no questions. call runs, the code checks the facts in the tool records, and a Jev-like decision model answers the questions about the conversation. If the call fails a check, it goes back to the agent with feedback so the agent can revise it.
On τ²-bench Telecom, Opus 4.6's success across all three runs rose from 39.5% to 80.7%.
On Telecom, all eight agents improved by 16.9 points on average in three-run success. All nine improved on BFCL. Airline results were smaller and mixed.
They show that the gains come from the harness.
arxiv.org/abs/2610.38620
Chat with Paper: academy.dair.ai/papers/think-b…
They add a runtime gate to tool-using agents and keep the models and tools the same.
JevAct is a check that sits between the agent and its tools. An LLM first turns the domain policy into code checks and yes/no questions. call runs, the code checks the facts in the tool records, and a Jev-like decision model answers the questions about the conversation. If the call fails a check, it goes back to the agent with feedback so the agent can revise it.
On τ²-bench Telecom, Opus 4.6's success across all three runs rose from 39.5% to 80.7%.
On Telecom, all eight agents improved by 16.9 points on average in three-run success. All nine improved on BFCL. Airline results were smaller and mixed.
They show that the gains come from the harness.
arxiv.org/abs/2610.38620
Chat with Paper: academy.dair.ai/papers/think-b…
98
43
250
19,707
Space Bunny was @MiniMax_AI M3.1 Flash all along. 🐰🎃
So I put it to work in MiniMax Code. One prompt gave me a playable space arcade game in 17 minutes.
The prompt asked for a bunny in a small spaceship, three lives, increasingly hard levels, and keyboard and touch controls. to run the game, playtest it, and fix any bugs it found.
M3.1 Flash wrote its own playtest suite. It ran 29 checks on desktop and on an emulated iPhone, and all 29 passed.
Great to see the model checking its own work.
The whole build cost about 15 cents in tokens at API prices ($0.15 per 1M input, $0.60 per 1M output).
Space Bunny now has a Halloween version. The bunny flies under a giant harvest moon, blasts glowing jack-o'-lanterns, and collects carrots. The levels add power-ups, combos, and meteor storms.
Play it here: 24erte7df6rjc.space.minimax.io
So I put it to work in MiniMax Code. One prompt gave me a playable space arcade game in 17 minutes.
The prompt asked for a bunny in a small spaceship, three lives, increasingly hard levels, and keyboard and touch controls. to run the game, playtest it, and fix any bugs it found.
M3.1 Flash wrote its own playtest suite. It ran 29 checks on desktop and on an emulated iPhone, and all 29 passed.
Great to see the model checking its own work.
The whole build cost about 15 cents in tokens at API prices ($0.15 per 1M input, $0.60 per 1M output).
Space Bunny now has a Halloween version. The bunny flies under a giant harvest moon, blasts glowing jack-o'-lanterns, and collects carrots. The levels add power-ups, combos, and meteor storms.
Play it here: 24erte7df6rjc.space.minimax.io
31
4
38
9,766
Build your own harness, folks!
Stop complaining about companies modifying harnesses however they want.
It's theirs, not yours!
Instead, build your own.
This isn't just about implementation disagreements; as models get more capable, the harness will keep adding constraints (some so subtle that not even the brightest will catch them).
I hate that idea. It's a terrible position for a user.
I prefer building my own and controlling what I can unlock with it and the models that power it.
Guardrails, capabilities, behaviors, efficiencies, patches, and constraints are not things you want to offload and wait for others to build for you. They're integral to how you produce value with agent harnesses.
As long as models need harnesses, you need full control over them.
That's why many full-stack AI companies are developing their own harnesses.
Stop complaining about companies modifying harnesses however they want.
It's theirs, not yours!
Instead, build your own.
This isn't just about implementation disagreements; as models get more capable, the harness will keep adding constraints (some so subtle that not even the brightest will catch them).
I hate that idea. It's a terrible position for a user.
I prefer building my own and controlling what I can unlock with it and the models that power it.
Guardrails, capabilities, behaviors, efficiencies, patches, and constraints are not things you want to offload and wait for others to build for you. They're integral to how you produce value with agent harnesses.
As long as models need harnesses, you need full control over them.
That's why many full-stack AI companies are developing their own harnesses.
Another reason why you wanna use your own harness and not one where companies change what you can or can’t do at will.
46
12
93
11,553