You can now run FrontierMemory to measure how well your model or harness improves with experience.
We’re also now accepting submissions to the official leaderboard.
Code and submission instructions are available at:
github.com/letta-ai/front…
We’re also now accepting submissions to the official leaderboard.
Code and submission instructions are available at:
github.com/letta-ai/front…
4
14
61
3,432
Honored to have been a featured ecosystem partner by @JensenHuang at GTC Berlin!
We're excited to partner with @nvidia on research towards developing agents that continually learn and self-improve.
We're excited to partner with @nvidia on research towards developing agents that continually learn and self-improve.
5
5
32
2,462
Letta retweeted
Recommended reading and a super-interesting benchmark for measuring continual learning capabilities through memory.
Agent memory is largely unsolved. As agents advance in capabilities like long-horizon work, it's unclear which models or harnesses work best, or which memory e.
Opus 5.5 at the top is not a surprise. But it's exciting that open-weight models like GLM-5.3 perform so well on memory generation.
My big question is how this (better memory systems and persistent agent capabilities) translates into more performant, efficient RSI agents and harnesses.
Agent memory is largely unsolved. As agents advance in capabilities like long-horizon work, it's unclear which models or harnesses work best, or which memory e.
Opus 5.5 at the top is not a surprise. But it's exciting that open-weight models like GLM-5.3 perform so well on memory generation.
My big question is how this (better memory systems and persistent agent capabilities) translates into more performant, efficient RSI agents and harnesses.
Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance.
While existing memory benchmarks focus on recall (saturated via agentic search), FrontierMemory focuses on how agents learn from experience.
42
12
108
13,561
Letta retweeted
Memory is becoming a core part of agent harnesses/products (e.g. Instinct, @Muse, @cognition Devin, @PrimeIntellect Prime Agent, @NousResearch Hermes, @openclaw, @LangChain Deep Agents, @Letta_AI) -- yet despite every agent claiming to "continually learn", there's no benchmark to actually measure that capability.
A big problem is that memory benchmarks today only measure recall (e.g. simple Q/A about past events). They don't measure if/how agents actually learn from experience to get better at work.
At the same time, benchmarks for coding/knowledge work consist of many isolated, unrelated tasks - so they also don't allow for measuring cross-task learning. The context needed for a task is fully specified up-front (very unrealistic).
In practice, both agents and humans work through sequences of many *related* tasks. Experience from past work provides context and learnings that apply to future work. This is why if you're a manager, you have to provide a lot more context when assigning tasks to a brand new employee or external contractor. Meanwhile, a great senior employee might not need any instructions at all.
With FrontierMemory, we adapt existing benchmark tasks for coding and knowledge work so that instead of all the necessary context being provided up-front and within the task, it has to be learned from past experience. This allows us to compare how well different models and harnesses learn over time.
A big problem is that memory benchmarks today only measure recall (e.g. simple Q/A about past events). They don't measure if/how agents actually learn from experience to get better at work.
At the same time, benchmarks for coding/knowledge work consist of many isolated, unrelated tasks - so they also don't allow for measuring cross-task learning. The context needed for a task is fully specified up-front (very unrealistic).
In practice, both agents and humans work through sequences of many *related* tasks. Experience from past work provides context and learnings that apply to future work. This is why if you're a manager, you have to provide a lot more context when assigning tasks to a brand new employee or external contractor. Meanwhile, a great senior employee might not need any instructions at all.
With FrontierMemory, we adapt existing benchmark tasks for coding and knowledge work so that instead of all the necessary context being provided up-front and within the task, it has to be learned from past experience. This allows us to compare how well different models and harnesses learn over time.
Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance.
While existing memory benchmarks focus on recall (saturated via agentic search), FrontierMemory focuses on how agents learn from experience.
31
15
215
16,280
Letta retweeted
FrontierMemory from @Letta_AI is the first benchmark to measure memory based on performance on downstream tasks.
Memory remains one of the biggest unsolved challenges in AI.
FrontierMemory ACTUALLY gives us a way to actually measure and compare memory capabilities across different harnesses and models.
Memory remains one of the biggest unsolved challenges in AI.
FrontierMemory ACTUALLY gives us a way to actually measure and compare memory capabilities across different harnesses and models.
Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance.
While existing memory benchmarks focus on recall (saturated via agentic search), FrontierMemory focuses on how agents learn from experience.
2
3
15
1,177
Letta retweeted
Truly capable agents must learn from experience, not merely recall facts. The FrontierMemory benchmark evaluates agent memory directly by measuring its impact on downstream task performance.
Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance.
While existing memory benchmarks focus on recall (saturated via agentic search), FrontierMemory focuses on how agents learn from experience.
10
7
55
3,451
FrontierMemory is a continuous benchmark. In our next release, we plan to add longer-horizon tasks that enable RSI from memory.
For more details see: frontiermemory.com
For more details see: frontiermemory.com
1
1
19
1,568
Thanks to our partners for providing early feedback and inference to support the benchmark @AnthropicAI @OpenAI @nvidia @databricks @Snowflake @baseten @FireworksAI_HQ
0
1
7
1,205
Letta retweeted
Hey, if you're still using @Letta_AI Classic like I am, you need an ADE (as Letta has EOL'd and retired the online version).
I'm maintaining the Sanctum branch of both Letta and the ADE here:
github.com/sanctumos/letta
github.com/sanctumos/ade
I'm maintaining the Sanctum branch of both Letta and the ADE here:
github.com/sanctumos/letta
github.com/sanctumos/ade
0
3
8
497
Letta Code now supports dynamic workflows to compose model-agnostic subagents and decision models. x.com/i/article/2105…
9
1
29
11,826
Letta retweeted
Using Muse to dispatch work to my @Letta_AI cloud agent, which can be orchestrated via MCP
MCP is the real A2A
MCP is the real A2A
4
1
13
2,150
Letta retweeted
It's AI Infra night at @Letta_AI HQ!
Here's Letta cofounder @sarahwooders showing off Letta, cloud sandboxes, Slack, and teleportation from her laptop to the cloud.
Here's Letta cofounder @sarahwooders showing off Letta, cloud sandboxes, Slack, and teleportation from her laptop to the cloud.
1
1
9
617
Letta retweeted
Mods in @Letta_AI are very cool, and they work on cloud agents too! (you just need to move the mod into the agent's memory so it becomes part of the agent-owned context)
letta.com/blog/introduci…
letta.com/blog/introduci…
1
2
11
1,318