Vmax retweeted
while working on agentick (to be presented at NeurIPS'26), @creus_roger found that ASCII inputs seemed to really help LLMs
during his internship at @VmaxAI he pushed on this idea and came up with GlyphBench, which serves as a benchmark, a research tool, & post-training suite!
during his internship at @VmaxAI he pushed on this idea and came up with GlyphBench, which serves as a benchmark, a research tool, & post-training suite!
1/ 🎮 Can training on games improve LLM reasoning?
Introducing GlyphBench: 360+ game tasks for language-model RL.
We use it as a benchmark, a research tool, and a post-training suite.
🧵⬇️
0
4
30
2,375
11/ Led by @creus_roger, in collaboration with @Cote_Marc, @GlenBerseth and @pcastr at @Mila_Quebec & @UMontreal.
📄 Paper: arxiv.org/abs/2609.34214
💻 Code: github.com/VmaxAI/glyphbe…
🌐 Blog: vmax.ai/team/glyphbench
📄 Paper: arxiv.org/abs/2609.34214
💻 Code: github.com/VmaxAI/glyphbe…
🌐 Blog: vmax.ai/team/glyphbench
0
1
22
306
Vmax retweeted
We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge.
External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
11
42
314
27,337
Vmax retweeted
Come join us next week at our CoLM workshop:
Context Beyond the Window
We have some great speakers coming to speak on the opportunities, limitations and safety-considerations of current LLMs.
We will also be cohosting a social alongside Reflection!
luma.com/eocj5cm5
Context Beyond the Window
We have some great speakers coming to speak on the opportunities, limitations and safety-considerations of current LLMs.
We will also be cohosting a social alongside Reflection!
luma.com/eocj5cm5
3
5
59
3,858
Vmax retweeted
Want to train your SWE agents 2× faster 🏎️💨?
We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer.
Unlike other works that on strategy, we show that KV-streams can work with any compaction method in agentic tasks.
Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀
arxiv.org/pdf/2609.35750
🧵
We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer.
Unlike other works that on strategy, we show that KV-streams can work with any compaction method in agentic tasks.
Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀
arxiv.org/pdf/2609.35750
🧵
11
50
304
55,438
Nethack is a really underrated benchmark
astra playing NetHack:
- score: 48,978
- BALROG progress: 49.4%
- max dungeon depth: 17
- max experience level: 14
important note: this is a cherry-picked episode (best of 10), and Astra has been able to modify its own game-playing harness
no privileged information or unfair game modifications, but it uses tools such as pathfinding to previously visited landmarks and code-written trackers for previously observed entities. these offload a fair amount of low-level reasoning, while Astra remains responsible for choosing what to do at every step
imo, the ability of LLMs to autonomously create tools that amortize low-level control costs (and then interpret and use their outputs in-context) is remarkable, and one of the reasons i'm excited about LLM agents. like, we could run the same tool computations and... concatenate the outputs into a flat vector for an MLP... but it would still have to learn what those outputs mean and how to use them. with LLMs we get free hierarchies and natural separation of responsibilities...
anyway, i'm now running episodes with the default settings from BALROG for a more fair comparison
the auto-research programme continues. will report back!
full run below ⤵️
0
0
3
877
Vmax retweeted
Interesting details from @deepseek_ai on the benefits of scaling environment/task synthesis.
If you want to scale open-ended environment generation for your own use case, sign up to the @VmaxAI waitlist - link below
If you want to scale open-ended environment generation for your own use case, sign up to the @VmaxAI waitlist - link below
1
6
32
1,892
Vmax retweeted
Greatest product placement ever. Placing the Vmax name in the latest Spiderman movie. 👏
Congrats @VmaxAI
Congrats @VmaxAI
0
2
3
296
Vmax retweeted
Legendary discussions at the office. @rronak_ discussed applying and scaling up SDPO for continual learning in the real world to prevent context degradation and more
@kevingu gave a talk on learning from production traces for knowledge-work tasks that have no direct verifier: signal without ground truth and keep organizational context current as the work changes
@matthewjsargent explained how to stabilize asymmetric self play and new research fields in open ended learning
Thanks to all those who came out and asked great questions we’ll be hosting more of these soon!
@kevingu gave a talk on learning from production traces for knowledge-work tasks that have no direct verifier: signal without ground truth and keep organizational context current as the work changes
@matthewjsargent explained how to stabilize asymmetric self play and new research fields in open ended learning
Thanks to all those who came out and asked great questions we’ll be hosting more of these soon!
Most agents are static once deployed. Trained once, frozen, and brittle outside their training distribution.
We’re bringing together researchers and builders working on systems that keep learning, decompose problems into reusable pieces, and improve their own reasoning over evingu CTO of Thirdlayer, @rronak_ CEO of Trajectory, @matthewjsargent CEO of Vmax followed by open Q&A.
Join us! luma.com/l666hxiw
5
8
51
13,497