Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is .
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Oct 13, 2026 · 6:26 PM UTC
32
129
1,665
164,118
We use Miles from @radixark as the RL framework, with @sgl_project samplers and a Megatron-LM trainer that updates the policy in BF16 on B200 GPUs. There are two setups:
1. Mixed samplers that span six kinds of accelerators and three precisions (BF16, FP8 and NVFP4).
2. at run only on B200s with BF16 weights and a BF16 KV cache.
1. Mixed samplers that span six kinds of accelerators and three precisions (BF16, FP8 and NVFP4).
2. at run only on B200s with BF16 weights and a BF16 KV cache.
1
6
113
15,522
Why not true on-policy RL? Full alignment of the sampler with the trainer is expensive. So why not align only certain aspects? Although the magnitude of the mismatch is lowered, it still continues to grow. Off-policy correction fixes this growth, which we argue matters more.
1
1
39
5,356
@modal Do you have some real world costs you can discuss for training models in this way?
0
0
1
907
@modal 15% cheaper and it still matches the homogeneous samplers. thats the part that got me honestly
0
0
0
14
@modal mixing b200s with old h100s and still converging is a nice way to stop wasting hardware
0
0
0
261
@modal Homogeneous clusters were never physics. Just an expensive habit.
0
0
0
688
@modal I’m amazed what our training team can stitch together.
1
0
19
1,316
@modal heterogeneous rollout workers add capacity, but stragglers can erase gains without strict scheduling
0
0
0
134
@modal This is mad impressive. Same quality, lower bill or more experiments, longer runs and higher sample counts for the same budget. Wild
0
0
2
1,022
@modal 4.7× higher sampler–trainer KL and still on par with homogeneous samplers is the number that stands out
did most of that gap come from the NVFP4 samplers, or was it spread across accelerators? curious how many NVFP4 tokens masked importance sampling ends up throwing away
did most of that gap come from the NVFP4 samplers, or was it spread across accelerators? curious how many NVFP4 tokens masked importance sampling ends up throwing away
0
0
0
12
@modal homogeneous rl nodes throw away the inference work already done on older accelerators. that waste is why i would try a mixed fleet before another rack of the newest cards
0
0
0
10
@modal the limitations section gives average staleness of about 4. do you also have the tail broken down by sampler group and trajectory length? that would help show how far the uncapped-staleness result extends when slower workers or longer episodes fall behind.
0
0
0
113
@modal Throwing away non-homogeneous inference capacity for RL stability is expensive. Matching homogeneous performance at 15% lower cost with off-policy correction is the part I'd copy first.
0
0
0
4
@modal How did the founder of Modal had the idea for the company? Anywhere I can read your origin story?
0
0
0
71
@modal Asynchronous mixed-accelerator sampling sounds like the practical unlock here. The key question is whether it also makes agentic coding runs reproducible across hardware.
0
0
0
60
@modal The mixed setup is the part I want to try. It shows how much good scheduling can matter before adding more hardware.
0
0
0
59
@modal Very cool ! (I’ve done post-training over p2p networks with consumer GPUs in the past)
0
0
0
198
@modal Most agent calls are classify, route, or extract, not free-form generation. Constrained decoding at that price fits that slice better than burning a frontier model on every ticket.
0
0
0
4
@modal m8, last week tried running vocals thru a mix of old and new GPUs. sounded like a choir of dying fax machines. yeah? saved cash but the artifacts... hah, guess that’s the tax for not splurging on shiny new hardware. ... anyway
0
0
0
100
Today, @via_ai_ is coming out of stealth to provide enterprises frontier AI coding at half the cost.
We are a team of Olympiad camp qualifiers (with multiple gold medals), researchers, and repeat founders from Stanford and IIT.
Via is backed by @jeffawilke (ex-CEO Amazon anikgupta (ex-Uber CPO), @shaileshrao (ex-Google MD), and other legends from Anthropic, Apple, Snowflake, and Twitter.
@ShrayAlag and I left Stanford CS to build Codegraph, an LLM-optimized compression algorithm. Codegraph put Via at the top of SWE-Bench Verified for several months of 2025, ahead of Anthropic, OpenAI, Cognition, and Cursor (single model, no test-time compute).
Now, we’re bringing this research, alongside our new Umami agent architecture, to market. Via integrates in 10 seconds with any CLI / IDE and runs within your enterprise’s VPC.
On benchmarks, Via and Via US (US-only models) set a new standard of performance:
- On DeepSWE v1.1, Claude Code (Opus 5.5) is 306% the cost of Via US. Via US is equally accurate.
• Devin Fusion is 244% the cost of Via US, Cursor Router is 294%.
- On Terminal Bench 4.0, Claude Code (Opus 5.5) is 150% the cost of Via. Via is more accurate.
• Devin Fusion is 155% the cost of Via, Cursor Router is 134%.
We are in production at a public healthcare company, with trials underway at several of the largest tech companies in Silicon Valley.
We have room for 5 more enterprises before the end of the year. DM me or email pilot@via.dev and we’ll get back to you very soon.
See the comments for how Via would deploy for your team.
We are a team of Olympiad camp qualifiers (with multiple gold medals), researchers, and repeat founders from Stanford and IIT.
Via is backed by @jeffawilke (ex-CEO Amazon anikgupta (ex-Uber CPO), @shaileshrao (ex-Google MD), and other legends from Anthropic, Apple, Snowflake, and Twitter.
@ShrayAlag and I left Stanford CS to build Codegraph, an LLM-optimized compression algorithm. Codegraph put Via at the top of SWE-Bench Verified for several months of 2025, ahead of Anthropic, OpenAI, Cognition, and Cursor (single model, no test-time compute).
Now, we’re bringing this research, alongside our new Umami agent architecture, to market. Via integrates in 10 seconds with any CLI / IDE and runs within your enterprise’s VPC.
On benchmarks, Via and Via US (US-only models) set a new standard of performance:
- On DeepSWE v1.1, Claude Code (Opus 5.5) is 306% the cost of Via US. Via US is equally accurate.
• Devin Fusion is 244% the cost of Via US, Cursor Router is 294%.
- On Terminal Bench 4.0, Claude Code (Opus 5.5) is 150% the cost of Via. Via is more accurate.
• Devin Fusion is 155% the cost of Via, Cursor Router is 134%.
We are in production at a public healthcare company, with trials underway at several of the largest tech companies in Silicon Valley.
We have room for 5 more enterprises before the end of the year. DM me or email pilot@via.dev and we’ll get back to you very soon.
See the comments for how Via would deploy for your team.
63
32
245
73,637