The best inference config depends on your model, your hardware and your traffic, and all three keep changing. Settings that were ideal last week are rarely the right ones today.
Tonight at @MiniMax_AI's #SFTechWeek research night, Modular AI Performance Engineer Tyler Kenney is ng ML inference systems, exploring how we can teach inference systems to tune their own settings.
📍 UpScaleX, San Francisco
🗓 Thursday, Oct 8 · 5:30–8:30 PM PT
RSVP: partiful.com/e/DKY1zYjyuOvM…
Tonight at @MiniMax_AI's #SFTechWeek research night, Modular AI Performance Engineer Tyler Kenney is ng ML inference systems, exploring how we can teach inference systems to tune their own settings.
📍 UpScaleX, San Francisco
🗓 Thursday, Oct 8 · 5:30–8:30 PM PT
RSVP: partiful.com/e/DKY1zYjyuOvM…
Oct 8, 2026 · 3:47 PM UTC
1
3
15
1,247
@Modular @MiniMax_AI batch size is the hidden axis there because the sparsity that helps a single stream disappears once enough requests share a step so the best config for one user and for sixty are close to opposites
0
0
0
3