The session will examine how TorchPass coordinates distributed ranks at a safe NCCL boundary, captures CPU and GPU execution state, migrates affected processes and resumes without rolling back the entire job.

Attendees will leave with a practical framework for:

• Turning GPU, ry into remediation decisions
• Integrating transparent migration with Kubernetes and Slurm/Slinky
• Measuring recovery latency, exact-step resume rate, recomputed GPU-hours and job goodput
• Deciding when to migrate, when to checkpoint and whether flexible capacity is viable for stateful training

The results are concrete. In Clockwork’s 64-H200 failure-injection benchmark, 𝑇𝑜𝑟𝑐ℎ𝑃𝑎𝑠𝑠 𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝑑 𝑡𝑟𝑎𝑖𝑛𝑖𝑛𝑔 𝑖𝑛 𝑟𝑜𝑢𝑔ℎ𝑙𝑦 ℎ𝑎𝑙𝑓 𝑡ℎ𝑒 𝑡𝑖𝑚𝑒 𝑜𝑓 𝑐ℎ𝑒𝑐𝑘𝑝𝑜𝑖𝑛𝑡-𝑎𝑛𝑑-𝑟𝑒𝑠𝑡𝑎𝑟𝑡, 𝑤𝑖𝑡ℎ 𝑧𝑒𝑟𝑜 𝑙𝑜𝑠𝑡 𝑠𝑡𝑒𝑝𝑠 𝑣𝑒𝑟𝑠𝑢𝑠 869.

@SemiAnalysis_ also tested TorchPass independently on a 64-H200 gpt-oss-120B workload. Jordan Nanos called it “the fastest and most efficient fault-tolerant” approach tested. Dylan Patel highlighted live migration that “keeps MFU high.”

If you build, run or lead distributed AI training, put these 20 minutes on your calendar.

📅 Thursday, September 17, 2026
⏰ 11:00–11:20 a.m. PT
📍 Data & Models Track, AI Infra Summit
🏢 Santa Clara Convention Center
🎤 Clark Zinzow, Director of Engineering, Together AI
🎤 Prashanth Thinakaran, Distinguished AI Infrastructure Engineer, Clockwork Systems, Inc.

Agenda: lnkd.in/g3hTTfBV

#AIInfrastructure #DistributedTraining #GPUComputing #FaultTolerance