BackboardIO retweeted
We're so bad at comms, but we're getting better. dev.to/jon_at_backboa…
0
2
5
75
BackboardIO retweeted
We beat DeepInfer head to head, more to come, but hope @SemiAnalysis_ notices @BackboardIO soon, even though we’re a Canadian lab without hundreds of millions of dollars and thousands of contributors. A win is a win.
0
3
6
541
More useful work from the same GPUs.
Using Qwen3.8-Flash-Next on eight H100 GPUs. Backboard managed up to 2.2× higher output throughput per GPU than the SGLang results on SemiAnalysis AgentX.
This was done at BF16 versus SGLang at FP8. Better throughput without sacrificing model precision.
Using Qwen3.8-Flash-Next on eight H100 GPUs. Backboard managed up to 2.2× higher output throughput per GPU than the SGLang results on SemiAnalysis AgentX.
This was done at BF16 versus SGLang at FP8. Better throughput without sacrificing model precision.
0
2
3
133
Waiting for the first token shouldn't slow down an AI agent.
Backboard offered up to 13.6× faster p90 time to first token than the SGLang results on SemiAnalysis AgentX, running Qwen3.8-Flash-Next on eight H100 GPUs.
And Backboard did it at BF16 precision, versus SGLang's FP8.
Backboard offered up to 13.6× faster p90 time to first token than the SGLang results on SemiAnalysis AgentX, running Qwen3.8-Flash-Next on eight H100 GPUs.
And Backboard did it at BF16 precision, versus SGLang's FP8.
0
1
2
91
Agentic AI needs more than fast tokens. It needs responsive tokens.
On SemiAnalysis AgentX, Backboard delivered up to 12.8× higher p90 interactivity than SGLang on Qwen3.8-Flash-Next across eight H100 GPUs. That’s quicker starts, smoother answers, and a more consistent ran BF16. The SGLang reference ran FP8.
On SemiAnalysis AgentX, Backboard delivered up to 12.8× higher p90 interactivity than SGLang on Qwen3.8-Flash-Next across eight H100 GPUs. That’s quicker starts, smoother answers, and a more consistent ran BF16. The SGLang reference ran FP8.
0
1
4
364
More concurrent requests shouldn't mean giving up throughput.
On Qwen3.8-27B, Backboard maintained up to a 1.26× output-throughput advantage over the fastest results from vLLM 0.33 and SGLang, with up to 8K inputs and concurrency up to 64 users.
All on a single RTX PRO 6000 Blackwell GPU.
On Qwen3.8-27B, Backboard maintained up to a 1.26× output-throughput advantage over the fastest results from vLLM 0.33 and SGLang, with up to 8K inputs and concurrency up to 64 users.
All on a single RTX PRO 6000 Blackwell GPU.
0
1
3
88
Your inference engine should be consistently fast, not just in a few conditions.
Backboard won 102 of 104 measured InferenceX metrics against SGLang on Qwen3.8-27B using a lone RTX PRO 6000 Blackwell GPU. That includes throughput, time to first token, and generation speed.
This nds up to scrutiny.
Backboard won 102 of 104 measured InferenceX metrics against SGLang on Qwen3.8-27B using a lone RTX PRO 6000 Blackwell GPU. That includes throughput, time to first token, and generation speed.
This nds up to scrutiny.
0
2
3
113
You want your AI inference to win virtually every time. Ours does.
Backboard won every measured InferenceX metric against vLLM 0.33 on Qwen3.8-27B, across test configurations covering up to 8K inputs and 64-user concurrency.
That’s with one RTX PRO 6000 Blackwell GPU, BF16, and tion.
Backboard won every measured InferenceX metric against vLLM 0.33 on Qwen3.8-27B, across test configurations covering up to 8K inputs and 64-user concurrency.
That’s with one RTX PRO 6000 Blackwell GPU, BF16, and tion.
0
2
3
99
At ~1.4s P90 time to first token:
Backboard: $6.62/GPU-hour
SGLang: ~$2.75/GPU-hour
~2.4× modeled token revenue. Same 8× H100, same Qwen3.8 Flash Next model.
Makin H100's Great Again. (<-- can we still say that?)
@SemiAnalysis_ @dylan522p
@RobImbeault @jon_curio
Backboard: $6.62/GPU-hour
SGLang: ~$2.75/GPU-hour
~2.4× modeled token revenue. Same 8× H100, same Qwen3.8 Flash Next model.
Makin H100's Great Again. (<-- can we still say that?)
@SemiAnalysis_ @dylan522p
@RobImbeault @jon_curio
0
1
6
351
RT @RobImbeault: Fresh out of the oven. We were told to use the AgentX by @SemiAnalysis_ benchmark so we ran it against the current best s…
0
1
0
1
AI workloads don't always arrive in a steady line. Sometimes hundreds of requests hit at once.
In bursts, Backboard delivered up to 1.78× higher aggregate output throughput and 8.56× faster median time to first token than vLLM, running Qwen3.6 on a single RTX PRO 6000 GPU.
up when demand spikes.
In bursts, Backboard delivered up to 1.78× higher aggregate output throughput and 8.56× faster median time to first token than vLLM, running Qwen3.6 on a single RTX PRO 6000 GPU.
up when demand spikes.
0
1
3
143
Throughput isn't the whole story for AI inference. Users also care about how long they wait for a response.
On one RTX PRO 6000 GPU, Backboard’s median time to first token in Qwen3.6 was 5.75× faster than vLLM on Qwen3.6 under sustained traffic.
More capacity, less waiting.
On one RTX PRO 6000 GPU, Backboard’s median time to first token in Qwen3.6 was 5.75× faster than vLLM on Qwen3.6 under sustained traffic.
More capacity, less waiting.
2
0
3
122
More inference performance doesn't have to mean more GPUs.
On a single NVIDIA RTX PRO 6000 Blackwell GPU driving Qwen3.6, Backboard delivered 1.57×–1.95× the aggregate output throughput of vLLM across concurrency levels from C8 to C256.
More output from the same hardware.
On a single NVIDIA RTX PRO 6000 Blackwell GPU driving Qwen3.6, Backboard delivered 1.57×–1.95× the aggregate output throughput of vLLM across concurrency levels from C8 to C256.
More output from the same hardware.
0
1
4
153
Speed or quality? Today, if you serve AI models, improving one means giving up some of the other.
To make models fast enough for heavy traffic, teams compress them, and that costs some accuracy. Backboard removes that trade-off.
We ran the full, uncompressed Qwen3.8-Flash-Next on Backboard against a compressed version of the same model on a popular serving engine. The test was @SemiAnalysis_ independent benchmark.
The full model on Backboard:
→ Started answering 2.9x sooner
→ Served 24% more requests on the same GPUs
You run the full model, get more out of the hardware you already pay for, and have your cake and eat it too.
Try it with your own models and traffic: backboard.io/blog/qwen-flas…
To make models fast enough for heavy traffic, teams compress them, and that costs some accuracy. Backboard removes that trade-off.
We ran the full, uncompressed Qwen3.8-Flash-Next on Backboard against a compressed version of the same model on a popular serving engine. The test was @SemiAnalysis_ independent benchmark.
The full model on Backboard:
→ Started answering 2.9x sooner
→ Served 24% more requests on the same GPUs
You run the full model, get more out of the hardware you already pay for, and have your cake and eat it too.
Try it with your own models and traffic: backboard.io/blog/qwen-flas…
0
1
9
15,701
Backboard’s full 16-bit Qwen p90 first token was 1.21s vs. 3.50s for the InferenceX SGLang FP8 baseline.
That’s 2.9X faster to first token and 24% more throughput.
It's a new way of delivering inference. Read more below.
That’s 2.9X faster to first token and 24% more throughput.
It's a new way of delivering inference. Read more below.
2
2
4
251
BackboardIO retweeted
What happens when the workload keeps coming?
In our GLM 5.3 sustained test at 32 concurrent users:
@BackboardIO: 370 output tokens/sec
vLLM: 74.9
4.94X aggregate output throughput.
Full model. No Quant!
In our GLM 5.3 sustained test at 32 concurrent users:
@BackboardIO: 370 output tokens/sec
vLLM: 74.9
4.94X aggregate output throughput.
Full model. No Quant!
2
3
5
252
BackboardIO retweeted
Our latest GLM 5.3 full-model BF16 test:
@BackboardIO: 165.5 output tokens/sec
vLLM: 63.5 output tokens/sec
2.61X aggregate output throughput.
Same 96 workflows. Same request bodies. 32 concurrent users.
@BackboardIO: 165.5 output tokens/sec
vLLM: 63.5 output tokens/sec
2.61X aggregate output throughput.
Same 96 workflows. Same request bodies. 32 concurrent users.
0
3
6
420
Backboard just got a little more powerful. ⚡
35 new fal.ai media tools.
More reliable video generation.
Better usage reporting.
Plus a few product improvements.
Read the full update → backboard.io/changelog/new-…
35 new fal.ai media tools.
More reliable video generation.
Better usage reporting.
Plus a few product improvements.
Read the full update → backboard.io/changelog/new-…
0
1
0
190