I got a 1T (trillion) parameter model running on my MacBook Pro.

Kimi-K2. 1.029T params.

~1 TB raw weights.
524 GB converted.

~1.7 tok/s.

Yesterday it was 671B. Today it's 1T.

Same laptop. Same M4 Max. No cloud.

When I say we: I mean Claude and me.

the fastest setting i tried for a code edit on my DGX Spark was 7.3x faster and broke the file. one stray line, every run. the setting that was 3.4x faster got it exactly right

same task as YASAKANI's 5090 post: rename one name in textwrap.py, return the whole 3.8-27B, greedy, thinking off, same speed math

no speculation, both 4-bit engines hit the same wall:
llama.cpp Q4_K_XL: 11.3 tok/s
vLLM NVFP4: 11.2

speculation on the edit, vLLM NVFP4:
MTP2: 26.3, exact
MTP4: 38.0, exact, byte-identical
n-gram: 82.0, wrong. Python won't parse the file

the n-gram file has one line the original never had: "_WHITESPACE_CHARS." on line 15, right above the real one. same spot in all 3 runs

n-gram speculation drafts 8 tokens at a time by copying earlier text. on a file that is mostly copy, 89.7% of drafts land, so it flies

greedy to be lossless. here it wasn't. MTP on the same vLLM build was exact. only checking the output caught it

the more the output copies its input, the more drafts land. MTP4 accepts 57% on a summary, 85% on fresh code, 99.8% on this edit: 2.25x -> 3.0x -> 3.39x. it tops out because it only drafts 4 tokens

YASAKANI's post checked that all three engines returned the same 4,711 tokens. that cspeed means nothing until the file parses

next on this box: llama.cpp with its own speculation on the same edit, then more engines

Qwen3.8-27B · vLLM 0.28 NVFP4 · llama.cpp 4801e3c56 · one DGX Spark · 3 runs per setting · llama.cpp plain decode from an earlier run on another prompt · GPU-only energy 2.98 -> 0.89 J/token with MTP4

apple limiting their own hardware

"An AI (gpt-6 astra) edited 0.016% of a 0.8B model's weights and its math score jumped 6x."

reproduced it with claude and measured the energy: 21.7 kJ → 2.6 kJ per correct answer.

asked a 0.8B model: 441 + 2(21)(19) + 361 = x

it got 1600, talked itself out of it, then wrote ." 1,428 times, until it hit the 32k token cap

edit 130k of its weights, no training, and it answers 1600 in 4k tokens

Qwen3.5-0.8B · MATH-500 · my DGX Spark
stock: 7.2%
edited: 44.0%
energy per correct answer: 8.4x less
21.7 kJ -> 2.6 kJ

the edit is from a paper out yesterday, Weight-Space Programming.

energy per correct answer is the number that moved most

per kWh at the GPU, the stock model gets 166 answers right. the edited one gets 1,390

energy per token barely moved: 0.051 -> 0.048 J. the edit does not make tokens cheaper. it stops the model wasting them. 429k tokens per cort loop is not one bad problem. 91% of the stock model's answers looped until the cap cut them off. edited: 28%

this is why i count joules per correct answer, not per token. per token, these two models look the same

how the edit works: a frontier model in the loop looked for the direction two MLP layers (15 and 19) write when the model starts repeating itself, then subtracted it six times over from what 64 neurons in each layer write. it needed a 50 forward-pass budget. with 10 or 25 passes 12-14%

they released the diff. i applied it, matched both edited tensors' hashes to their manifest, and ran stock and edited back to back: same 125 problems, same seed, same sampling

theirs: 7.9% -> 48.1%. mine: 7.2% -> 44.0%. both of theirs sit inside my 95% intervals. 47 problems flipped to solved, 1 went the other way

caveats: one sample per problem, on every 4th MATH-500 problem. energy is the Spark's GPU module, not the wall

Qwen3.5-0.8B BF16 · vLLM 0.28 · T0.6 top_p 0.95 top_k 20 presence 1.5 · 32,768 token cap · graded with math-verify 0.9.0 · DGX Spark GB10

256k-token prompt, then a 1,000-token answer:
DGX Spark: 12.7 min, 16.6 Wh
MacBook Pro M4 Max: 61.5 min, 39.1 Wh

4k-token prompt, same answer:
DGX Spark: 94 s, 1.3 Wh
MacBook Pro M4 Max: 72 s, 0.9 Wh

same Qwen3.8-27B file on both, same llama.cpp build, one stream, raw chip request into its two stages and put time and energy on one chart for each

reading the prompt: the Spark lands lower and further left at every depth. at 256k it gets to the first token 5.9x sooner on 2.5x less energy

writing the answer: the Mac lands lower and further left up to 128k, 1.2-1.9x sooner on 1.5-2.0x less energy. at 256k the Spark writes 1.1x faster and the Mac spends 7% less per token

a short prompt goes to the Mac because the answer is most of the work. a long prompt goes to the Spark because the reading is

and the shared GGUF is not the Spark's best format

256k prompt on the Spark, time to first token: NVFP4 in vLLM 5.6 min, FP8 6.2, BF16 7.1 (an August run, no power meter on it), the GGUF 9.9

decoding there, NVFP4 with 3-token speculation does 19.8 tok/s at 1.9 J per token, GGUF does 5.8 tok/s at 6.8 J

the Mac's fastest answer at 256k is a different model: the 35B-A3B MoE, 3B active, 17.3 tok/s at 1.8 J. a 3B-active token and a 27B token are not the same work, so it sits outside the head to head

the rule i ended up with, per request:

long prompt, short answer -> Spark
short prompt, long answer -> Mac
256k prompt -> Spark, at any answer length

where it flips, the answer length at which the two tie, joules | time:
512 context: 48 | 41 tokens
4k: 224 | 426
16k: 987 | 1,637
32k: 1,849 | 3,586
64k: 3,906 | 9,531
128k: 14.9k | 37.0k
256k: 182k | never. no 256k context has room for that answer

i drew a first version of this line yesterday. this one has the Spark on the same llama.cpp build as the Mac, so the numbers shifted

the caveats. these are raw chip watts with idle included: the Spark GPU module idles near 4.4 W, the Mac SoC near 0.1 W, and the SoC number leaves out the Mac's memory and fans. subtract idle and the Spark is 5% cheaper per decoded token at 256k. no wall plug on the Spark yet

Qwen3.8-27B UD-Q4_K_XL · llama-bench 4801e3c56 on both · f16 KV · DGX Spark GB10 vs M4 Max 128GB

223 decode runs: 0.6B to 35B on both boxes, across llama.cpp, MLX, vLLM and SGLang

raw chip power: 14 to 66 W
decode speed: 4.3 to 448 tok/s
energy per token: 0.066 to 9.4 J

power moves 5x. speed moves 100x. energy per token moves 142x, and speed alone explains 97% of that to arithmetic: joules per token is watts divided by tokens per second. the chip draws about the same whatever you run on it, so the token rate does the rest. joules per token is mostly a speedometer

reading the prompt is the same story: 222 prefill runs, 31 to 96 W at the chip, and speed explains 94% of the energy per prompt token

the cheapest token on my bench: Llama 3.2 1B on MLX at 512 context, 0.066 J
the most expensive: Qwen3.8-27B in BF16 on the Spark at 32k, 9.4 J

142x apart, and nobody swaps a 27B for a 1B to save power. the tokens are not the same work

that is the catch with joules per token. it he answer. it is also why a 35B MoE with 3B active looks 3.3-4.5x cheaper per token than the dense 27B on the same Mac

the number i actually want is joules per correct answer

the first one i have: Qwen3.6-35B-A3B on all 164 HumanEval problems, greedy, on the Mac. 154 solved, 41,294 tokens, 53.9 kJ. that is 350 J per solved problem, measured at the whole machine, not at the chip like the charts ne-tune of the same model in a 9% smaller file, solved 152 (a 2-problem gap is within noise) and came out at 336 J per solve, 4% less. it also wrote 5% faster

that is the point of the unit: speed and accuracy end up in one number

next: the same 164 problems from 0.6B up to 27B, so the cheap tokens have to show what they buy

scatters: raw chip watts (Spark GPU module, Mac SoC) · Mac llama.cpp 4801e3c56 and MLX · Spark llama.cpp 4801e3c56 and dde879bc4, vLLM plus a DFlash2 fork, SGLang · HumanEval: whole machine · M4 Max 128GB · DGX Spark GB10

DGX Spark vs Mac

performance as a function of time & energy

64k of context, 2k-token answer:
DGX Spark: 5 min, 17 kJ
MacBook Pro M4 Max: 10 min, 23 kJ

4k prompt, 8k-token answer:
DGX Spark: 12 min, 33 kJ
MacBook Pro M4 Max: 7 min, 19 kJ

same Qwen3.8-27B file, llama.cpp on bothwins one request outright, on the clock and on joules

so i found the line. answer length where the two tie, joules | time:

512 context: 55 | 38 tokens
4k: 248 | 457
16k: 1,071 | 1,809
32k: 1,971 | 4,071
64k: 4,640 | 11.5k
128k: 17.5k | 58.8k
256k: never | never. the Spark wins every request

the Spark reads 2.2-2.8x cheaper and 2.7-6.5x faster at every depth
the Mac writes 1.6-1.9x cheaper up to 64k and ~1.7x faster up to 32k. by 256k the Spark writes faster too

at 64k the Spark starts answering in 88 s. the Mac takes 7.5 min

64k context with an 8k answer is the odd one: the Spark finishes first, the Mac spends fewer joules

chip watts only, Spark GPU module vs Mac SoC. no wall plug yet. and the Spark ran llama-server while the Mac ran llama-bench, matched rerun queued

Qwen3.8-27B UD-Q4_K_XL · llama.cpp · f16 KV · flash attn · DGX Spark GB10 vs M4 Max 128GB

fine tuning on the DGX Spark -

opssat anomaly: 26 -> 96 (27B: 97)
clinc intents: 59 -> 99 (98)
financial phrasebank: 51 -> 91 (93)
typed decisions: 59 -> 82 (82)
jigsaw toxicity: 80 -> 97 (97)
chaosnli: 60 -> 77 (83)
boolq: 82 -> 90 (93)
arc: 92 -> 96 (97)

decision accuracy, one Qwen3.5-4B. left numhe chat template, right is after a LoRA fine-tune (r16, 3 epochs, 6k labelled decisions). in brackets: the same fine-tune on Qwen3.8-27B.

fine-tuning moved the 4B by 3 to 70 points.
swapping it for a model six times bigger then moved it by -0.5 to +5.8.
that swap costs 4.4x the training time and 59 GB instead of 12.

chaosnli and boolq are where the big model still earns its keep: ambiguous labels and true/false judgments. everywhere else the 4B is done.

one seed per dataset except typed decisions and opssat (3 each). seed spread at 4B is 0.4 points.

the labels did the work. the parameters watched.

spark-train · DGX Spark GB10 · Qwen3.5-4B / Qwen3.8-27B · LoRA r16

follow up to the energy per token post. same 27B GGUF, same llama.cpp, DGX Spark vs my M4 Max, now run out to the model's full 256k context

to be clear - this is not playing to each machine's strengths and ofc this is the M4 max. chip power only on both sides: GPU module on the not a perfect comparison..

to 64k the Mac SoC is 1.6 to 1.9x cheaper per decoded token. 2.3 to 2.5 J vs 3.9 to 4.7 J on the Spark GPU

then the curves meet. at 256k it is 6.38 J on the Mac, 6.59 J on the Spark. same energy per token

what changed is the balance, not the winner. the Spark reads the 256k prompt in 9 minutes, the Mac takes 58. the Spark decodes 6.4 tok/s there, the Mac 5.3. the Mac gets to parity on energy by drawing 34 W to the Spark's 42, not by being faster

it doesn't seem quite clear if 2 sparks are better or worse than an mac m5 ultra with 256gb..

sure if you're just doing some quick questions the mac wins

when you pass the point of 128k tok its not as clear. the sparks prefill makes up for its slower decode.

the mac beings to slow down and the energy seems to almost match

im currently wondering if i should get another spark or get the 256gb m5 ultra.

i dont think the its as simple to say the mac is better because it has 1.2tb/s of bandwidth on mem.

in practical use cases (subjective) you'll see a substantial amount of context.

what is great though is the macs RDMA which for the price of 'only' 80k you could have 2tb of high bandwidth mem. which is objectively a good price in this market for those that can afford it.

mlx is ~20% faster on macOS 27.2 than on 15.5. llama.cpp did not move

i had been putting off this upgrade for months and wondered if any performance would come of it. so:

before and after on the same M4 Max 128 GB,
same binaries,
3 models on mlx and on llama.cpp

decode x-lm 4-bit
Llama 3.2 1B · 363 -> 466
Qwen3 8B · 75.7 -> 94.9
Qwen3.6 35B-A3B · 82.4 -> 104.1

llama.cpp Q4_K_M
Llama 3.2 1B · 380 -> 367
Qwen3 8B · 83.3 -> 83.1
Qwen3.6 35B-A3B · 79.7 -> 80.5

mlx up 17-28% depending on model and phase. llama.cpp flat within 3.4%. nothing rebuilt, nothing reinstalled

one mention: spotlight was still indexing during the 15.5 run, so that tainted those numbers regardless. the 27.2 side was reconfirmed with two gated 5-rep passes