Pinned Tweet
not wrong at all, i ran the same kind of thing with a 3d car configurator. opus 5.5 looked best but hit my 40 min cap at $5.76, haiku 5.5 got a working one for $0.38. did you track cost per model? curious how sol 6.1 compared on yours
1
0
12
948
can claude opus 5.5 build smarter traffic lights?
it built a pretend city full of cars and tested 2 kinds of lights:
normal ones on a timer, and smart ones that react to waiting cars
quiet roads: smart lights were a bit faster
rush hour: smart lights made trips 6.6% longer. side normal, right side smart 👇
it built a pretend city full of cars and tested 2 kinds of lights:
normal ones on a timer, and smart ones that react to waiting cars
quiet roads: smart lights were a bit faster
rush hour: smart lights made trips 6.6% longer. side normal, right side smart 👇
0
0
0
25
just noticed xai's grok build can pick up your codex session
was working in codex, opened grok build for something else, and it offered to resume where codex left off (ctrl+u, or /resume-codex <session-id>)
→ reads the codex transcript to find where it stopped
→ checks the machining
→ keeps going on grok 4.7 (high)
tried it on a real task. it found the spot and kept going. handy when codex hits its limit
was working in codex, opened grok build for something else, and it offered to resume where codex left off (ctrl+u, or /resume-codex <session-id>)
→ reads the codex transcript to find where it stopped
→ checks the machining
→ keeps going on grok 4.7 (high)
tried it on a real task. it found the spot and kept going. handy when codex hits its limit
1
0
0
27
this holds up in claude code. i tested it at 380k tokens same 4-step job, sonnet 5.5, 3 runs each: compact once: $2.98 never compact: $3.44 new chat every step: $3.42 same hidden tests passed every time last week on a small session, compacting lost. size decides
New NVIDIA paper finds that AI agents barely lose accuracy when they keep contexts short by compacting their history often.
Each new token costs more to generate as the context grows. Serving open models at a 256K context window instead of 16K cost 3.6x to 7x more per generated stop at a set limit, write a short handoff note, and restart with only the task and that note. For long documents, the agent searched files with Bash tools instead of reading everything at once.
On 6 long-horizon benchmarks, at most 5 of 22 compaction settings per model lost more than 2 points versus full history. Compacting after every action kept Claude Opus 4.5 at 70.2% on SWE-bench Verified, versus 70.0% with full history.
Before paying for long context, give agents a structured handoff note and file search tools, then check if short contexts match your accuracy.
1
0
1
40
setup: claude read 950kb of the codebase first (~380k tokens), then built a 4-step feature graded by hidden tests. compacting once right after the read made the coding steps 25% cheaper weird bonus: the "never compact" runs dropped to ~140k on their own when coding started. claude code clears old file reads by itself effort high, api list prices from token logs
0
0
0
21
GO ONE STEP LOWER. OPUS 5.5 ON LOW PASSED THE SAME HIDDEN TESTS AS MEDIUM
claude --effort low
→ 8-file feature: $0.73 vs $1.82
→ bug fix: $0.29 vs $0.51
→ xhigh on a 3d scene: $6.74, never finished
on real code, medium is already more thinking than most jobs need
claude --effort low
→ 8-file feature: $0.73 vs $1.82
→ bug fix: $0.29 vs $0.51
→ xhigh on a 3d scene: $6.74, never finished
on real code, medium is already more thinking than most jobs need
Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams
The update applies to the ~1,000 serverless endpoints we s a day for speed, latency and caching.
Key changes:
➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.
➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts.
➤ Agentic workloads are now part of every measurement: we benchmark short, document-length and long-context prompts, plus multi-turn sessions across coding and knowledge work. Headline figures are the average across these workloads over the past 5 days.
➤ API default effort is now tracked alongside highest effort: most developers run models at the effort level each lab sets as its API default, so we now benchmark key models at that setting as well as at their highest effort. Effort names are not comparable across labs, so we use each lab's own default rather than one shared setting.
Key takeaways:
➤ OpenAI models make up most of the Intelligence vs. Time per Task Pareto frontier despite slower output speeds: across effort levels, GPT-6 Astra and GPT-6.1 Sol hold the frontier from ~45 to ~55 on the Intelligence Index. At max effort, GPT-6 Astra runs at 51 tokens/s but finishes a task in ~17 minutes, against ~26 for Claude Opus 5.5 (max, 111 tokens/s) and ~29 minutes for Gemini 4 Carbon (high, 71 tokens/s), because it uses ~50k output tokens per task against ~120k to ~170k.
➤ API default effort cuts Time per Task by ~3 to 5x and Cost per Task by ~2 to 4x for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5: all three default to medium effort, which scores ~5 to 11 points lower on the Intelligence Index in exchange. Claude Opus 5.5 falls from ~26 minutes and $10.04 per task at max to ~6.8 minutes and $2.45 at medium effort.
➤ Endpoints that optimize for decode speed alone see the largest drop in End-to-End Output Speed: high decode speeds can come at the cost of latency, for example by queuing requests to run at lower batch sizes, or by holding output and sending it in a single burst. Both lengthen the wait before the first token, which End-to-End Output Speed now counts.
2
0
0
63
how i tested: same prompt, fresh container per run, graded by hidden tests the agent never saw. costs at api list prices from token logs. transcripts confirm every run ran at the effort i set
0
0
0
11
"track tokens per task" is the right call. i did, and the fix most people reach for made it worse
same 4-step job in claude code:
one long session: $1.12
/compact between steps: $1.43
new chat every step: $1.42
same tests passed. on a normal-size job, keep the session going
same 4-step job in claude code:
one long session: $1.12
/compact between steps: $1.43
new chat every step: $1.42
same tests passed. on a normal-size job, keep the session going
CLAUDE CODE USERS ARE BURNING TOKENS 3.8X FASTER....SAME PLAN, SAME PRICE
Turns out the model might not be the only problem.... It's the subagents.
Lessons from one dev's token analysis:
- Track tokens per task, not just your bill
- Subagents re-read the same files again gents means more steps, not better results
- Past 350K context, agents keep working after they're done
- Start a fresh session around 300K tokens
More agents doesn't mean more work done. 👀
7
0
3
242
why: earlier context is re-read from cache at ~10% of the normal price. /compact and fresh chats throw that away and the model re-reads your files at full price
restarting around 300k probably still makes sense, my job never got that big. sonnet 5.5, effort high, 3 runs per method, hidden tests
restarting around 300k probably still makes sense, my job never got that big. sonnet 5.5, effort high, 3 runs per method, hidden tests
0
0
0
17
opus 5.5 wins the hard benchmarks. fair.
but do you really need it to build a plumber's booking website?
i tested 4 models, same prompt, 3 runs each:
haiku: 6¢ — 32/33 tests
sol: 17¢ — 32/33
sonnet: 35¢ — 33/33
opus: $1.74 — 33/33
29x cost gap.
all 4 websites side by side 👇
but do you really need it to build a plumber's booking website?
i tested 4 models, same prompt, 3 runs each:
haiku: 6¢ — 32/33 tests
sol: 17¢ — 32/33
sonnet: 35¢ — 33/33
opus: $1.74 — 33/33
29x cost gap.
all 4 websites side by side 👇
Super interesting Benchmark. GDPx measures how well AI agents handle 206 challenging tasks across 11 economically valuable fields, including chip design, software engineering, finance, and biology.
But here's the interesting part: Opus 5.5 wins with 12%, closely followed by .9%. But while performing just 1.1% worse, Gemini 4 is about 4x cheaper than Opus 5.5. So cost-performance wise Gemini 4 wins by a lot.
0
0
1
40
opus 5.5 built this ship for $6.94. gpt-6.1 sol changed the sails for $0.20
one edit: "make the sails triangular, change nothing else"
sol did it in 2 minutes. hull, ocean, light and camera untouched
building is the expensive part. changing it doesn't have to be
one edit: "make the sails triangular, change nothing else"
sol did it in 2 minutes. hull, ocean, light and camera untouched
building is the expensive part. changing it doesn't have to be
2
0
3
79
all 18 edits worked. each model editing its own ship: terra $0.02, sol $0.13, sonnet 5.5 $0.42, opus 5.5 $1.40
the reverse costs more: opus editing sol's ship was $1.95
all on high effort, 3 runs each, same one-line prompt
the reverse costs more: opus editing sol's ship was $1.95
all on high effort, 3 runs each, same one-line prompt
0
0
1
23
tested exactly this. same prompt, both on high, 3 runs each: opus 5.5 $6.94, sol $0.53
opus does look better. the light, the haze, the detail. but sol gets you most of the way for 13x less, and opus hit my 40 min cap in 2 of 3 runs
better, yes. 13x better, not on this one
opus does look better. the light, the haze, the detail. but sol gets you most of the way for 13x less, and opus hit my 40 min cap in 2 of 3 runs
better, yes. 13x better, not on this one
Note that 5.5 on high is a lot better than GPT 6.1 Sol on high, so though the cost gap is a lot smaller when you consider subsidization, you're getting a better model.
If you compare a lower reasoning effort on Opus to a higher one on Sol to be more fair, the Claude advantage ompetitive, tho
0
0
1
87
gpt-6.1 sol built this galleon for 53 cents. opus 5.5 cost $6.94
same prompt, 4 models, all on high effort, 3 runs each:
terra: 1m 25s, $0.07
sol: 16 min, $0.53
sonnet 5.5: 40 min*, $4.97
opus 5.5: 37 min, $6.94
*sonnet hit my 40 min cap in all 3 runs, opus in 2 of 3
opus has 13x cheaper and close enough that it's the one i'd pick
same prompt, 4 models, all on high effort, 3 runs each:
terra: 1m 25s, $0.07
sol: 16 min, $0.53
sonnet 5.5: 40 min*, $4.97
opus 5.5: 37 min, $6.94
*sonnet hit my 40 min cap in all 3 runs, opus in 2 of 3
opus has 13x cheaper and close enough that it's the one i'd pick
1
0
0
148
setup: every model on high effort, and the logs confirm every turn ran at high. fresh container per run, one prompt pointing at a spec file, costs at api list prices from token logs. each tile is that model's median-cost run
0
0
0
22
Sergey | AI Builds & Tests retweeted
Psychedelic Neon Portrait
TFTT @EdithDel_Carmen
Here is my take..
Irregular rhythms.
Ether flowing between two stars.
Our pleasures, one infinity. ♾️
Inviting
@carolletta @RoseDenize @Stringerjohn007 @By0yom1
All invited 🪐
TFTT @EdithDel_Carmen
Here is my take..
Irregular rhythms.
Ether flowing between two stars.
Our pleasures, one infinity. ♾️
Inviting
@carolletta @RoseDenize @Stringerjohn007 @By0yom1
All invited 🪐
Psychedelic Neon Portrait
Prompt: A stylized neon portrait of a young adult woman centered in a close-up, three-quarter view, facing slightly left with her chin raised and eyes closed. She has blue-violet skin, full glossy lips, and straight dark hair framing her face. Oversized it beneath an enormous, broad-brimmed hat whose vivid pink and electric-purple concentric patterns dominate the upper frame. She wears a high, brightly striped neon choker and a vivid magenta-and-lime patterned garment over her shoulders. Several dark-furred chimpanzees with expressive faces surround her in the background, peering from the shadows at different heights. The composition is tightly cropped, with the woman in sharp focus and the animals partially obscured around her; the perspective is intimate and slightly low. Saturated ultraviolet blues, hot pinks, and fluorescent greens glow against a nearly black, jungle-like setting.
Made with @AdobeFirefly Animated with Gemini Omni Flash 1.1 Inviting: @NecoHashimoto @JimAlbany @Bic_Revelation Everyone is Welcome!
15
8
46
2,470
6 cents got me a toy. $3.78 got me a movie.
Pawel found Terra drops off hard below max effort. I hit the same wall at default.
One prompt, no help, a WWII plane chase in Three.js:
Terra (low): 56s, $0.06
Sol (medium): 17 min, $0.61
Sonnet 5.5: 39 min, $3.78
Opus 5.5: 40 min*, ap in all 3 runs
Pawel found Terra drops off hard below max effort. I hit the same wall at default.
One prompt, no help, a WWII plane chase in Three.js:
Terra (low): 56s, $0.06
Sol (medium): 17 min, $0.61
Sonnet 5.5: 39 min, $3.78
Opus 5.5: 40 min*, ap in all 3 runs
I tested GPT-6.1 Terra on real work. 2 repos. 105 replanted bugs. Find and fix what you can.
The results:
- GPT-6.1 Terra (max): 40.7
- GPT-5.6 Terra (max): 32
- GPT-5.6 Terra (xhigh): 20
- GPT-5.6 Terra (high): 18
- GPT-6.1 Terra (xhigh): 11.3
- GPT-6.1 Terra (high): 9.7
It's . But right after max, its performance drops dramatically - far below Haiku 5.5 (xhigh).
1
0
1
90
3 runs per model, the run shown is each model’s median by cost. Fresh Docker container every run, each model at its default effort. Cost is API list price worked out from the token counts.
Sol at 61 cents is the one I’d actually use.
Sol at 61 cents is the one I’d actually use.
0
0
0
29
you don't need opus for this. sonnet 5.5 built this jet engine in blender 12 times, all from code, about $0.57 a run
tried 3 add-ons people use to cut the bill too: ponytail $0.57, caveman $0.51, rtk $0.58 per engine. same quality, bills within ~10%
tried 3 add-ons people use to cut the bill too: ponytail $0.57, caveman $0.51, rtk $0.58 per engine. same quality, bills within ~10%
Everyone's using Claude Opus 5.5 to build entire worlds in #threejs.
I spent 7 days on a single human eye.
No textures. No 3D models. No photos. Just math, light and ~400 parameters.
Sound on 🔊 Link below 👇
2
0
1
99
measured what one 3d build actually costs. sonnet 5.5 wrote this jet engine in blender 12 times: plain, and 3 add-ons people install to cut their bill (ponytail, caveman, rtk). 3 runs each
median cost per engine:
plain $0.57
ponytail $0.57
caveman $0.51
rtk $0.58
1 run said 0%. 3 runs said 0%. test an add-on once and you're mostly measuring luck
median cost per engine:
plain $0.57
ponytail $0.57
caveman $0.51
rtk $0.58
1 run said 0%. 3 runs said 0%. test an add-on once and you're mostly measuring luck
Stop burning Claude or GPT quota on basic 3D animations. Save this $0 pipeline:
1 - create a free model on Tripo or grab any from Sketchfab
2 - upload your mesh to Mesh2Motion, rig it, and apply existing animations
just a couple of minutes, literally $0 spent, and I have a my project
1
0
0
45