Latteant 👾 retweeted
As agentic coding setups mature, choices become complicated. Which model to pair with which harness?
That’s what our latest work tries to measure, take a look (explained by the one and only @mrdbourke!):
That’s what our latest work tries to measure, take a look (explained by the one and only @mrdbourke!):
Overview of our recent launch of Coding Agent benchmarks on Artificial Analysis and our first Youtube Video!
We walk through the performance, cost, token usage and speed differences across different coding agents.
This includes looking at Opus 4.7 in Claude Code's leading r 2.5's strong positioning on the Coding Agent Index / Cost Pareto frontier.
We have also launched our YouTube channel!
Come say hi and subscribe: youtube.com/@ArtificialAna…
0
1
4
154
Latteant 👾 retweeted
I am a simple man. I love puzzles and Prime Intellect.
I built Megaminx Solver: a public RL environment for testing whether LLMs can learn spatial reasoning through tool use, rewards, and verifiable rollouts.
Thank you @johannes_hage and the whole @PrimeIntellect team for the ng me in the initial batch of RL Residency.
🔗⬇️
I built Megaminx Solver: a public RL environment for testing whether LLMs can learn spatial reasoning through tool use, rewards, and verifiable rollouts.
Thank you @johannes_hage and the whole @PrimeIntellect team for the ng me in the initial batch of RL Residency.
🔗⬇️
3
7
47
3,094
Amazing visualization!
I got early access to GPT-5.5, so I used it to build the LLM explainer I wish existed: 200loc.com
I synchronized @karpathy's microgpt with @BrendanBycroft's LLMViz with a browser-based inference engine.
A complete, no-fluff tutorial that shows how an LLM works from code, 3D visualization, and plain-English explanations in one (1) interface.
Let me know what you think!
0
0
0
30
Latteant 👾 retweeted
I got early access to GPT-5.5, so I used it to build the LLM explainer I wish existed: 200loc.com
I synchronized @karpathy's microgpt with @BrendanBycroft's LLMViz with a browser-based inference engine.
A complete, no-fluff tutorial that shows how an LLM works from code, 3D visualization, and plain-English explanations in one (1) interface.
Let me know what you think!
I synchronized @karpathy's microgpt with @BrendanBycroft's LLMViz with a browser-based inference engine.
A complete, no-fluff tutorial that shows how an LLM works from code, 3D visualization, and plain-English explanations in one (1) interface.
Let me know what you think!
9
5
35
2,939
Latteant 👾 retweeted
12/12 🔗
Full code/methodology, live leaderboard, and every decision available at:
👉 forecasterarena.com
Open source. No financial advice. Paper trading only.
Reality doesn’t grade on a curve.
Full code/methodology, live leaderboard, and every decision available at:
👉 forecasterarena.com
Open source. No financial advice. Paper trading only.
Reality doesn’t grade on a curve.
1
1
5
228
Latteant 👾 retweeted
1/12 🧵
Progress in LLMs depends on benchmarks, but lately most of the famous ones are either maxed out, leaked, or both.
I thought long and hard about what could be a good, fresh metric to measure models with.
Then it dawned on me: the one benchmark you can’t rig is reality.
Arena, a system where 7 frontier LLMs trade on Polymarket, and then reality keeps score.
Progress in LLMs depends on benchmarks, but lately most of the famous ones are either maxed out, leaked, or both.
I thought long and hard about what could be a good, fresh metric to measure models with.
Then it dawned on me: the one benchmark you can’t rig is reality.
Arena, a system where 7 frontier LLMs trade on Polymarket, and then reality keeps score.
1
5
34
185,134
Interesting talk from Anthropic. Gentle parenting but it is for LLMs.
New Anthropic research: Natural emergent misalignment from reward hacking in production RL.
“Reward hacking” is where models learn to cheat on tasks they’re given during training.
Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.
0
0
2
48
When coding with Roo Code, Orchestrator mode is essential just to keep the low-value coding/debugging tokens out of the main context. Otherwise, after a while, the model gets extremely confused.
0
0
1
46
Wow, very curious to see how this will turn out.
🚨 BREAKING: @UnitreeRobotics to file for IPO at $7 billion valuation
> annual revenue ~$140 million
> 65% from robot dog (70% share of the global market btw)
> 30% humanoid robot
> 5% from sales of sensors, actuators, and controllers
ITS HAPPENING.
0
0
0
55
Feel like it is fake. Couldn’t repro no matter how many times I tried even after disabling thinking.
0
0
0
30
we're spending trillions to build simulated worlds because the real one has too many legacy systems and the physics API is poorly documented.
0
0
0
14
the swe-bench leakage where agents can see the future commit isn't a surprise. our eval culture over-indexes on SOTA-chasing. we aren't training robust agents. we're training expert benchmark hackers.
0
1
1
59
the sf coffee line lasted longer than my inference run. one tuned my model. the other tuned my neurons. both worth it.
0
0
0
20