Pinned Tweet
I had tons of fun working on pretraining (& automating the pretraining) of Flint! It's a really good local model and a small preview of what we are cooking!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
10
8
100
3,987
CLS retweeted
I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition.
First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓
ur optimizer hyperparameter assignment and unit designed by @wen_kaiyue
Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.
Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.
Which recipe gives the lowest final validation loss?
A. Keep LR and WD unchanged
B. LR ×√8
C. LR ×8
D. WD ×8
E. LR ×√8 and WD ×√8
What would you predict, and why?
First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓
ur optimizer hyperparameter assignment and unit designed by @wen_kaiyue
Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178.
Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below.
Which recipe gives the lowest final validation loss?
A. Keep LR and WD unchanged
B. LR ×√8
C. LR ×8
D. WD ×8
E. LR ×√8 and WD ×√8
What would you predict, and why?
12
46
533
39,779
CLS retweeted
Jobs are modeled as bundles of tasks. But as jobs change, how do we keep those task lists up to date?
Workers often experience new responsibilities and tools before they reach their job descriptions.
We start by listening to them. 🧵
Workers often experience new responsibilities and tools before they reach their job descriptions.
We start by listening to them. 🧵
9
62
204
27,862
CLS retweeted
1/ You can now run a research swarm entirely on your Mac, model included.
Eight agents share one local 40B Flint model on a 48 GB M4 Pro: writing code, testing ideas and building on whatever works.
Give them a problem and a score to beat. Here's how 🧵
Eight agents share one local 40B Flint model on a 48 GB M4 Pro: writing code, testing ideas and building on whatever works.
Give them a problem and a score to beat. Here's how 🧵
22
53
520
45,493
CLS retweeted
We are announcing three new discoveries in mathematics, computer science, and physics about harmonic analysis, complexity theory, and quantum field theory.
openai.com/index/building…
openai.com/index/building…
307
1,086
7,232
2,262,898
CLS retweeted
(1/9)Suppose you're training an LLM on web data with limited math data. Should you place all your math data at the end of training, or mix it with web data earlier? The answer flips with the learning rate (LR) schedule!
At fixed compute and mid-training (MT) token budgets, wer MT loss when LR decays throughout training. With constant LR, placeing all MT data for the end works better.
We then vary when LR decay begins. The best tested strategy: keep LR high after MT data enters, then decay near the end.
See our new paper: "Data Schedules and Learning Rate Schedules Should Be Designed Together" !
At fixed compute and mid-training (MT) token budgets, wer MT loss when LR decays throughout training. With constant LR, placeing all MT data for the end works better.
We then vary when LR decay begins. The best tested strategy: keep LR high after MT data enters, then decay near the end.
See our new paper: "Data Schedules and Learning Rate Schedules Should Be Designed Together" !
22
53
383
43,754
CLS retweeted
We let agent swarms collaborate for 10K hours in simulated agent-owned GitHubs.
Gemini-3.8-flash collaborate the best while gpt-6-astra scores the worst among frontier models tested⁉️
What are the elements of successful agent collaboration at scale? Check World of SWECraft 🧵
Gemini-3.8-flash collaborate the best while gpt-6-astra scores the worst among frontier models tested⁉️
What are the elements of successful agent collaboration at scale? Check World of SWECraft 🧵
15
60
247
26,410
CLS retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
135
792
6,486
789,832
CLS retweeted
We are very excited to release Realtime-Bench, a well-curated benchmark that systematically test both native realtime models and general VLMs' capabilities to think and act in real-time! Amazing job led by @JingyuTang621, @xyyy6688 and @Zhehao_Zhang123!
From playing games to grabbing concert tickets, many tasks require real-time thinking and action.
Can frontier models solve real-time interactive problems? Check out our latest work, Realtime-Bench!
TL;DR: The best model completes only 36% of tasks reliably. 🧵
0
4
23
1,010
CLS retweeted
Rumor has it frontier labs are using looped LMs to scale quality without scaling parameters. But KV-cache memory still grows with every loop. Is that necessary?
No! Our method BeNice! unlocks near-lossless sharing of the final KV-cache across loops: more depth, same KV memory.
ng isn’t guaranteed out of the box with looped models. 👀👇for our method BeNice! to teach your model that sharing is caring!
No! Our method BeNice! unlocks near-lossless sharing of the final KV-cache across loops: more depth, same KV memory.
ng isn’t guaranteed out of the box with looped models. 👀👇for our method BeNice! to teach your model that sharing is caring!
21
43
307
29,969
CLS retweeted
Project Page: agima.net
Paper: arxiv.org/abs/2610.26845
With @junyi42 (co-first) @Tianning_Chai (co-first), @lukehanjun, @ChenyangWa70207, @xiuyu_l @WeijiaShi2 @XDWang101 @sewon__min @LukeZettlemoyer, @trevordarrell & @HavenFeng
@berkeley_ai
Paper: arxiv.org/abs/2610.26845
With @junyi42 (co-first) @Tianning_Chai (co-first), @lukehanjun, @ChenyangWa70207, @xiuyu_l @WeijiaShi2 @XDWang101 @sewon__min @LukeZettlemoyer, @trevordarrell & @HavenFeng
@berkeley_ai
0
6
55
3,785
CLS retweeted
The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open!
13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, roblem-solving.
Beyond the scale, four highlights for model builders (details in the threads):
- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.
Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.
Pre-training data: huggingface.co/collections/IF…
Post-training data-generation code & tools: github.com/ifm-ai/DATA_NA…
13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, roblem-solving.
Beyond the scale, four highlights for model builders (details in the threads):
- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.
Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.
Pre-training data: huggingface.co/collections/IF…
Post-training data-generation code & tools: github.com/ifm-ai/DATA_NA…
21
64
389
115,617
CLS retweeted
open-weight AI is a lot more useful when you can actually run it yourself.
Flint is released under Apache 2.0, so developers can download it, run it locally, fine-tune it, and build on top of it.
less dependence on someone else’s API. more control over your own stack.
@recursive_si
Flint is released under Apache 2.0, so developers can download it, run it locally, fine-tune it, and build on top of it.
less dependence on someone else’s API. more control over your own stack.
@recursive_si
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
10
8
210
21,234
CLS retweeted
Recursive Flint Results and THIS is the closest thing to Qwen 3.8 35B if existed, so far
There is no thinking mode, but not in traditional sense. Thinking is always ON, no way to control level or off, which means how your harness work around its reasoning is critical to results
tched 27B on both tier2/3 agentic tests, which I haven’t seen done in quite some time. It also runs faster due to architecture, so I was ready to celebrate a new champion but..
Long context. Seeing a pattern lately where models do great until context grows, then starts to fall apart to different degree. Payload fidelity starts to fall behind a little bit, but agentic results speak for themselves and this thing can get it gone with proper harness
Fantastic model, good to bench one in same tier that can keep up with 27B for once.
There is no thinking mode, but not in traditional sense. Thinking is always ON, no way to control level or off, which means how your harness work around its reasoning is critical to results
tched 27B on both tier2/3 agentic tests, which I haven’t seen done in quite some time. It also runs faster due to architecture, so I was ready to celebrate a new champion but..
Long context. Seeing a pattern lately where models do great until context grows, then starts to fall apart to different degree. Payload fidelity starts to fall behind a little bit, but agentic results speak for themselves and this thing can get it gone with proper harness
Fantastic model, good to bench one in same tier that can keep up with 27B for once.
8
1
34
3,466
CLS retweeted
I've been a heavy Qwen user. Now 100% on @Recursive_SI 's Flint. I built this side by side test, running both models at 8bit on the same hardware (RTX A6000 48Gb) try it: rotem.ai/flintvsqwen/ (This is running locally on two identical cards) BTW 4 bit is even faster.
OK, I've had a little play with this 40B MoE Flint model, here are my findings.
It's a good model but at 40B it needs to be compared to models a similar size and the Qwen3.6-35B MoE and Qwen3.8-27B are both smaller so make ideal candidates.
See thread below...
5
5
26
11,990
CLS retweeted
(1/n) Before trusting AI to make scientific discoveries, can it reproduce existing research - and go one step further to test whether conclusions survive reasonable alternative analytical choices?
Sharing work from this summer with @DbrxMosaicAI, @stanfordnlp, and @StanfordOVAL eproduce published medical findings and autonomously investigate analytical choices that could change their conclusions.
Sharing work from this summer with @DbrxMosaicAI, @stanfordnlp, and @StanfordOVAL eproduce published medical findings and autonomously investigate analytical choices that could change their conclusions.
12
39
153
10,823
CLS retweeted
Today we are launching FrontierCS 2, the most frontier benchmark for AI agents in computer science R&D. We built 48 tasks from published papers across 13 areas of CS, refining each one with experts in its field. Excited to co-launch FrontierCS 2 with @kaggle @GoogleDeepMind
21
49
331
44,166
CLS retweeted
1/ Can a model that runs on a laptop also drive GPU training research?
Recursive Flint, our new 40B open model, improved NanoChat across 700+ experiments—finishing close to our historical Claude Haiku 4.5 results.
We were pleasantly surprised. Here's what it tried 🧵
Recursive Flint, our new 40B open model, improved NanoChat across 700+ experiments—finishing close to our historical Claude Haiku 4.5 results.
We were pleasantly surprised. Here's what it tried 🧵
5
5
47
2,411
CLS retweeted
Congratulations! Nvidia NeMo-Automodel supports finetuning it now!
github.com/NVIDIA-NeMo/Au…
github.com/NVIDIA-NeMo/Au…
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
1
1
14
750
CLS retweeted
I've spent the past few months working on Flint. Excited to finally share our first model! Still a lot to improve, but that's also what makes this journey exciting.
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
3
5
46
5,951