This project started with a literature search on "growing" work in modern vision pipelines, and I was surprised that there was not much, although there have been more recent works in language. Since I started my PhD by working on vision (object-centric?!) I was interested.
Excited to share “Growing to See”: growing SSL vision models provide better representations with up to 25% fewer FLOPs!
Co-led with @KaplFer.
With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer.
1/13
1
4
9
368
Early to middle layers contain more information (better downstream metrics vs. baseline), and early layers are more important than late ones (skipping early layers has a bigger and later smaller influence vs. baseline). For more details check out the thread or the paper!
1
0
1
16
There could have been no one better than @AMK6610 to work on this together. He really is one of the kindest and most meticulous people I know in research, and I am happy to have started my PhD journey with him! Fyi: He will graduate soon and is looking for positions in Sweden.
0
0
1
22
Ferdinand Kapl retweeted
Excited to share “Growing to See”: growing SSL vision models provide better representations with up to 25% fewer FLOPs!
Co-led with @KaplFer.
With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer.
1/13
Co-led with @KaplFer.
With @Google’s Paradigms of Intelligence team (@kaitlinmaile & @oswaldjoh), @andrea_dittadi, Karl H. Johansson & @stefanAbauer.
1/13
1
12
36
1,626
Ferdinand Kapl retweeted
Today we release crowd-code 2.0, our second phase of crowd-sourcing the largest long-horizon open software engineering dataset.
Install once. Forget about it.
Install once. Forget about it.
We introduce crowd-code 2.0: An open system for crowd-sourcing long-horizon human software engineering traces as action-observation rollouts
Today’s models have short horizons not because of architectural bottlenecks, but because we don’t train them on months-long human behavior
2
1
18
1,436
SOTA LLMs are trained as large monolithic models. But looping or depth growth can beat that. How?
Co-led with @AngelisManos and @TobiasHppe1.
Incredible collaboration with Paradigms of Intelligence Team @Google 🙌
@kaitlinmaile
@oswaldjoh
@ninoscherrer
and @stefanAbauer. 🧵🔽
Co-led with @AngelisManos and @TobiasHppe1.
Incredible collaboration with Paradigms of Intelligence Team @Google 🙌
@kaitlinmaile
@oswaldjoh
@ninoscherrer
and @stefanAbauer. 🧵🔽
1
12
17
7,630
🔮 Outlook: towards Looped Transformers.
With depth growth, we see compact computational blocks repeated along depth with an internal cyclical structure. This is reminiscent of Universal Transformers: loop-like computation emerges from depth growth without explicit weight tying.
With depth growth, we see compact computational blocks repeated along depth with an internal cyclical structure. This is reminiscent of Universal Transformers: loop-like computation emerges from depth growth without explicit weight tying.
1
0
3
177
📜 Do Depth-Grown Models Overcome the Curse of Depth?
An In-Depth Analysis
Paper: arxiv.org/abs/2512.08819
An In-Depth Analysis
Paper: arxiv.org/abs/2512.08819
0
0
4
161