Pinned Tweet
It was great fun contributing to this whitepaper, and it turned into a great read. A deep dive into a number of important and new research directions in neuroAI.
Excited to release what we’ve been working on at Amaranth Foundation, our latest whitepaper, NeuroAI for AI safety! A detailed, ambitious roadmap for how neuroscience research can help build safer AI systems while accelerating both virtual neuroscience and neurotech. 1/N
1
3
13
2,297
I’m hiring a Research Operations Manager to join us at Basis Research Institute in NYC! At Basis, we’re building AI systems that can reason, learn, and support scientific discovery. I’m looking for someone to help drive ambitious research. Apply + share: bit.ly/4jgp59f
1
0
6
292
Karen Schroeder retweeted
Last week Basis researchers attended the National Urban Rat Summit, where @ralph_ep presented our work using acoustic monitoring to study rat behavior, while Nick Jourjine showcased monitoring devices on a nighttime “rat walk” 🐀.
Ralph and @e_mackevicius also met @MayorWu. Ralph spoke with WBUR about the research: wbur.org/news/2026/09/2…
Ralph and @e_mackevicius also met @MayorWu. Ralph spoke with WBUR about the research: wbur.org/news/2026/09/2…
0
3
11
1,066
Karen Schroeder retweeted
New paper from Basis' Project MARA team and collabs. The ability to learn and use world models is a key aspect of human intelligence, but evaluating this ability remains elusive. In this work we propose WorldTest, a representation-agnostic, behavior-based agent eval framework.
1
11
20
4,278
Karen Schroeder retweeted
Thrilled to see our TinyRNN paper in @Nature! We show how tiny RNNs predict choices of individual subjects accurately while staying fully interpretable. This approach can transform how we model cognitive processes in both healthy and disordered decisions. doi.org/10.1038/s41586…
4
55
267
21,426
Karen Schroeder retweeted
We’re making the @BasisOrg organisation document public today. It’s less a charter and more a design doc—our spec for why a new kind of technology and research organisation is needed, and how to build it.
(link below)
(link below)
1
14
49
6,926
Karen Schroeder retweeted
This was a lot of fun.
We just dropped a fun panel discussion with @fchollet @ZennaTavares and @ellisk_kellis on program synthesis. This was a powerhouse conversation with three legends in the field, enjoy.
0
3
38
1,562
Karen Schroeder retweeted
Clip of @fchollet reflecting on the @arcprize challenge with @ZennaTavares and @ellisk_kellis at #neurips2024
1
6
67
5,696
Karen Schroeder retweeted
Chollet @fchollet : [paraphrase] "using NNs for discrete problems i.e. finding new prime numbers - is a terrible idea" with @ellisk_kellis @ZennaTavares
9
17
166
23,258
Karen Schroeder retweeted
Thank you, François, Mike, & team, for the ARC challenge. It has been a durable source of inspiration, and brings fresh ideas to AI.
The paper award first authors are Keya Hu (applying to PhDs @HuLillian39250) and Wen-Ding Li (at NeurIPS hunting for industry gigs @xu3kev). They're amazing: Anyone would be lucky to get them.
It's also our first collab w/ @ZennaTavares from @BasisOrg as part of MARA: basis.ai/blog/mara/
MARA is recruiting at all levels: join us!
The paper award first authors are Keya Hu (applying to PhDs @HuLillian39250) and Wen-Ding Li (at NeurIPS hunting for industry gigs @xu3kev). They're amazing: Anyone would be lucky to get them.
It's also our first collab w/ @ZennaTavares from @BasisOrg as part of MARA: basis.ai/blog/mara/
MARA is recruiting at all levels: join us!
Today we're announcing the winners of ARC Prize 2024. We're also publishing an extensive technical report on what we learned from the competition (link in the next tweet).
The state-of-the-art went from 33% to 55.5%, the largest single-year increase we've seen since 2020. The benchmark remains unbeaten, but we're happy to see that research progress on the key bottleneck to AGIs (in particular on-the-fly adaptation to novel tasks) has been reignited in 2024 -- in part thanks to ARC Prize.
In particular the competition has popularized Test Time Training (TTT), originally pioneered for ARC-AGI by Jack Cole last year. I believe TTT represents the largest jump in LLM generalization capabilities since the initial findings regarding in-context-learning circa 2019-2020. ARC Prize has also led to a considerable surge of research interest towards program synthesis.
Competition winners:
🥇 the ARChitects (Daniel Franzen, Jan Disselhoff)
🥈 @guille_bar
🥉 alijs (Agnis Liukis)
Paper Award winners:
🥇 "Combining Induction and Transduction For Abstract Reasoning" by @xu3kev et al.
🥈 "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning" by @akyurekekin et al.
🥉 "Searching Latent Program Spaces" by @ClementBonnet16 & @MattVMacfarlane
ARC-AGI-Pub Leaderboard (solutions using commercial APIs):
🥇 @jeremyberman
🥈 @ellisk_kellis & @akyurekekin
🥉 @ryangreenblatt
3
14
57
10,743
Karen Schroeder retweeted
Thrilled that joint work by @ellisk_kellis's lab and
@BasisOrg won 1st prize in @arcprize Paper Awards and 2nd prize in ARC-AGI-PUB (w/ MIT)
This is our first result from Project MARA: an effort to build Modeling, Abstraction, and Reasoning Agents capable of "everyday science"
@BasisOrg won 1st prize in @arcprize Paper Awards and 2nd prize in ARC-AGI-PUB (w/ MIT)
This is our first result from Project MARA: an effort to build Modeling, Abstraction, and Reasoning Agents capable of "everyday science"
Proud to share that our work with @ellisk_kellis and collabs won the 1st prize ARC Paper Award! This is the first work to come out of the MARA project. Much more to come.
2
9
39
5,421
Karen Schroeder retweeted
My big list of @arcprize 2024 surprises:
1. TTT works really well to solve "novel" problems. Assuming you have a way to do data augmentation on the fly.
2. Brute force program synthesis is competitive with frontier LLM/AI approaches. We'll fix this in v2.
3. The private and public SOTAs tracked. Both ~55%, despite public having 1000X more compute budget.
4. At least 7 startups with >$1M funding changed research roadmaps to work on ARC.
5. Startups don't have the same incentives for sharing as private teams. Resulted in MindsAI choosing not to open source. We'll address this in 2025.
6. Over 1k people told us to test o1 on ARC. Reporting results on Claude 3.5 Sonnet similarly was a big hit. We'll keep doing this.
7. The #1 and #2 papers dropped out of the blue, during last 24 hours in the contest. Each went trending too. The paper award track was a big success and I'm glad we bumped the prize at 3 months.
8. The #1 winning team (the ARChitects) added 10% to their score in the last 72 hours of the contest, catching up to MindsAI. And they both are using TTT.
9. We have been developing ARC-AGI-2 in parallel this summer to address long-standing v1 flaws (eg. small sample size, brute forcibility, no human difficulty calibration). And while the SOTA is 55.5% today, early results suggest v2 will bring SOTA down more than I expected.
10. ARC Prize broke through. I saw it mentioned in many discussions we were not apart of. Reddit, HN, Discord, Twitter... to the extent a benchmark/nonprofit can have product-market fit, I sense we have it. We'll use this momentum to grow ARC Prize next year and steward attention as a north star towards AGI.
1. TTT works really well to solve "novel" problems. Assuming you have a way to do data augmentation on the fly.
2. Brute force program synthesis is competitive with frontier LLM/AI approaches. We'll fix this in v2.
3. The private and public SOTAs tracked. Both ~55%, despite public having 1000X more compute budget.
4. At least 7 startups with >$1M funding changed research roadmaps to work on ARC.
5. Startups don't have the same incentives for sharing as private teams. Resulted in MindsAI choosing not to open source. We'll address this in 2025.
6. Over 1k people told us to test o1 on ARC. Reporting results on Claude 3.5 Sonnet similarly was a big hit. We'll keep doing this.
7. The #1 and #2 papers dropped out of the blue, during last 24 hours in the contest. Each went trending too. The paper award track was a big success and I'm glad we bumped the prize at 3 months.
8. The #1 winning team (the ARChitects) added 10% to their score in the last 72 hours of the contest, catching up to MindsAI. And they both are using TTT.
9. We have been developing ARC-AGI-2 in parallel this summer to address long-standing v1 flaws (eg. small sample size, brute forcibility, no human difficulty calibration). And while the SOTA is 55.5% today, early results suggest v2 will bring SOTA down more than I expected.
10. ARC Prize broke through. I saw it mentioned in many discussions we were not apart of. Reddit, HN, Discord, Twitter... to the extent a benchmark/nonprofit can have product-market fit, I sense we have it. We'll use this momentum to grow ARC Prize next year and steward attention as a north star towards AGI.
11
41
330
39,282
Karen Schroeder retweeted
Proud to share that our work with @ellisk_kellis and collabs won the 1st prize ARC Paper Award! This is the first work to come out of the MARA project. Much more to come.
ARC Prize 2024 Paper Award Winners! 🏆
1st "Combining Induction and Transduction For Abstract Reasoning", @xu3kev, @HuLillian39250, Carter Larsen, Yuqing Wu, @simon_alford0, Caleb Woo, Spencer M. Dunn, @HaoTang_ai, Michelangelo Naim, Dat Nguyen, @WeiLongZheng1, @ZennaTavares, @evanthebouncy, @ellisk_kellis
2nd "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning", @akyurekekin, @MehulDamani2, @linluqiu, Han Guo, @yoonrkim, @jacobandreas
3rd "Searching Latent Program Spaces", @ClementBonnet16 & @MattVMacfarlane
Runners Up -
Daniel Franzen & Jan Disselhoff - "The LLM ARChitect: Solving the ARC Challenge Is a Matter of Perspective"
@guille_bar - "Omni-ARC"
@pfletcherhill - "Mini-ARC: Solving Abstraction and Reasoning Puzzles with Small Transformer Models"
@simonouellette6 - "Towards Efficient Neurally-Guided Program Induction for ARC-AGI"
@jfpuget - "A 2D nGPT Model For ARC Prize"
1
10
30
10,792
Karen Schroeder retweeted
🚀We’re hiring! @ForestNeurotech is looking for a Software Engineering Lead to build the core systems powering our ultrasound neurotech platform.
As a nonprofit FRO, we're advancing science for public good. If you’re excited about neurotech & impact let’s talk. 🌍🧠 Link below
As a nonprofit FRO, we're advancing science for public good. If you’re excited about neurotech & impact let’s talk. 🌍🧠 Link below
7
27
72
29,046
Karen Schroeder retweeted
📢🎡🐦⬛I'm looking to hire postdocs to join me on the Collaborative Intelligent Systems project at @BasisOrg, more info here: basis.ai/roles/postdoc-…, please apply 🐦⬛🎡📢
0
13
33
5,793
Karen Schroeder retweeted
🎡🦆New @BasisOrg paper by Rafal Urbaniak, @XieMarjorie and me!
Linking cognitive strategy, neural mechanism, and movement statistics in group foraging behaviors
nature.com/articles/s4159…
"Behind the paper" blog here: go.nature.com/3YBaVDK
Code: github.com/BasisResearch/… 🦆🎡
Linking cognitive strategy, neural mechanism, and movement statistics in group foraging behaviors
nature.com/articles/s4159…
"Behind the paper" blog here: go.nature.com/3YBaVDK
Code: github.com/BasisResearch/… 🦆🎡
0
18
45
4,673
Karen Schroeder retweeted
Looking forward to verifying this one. @ellisk_kellis and team are some of the best folks today working on program synthesis.
New ARC-AGI paper
@arcprize w/ fantastic collaborators @xu3kev @HuLillian39250 @ZennaTavares @evanthebouncy @BasisOrg
For few-shot learning: better to construct a symbolic hypothesis/program, or have a neural net do it all, ala in-context learning?
cs.cornell.edu/~ellisk/docume…
1
2
32
3,922
Karen Schroeder retweeted
@BasisOrg has started a joint project called MARA w/ @ellisk_kellis 's lab +others. This is our first output. More details soon.
We're sponsoring "Systems 2 Reasoning at Scale" workshop at Neurips & will present MARA there. We're hiring for it now! basis.ai/join-us/#caree…
We're sponsoring "Systems 2 Reasoning at Scale" workshop at Neurips & will present MARA there. We're hiring for it now! basis.ai/join-us/#caree…
New ARC-AGI paper
@arcprize w/ fantastic collaborators @xu3kev @HuLillian39250 @ZennaTavares @evanthebouncy @BasisOrg
For few-shot learning: better to construct a symbolic hypothesis/program, or have a neural net do it all, ala in-context learning?
cs.cornell.edu/~ellisk/docume…
0
11
32
5,815
Karen Schroeder retweeted
New work with @ellisk_kellis and collabs on @arcprize ARC-AGI, including new SOTA results.
New ARC-AGI paper
@arcprize w/ fantastic collaborators @xu3kev @HuLillian39250 @ZennaTavares @evanthebouncy @BasisOrg
For few-shot learning: better to construct a symbolic hypothesis/program, or have a neural net do it all, ala in-context learning?
cs.cornell.edu/~ellisk/docume…
0
2
5
1,290