Pinned Tweet
Astra’s spatial understanding is mindblowing to watch.
three uncalibrated cameras. no intrinsics, no extrinsics.
it nudged the robot arm, learned how each camera sees motion, worked out the brush tip’s 3d offset, and verified it at new angles -- all from one prompt and a tiny t was under 0.2 mm of camera-measured tip movement!
three uncalibrated cameras. no intrinsics, no extrinsics.
it nudged the robot arm, learned how each camera sees motion, worked out the brush tip’s 3d offset, and verified it at new angles -- all from one prompt and a tiny t was under 0.2 mm of camera-measured tip movement!
13
25
407
21,769
nil retweeted
AI isn't just learning to see the world anymore. It's learning to leave its mark on it. Literally.
last night’s watercolor session with Claude! getting much better at trees now.
it also has a much bigger repertoire of brush strokes now, which enables it to compose drawings like this. and in addition, some cool new titration tricks.
Opus then used those brush movements to ard”, an autumn tree leaning into the wind.
1
1
6
450
last night’s watercolor session with Claude! getting much better at trees now.
it also has a much bigger repertoire of brush strokes now, which enables it to compose drawings like this. and in addition, some cool new titration tricks.
Opus then used those brush movements to ard”, an autumn tree leaning into the wind.
it also has a much bigger repertoire of brush strokes now, which enables it to compose drawings like this. and in addition, some cool new titration tricks.
Opus then used those brush movements to ard”, an autumn tree leaning into the wind.
8
5
84
3,602
nil retweeted
Today we’re launching the first AI modeling agency 💅 right inside Glif.
Cast hundreds of real AI models, drop your product, idea or brief and let Glif create super compelling content for you end to end.
If you still think AI models aren’t ready for real work, watch this 👇
Cast hundreds of real AI models, drop your product, idea or brief and let Glif create super compelling content for you end to end.
If you still think AI models aren’t ready for real work, watch this 👇
618
384
4,949
2,885,737
there's stuff i'm doing with my robot arm, and then there's this...
Last night an AI agent performed brain surgery on a mouse.
We gave it 19 gene therapy vectors in a spreadsheet, a mouse model of epilepsy, access to a motorize surgery rig with cameras and one goal: reduce seizures.
It picked the drug. It picked the brain region. It derived the papers. Then it drove the stereotaxic arms into the brain and injected 250 nanolitres while watching the entry site through the rig camera for bleeding.
A licensed vet gated every irreversible step and held a physical kill switch to stop the AI agent.
Thread on what it decided, what it saw, and the moment it refused to let me override it.
[Writing this on the spot out of excitement, but I've been 18hrs in front of my laptop coding and coordinating this, so keeping it brief and descriptive, more cohesive thoughts on this soon...]
1
0
4
439
me: "hi, let's make a new fun painting"
Astra: comes up with the idea → models brush movements → creates a plan → digital twin tests for collisions → controls robot via SDK → starts camera recording via Sony's SDK → reads controller logs → downloads camera footage via edits video to sync with controller logs and music spectrograph → sprinkles some hyperframe magic
1 hour later... "a Small orbit" is ready!
Astra: comes up with the idea → models brush movements → creates a plan → digital twin tests for collisions → controls robot via SDK → starts camera recording via Sony's SDK → reads controller logs → downloads camera footage via edits video to sync with controller logs and music spectrograph → sprinkles some hyperframe magic
1 hour later... "a Small orbit" is ready!
10
21
252
32,558
i know it's not really the same, but all the neuron editing and exploration posts of Qwen remind of this short story by qntm: qntm.org/lena
1
0
1
305
one reason i love painting as a long-term project for my robot + the agents operating it is,
- infinite depth in technique and finesse.
- infinite breadth in creations.
it’s an eval + a creative outlet + a growing body of work, all at once.
the arm has 7 dof and ±0.1mm cameras, top-grade brushes, paper and pigments, a well-calibrated indoor workspace. in principle, i think something of Hokusai's or Monet’s calibre is possible with this setup.
you can see the progression here, bottom to top: calibration runs and basic brush tests, then colour mixing, wet-in-wet, glazing, angled strokes...
just getting started! so much more to learn + create 🦾
- infinite depth in technique and finesse.
- infinite breadth in creations.
it’s an eval + a creative outlet + a growing body of work, all at once.
the arm has 7 dof and ±0.1mm cameras, top-grade brushes, paper and pigments, a well-calibrated indoor workspace. in principle, i think something of Hokusai's or Monet’s calibre is possible with this setup.
you can see the progression here, bottom to top: calibration runs and basic brush tests, then colour mixing, wet-in-wet, glazing, angled strokes...
just getting started! so much more to learn + create 🦾
1
0
3
378
nil retweeted
@dhvanil the crux of the "memory problem" is not how you store things (embeddings/markdowns) or how you retrieve. all memories are associative and depend on the task at hand. i bet task planning/decomposition and memory systems will morph together.
0
1
2
104
my biggest frustrations with ai tools rn is their opaque memory systems.
chatgpt knows some things about me, claude remembers smth else, claude code has a different system, codex feels like a sprawling web of long-forgotten chats, and codebase docs + skills are yet another system.
i DO want these memories to be contextual and siloed. different models and harnesses have different strengths. i want their memories to reflect that.
but a few things that really bother me:
1. Opacity: how are memories created, curated, updated, and forgotten? i have almost no visibility into this.
2. Context rot: what random memories and old chats are seeping into the current context window?
3. Curation: no tools to curate memories. i'd like to have powerful tools to inspect, edit, prune, and shape these memories.
chatgpt knows some things about me, claude remembers smth else, claude code has a different system, codex feels like a sprawling web of long-forgotten chats, and codebase docs + skills are yet another system.
i DO want these memories to be contextual and siloed. different models and harnesses have different strengths. i want their memories to reflect that.
but a few things that really bother me:
1. Opacity: how are memories created, curated, updated, and forgotten? i have almost no visibility into this.
2. Context rot: what random memories and old chats are seeping into the current context window?
3. Curation: no tools to curate memories. i'd like to have powerful tools to inspect, edit, prune, and shape these memories.
17
1
16
1,369
nil retweeted
kay neurons are interpretable enough that models can modify their own weights toward some desiderata.
blogpost incoming
blogpost incoming
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com
19
38
667
28,050