Pinned Tweet
🥳 Announcing Stable Audio 3 🍕
🏆 fastest music models ever
💻 runs on MacBookPro M-series
🧪 break it plz
🧠 LoRA finetune in < 1h
📷 Sm = faster, Medium = qualityer
⚡ 59x realtime on M5 Pro
One-liner fast install:
curl -LsSf dadabots.com/_/sa3-mac | bash
🏆 fastest music models ever
💻 runs on MacBookPro M-series
🧪 break it plz
🧠 LoRA finetune in < 1h
📷 Sm = faster, Medium = qualityer
⚡ 59x realtime on M5 Pro
One-liner fast install:
curl -LsSf dadabots.com/_/sa3-mac | bash
Meet Stable Audio 3.0, the open-weight model family built for artistic experimentation.
This is our open invitation to experiment with generative audio. We believe the best innovations are still waiting to be built.
The 4-1-1 on 3.0:
📣 You own your outputs, and can distribute under the Stability AI Community License (up to $1 million in revenue).
🎵 New and improved capabilities include variable-length generation up to six minutes, and full song composition on portable devices, no GPU required.
✅ Trained on a fully licensed dataset.
🎨 You can customize the models on your own library with support for LoRa training, which we’ve documented for the first time.
More on the models 👇
19
38
327
63,594
dadabots retweeted
I manually inspected 40,000 animations to post train an LLM to make demoscenes.
Scraped 10 years of demos, did continued pretraining, curriculum learning and finally RLHF-ed with myself as the reward function.
Yes, models can have "taste".
All you need is data.
1/6
Scraped 10 years of demos, did continued pretraining, curriculum learning and finally RLHF-ed with myself as the reward function.
Yes, models can have "taste".
All you need is data.
1/6
23
45
669
20,961
dadabots retweeted
go read findings dot md paragraph 9 immediately
wtf this is amazing
wtf this is amazing
@treetowntree and i created neuron lens: a database of every single neuron and attention head in qwen 3 8b and gemma 4 12b read using the R-lens (a J-lens variant) and what we call the "S-lens". you can find it at neuron-lens.com, along with the tools we used to make it. ead neurons' input and output projections turns out to be very effective at determining what a neuron does. we're able to legibly read what a majority of neurons do when fired: the lens qualitatively predicts the result of steering over 90% of the time!
the same trick lets us read attention heads' QK circuits and OV circuits, showing us both what they attend to and what they do with it. however, the R-lens has two important flaws: its accuracy degrades in early layers, and it can't separate the MLP block and the attention block.
we found ways around both these issues. we first modified the r-lens by fitting it on input tokens rather than output tokens to get the s-lens, which is accurate in early layers but inaccurate in late layers. we then fitted R- and S-lenses to the activations after attention but before the MLP to get "mid-lenses", which are much better at deciphering MLP inputs and attention outputs. using both of these techniques, we were able to interpret most of the models' layers.
5
14
249
14,212
dadabots retweeted
L6N5401-0.75 seems to be the 'DJ neuron' - it puts everything into the context of a DJ set
48
124
2,432
286,659
dadabots retweeted
Hardest thing ever written award
Inspired by @Sauers_ and @jfonsecarivera and other fun recent work with personas in AI models, I built a partial neuron atlas for Talkie-1930, the historical language model created by @status_effects and @AlecRad. Here's the link: talkieneurons.vercel.app
19
65
1,035
30,629
dadabots retweeted
if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1
now you have evil qwen
recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input
now you have evil qwen
recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input
515
1,325
22,405
4,536,731
dadabots retweeted
What makes a good representation for diffusion models?
Surprisingly, more information isn't always better.
In our new work, Purify Before You Align, we show that removing locally predictable information can substantially improve diffusion alignment.
🎥 A quick overview 👇
🧵 1/n
Surprisingly, more information isn't always better.
In our new work, Purify Before You Align, we show that removing locally predictable information can substantially improve diffusion alignment.
🎥 A quick overview 👇
🧵 1/n
4
21
156
18,274
dadabots retweeted
Fresh take on diffusion distillation: the student learns mean flow by matching the teacher with the sum of a straight-line prediction and a curved zero-integral path at random sub-intervals. For sampling, just throw away the curved part.
Not that simple IMO😁 but very clever!
Not that simple IMO😁 but very clever!
Excited to introduce Simplified Matching Distillation (SMD) for few-step diffusion🧵
No auxiliary score model. No adversarial training. No generator modification.
You will get:
• Photorealistic appearance
• High-quality dynamics
• Memory-efficient training
• Mode covering MD works well across images, video, joint audio–video, and 3D gen, validated across model scales up to 33B parameters.
Besides, SMD supports:
- Multi-teacher capability merging (like OPD in LLM)
- Cross-model capability transfer
- Bidirectional teacher to causal distillation
Full release. Paper, code, model and data.
Project Page: zjwfufu.github.io/SMD-Page/
Paper: arxiv.org/abs/2610.18472
Code: github.com/zjwfufu/SMD
Model Card & Data: huggingface.co/zjwfufu/SMD
Demo Video 👇
2
20
148
10,766
dadabots retweeted
We released the code and model weights here: github.com/ayushjain1144/…. Have fun! 🙂
1/ We introduce TrackEverything: a 3D point tracker that tracks all points across all frames of long videos (1000+ frames).
Our key idea is to tie computation to unique 3D scene content rather than redundant 2D pixels in a video.
trackeverything.github.io
6
128
1,194
48,983
dadabots retweeted
Speech can survive being flipped backward every twenty milliseconds.
The same speech in backward pieces of 200 then 100 then 50 then 20 milliseconds.
The original comes last.
When do the words become clear?
A listening demo from @OrukLabs.
The same speech in backward pieces of 200 then 100 then 50 then 20 milliseconds.
The original comes last.
When do the words become clear?
A listening demo from @OrukLabs.
89
144
2,346
635,985
underfit in action
things i'm working on today:
- making my song
- trying to fine tune @dadabots' underfit, using claude ultracode to help me get a clean songaday dataset (i've tried this about half a dozen times in the last 2 years on various open source repos to no avail - it all comes out gobbledygook and not in an interesting way)
- making an autoclipper a la opus for the new podcast me and @bigvibessss are starting
- recording with @bigvibessss later
- working on the 3rd level for p(doom), thinking about issuing a challenge for people to be able to earn tokens [2PLF955KTb7CDpHbA5VZH8CgunkvWqfic9wufwJFpump]
- working on a plugin that makes it so i can drop my vocals in and it converts them to speech synthesis computer singing voice (a long held dream of mine)
- integrating some more enemies into wamp to take advantage of our new "boss" feature
- working through a huge checklist of things to add/fix in wamp
1
0
13
1,027