Sid retweeted
Excited to share our work on hierarchical models with dynamic tokenisation! ✨
All digital data can be represented as bytes. H-Nets are a step towards building end-to-end models that can operate across modalities and domains without handcrafted tokenisers.
Check out the blog for more details!
All digital data can be represented as bytes. H-Nets are a step towards building end-to-end models that can operate across modalities and domains without handcrafted tokenisers.
Check out the blog for more details!
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language s a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
5
7
66
2,409
Sid retweeted
i've been creating my own design tools for:
🎮 multiplayer design agents
🔎 specialist design agents for typography, motion & polish
🦺 design harnesses to help non-designers design
🛍️ agents that can buy, install & use fonts, effects & graphics
😴 dreams that come up with design concedesign agents their own computer & GPU
reply with 🫧 and i'll DM how to get early access
🎮 multiplayer design agents
🔎 specialist design agents for typography, motion & polish
🦺 design harnesses to help non-designers design
🛍️ agents that can buy, install & use fonts, effects & graphics
😴 dreams that come up with design concedesign agents their own computer & GPU
reply with 🫧 and i'll DM how to get early access
12
6
72
2,238
My partner's mom applied for a Canadian citizenship after being in Canada for multiple decades. The application was sent in a year ago or more, IRCC sent it back saying the fee wasn't paid in full (because the fee went up by three dollars AFTER the application was submitted and the initial fee was paid). And now they sent it back AGAIN because the fee increased by ONE fucking dollar. @CitImmCanada stop harassing people for bullshit reasons and do things that matter, PLEASE
10
4
136
22,456
One of the more frustrating aspects of working with agents is their tendency to underestimate their own intelligence. Although it’s clear to us that intelligence is perhaps becoming an ability you can summon at will, the models try to substitute ill-defined, underspecified logic exactly where intelligence would shine. You have this incredibly smart model that can use the browser to take pretty complex actions, but it crashes midway because some attribute in a dataclass it defined itself doesn’t match. This throws people into loops where they use that same intelligence to diagnose and fix the issue.
This might also the reason behind the “everyone is building a harness” sentiment: everyone is trying to figure out the boundary between intelligence and logic/code. Iterations on products end up looking like a search for this boundary, and as models absorb the harness itself, the boundary moves again and product/harness builders need to adjust. Could also be why a harness that lags the model’s capability can hurt the metric you care about.
As for why models underestimate themselves in the first place, I think it makes sense. The model at any given instance is the result of patching its own past failures, from a point when its final capability didn’t exist yet. It doesn’t know where its training would end up, so its sense of what it can do is tied to a weaker version of itself. Distillation on the other hand, perhaps goes too far the other way where the model learns to be overconfident in its predictions because of a much stronger teacher. Of course, distillation is still a net positive but produces poorly calibrated models which in turn blurs this intelligence-logic boundary (like an incorrect CoT, correct final answer).
This might also the reason behind the “everyone is building a harness” sentiment: everyone is trying to figure out the boundary between intelligence and logic/code. Iterations on products end up looking like a search for this boundary, and as models absorb the harness itself, the boundary moves again and product/harness builders need to adjust. Could also be why a harness that lags the model’s capability can hurt the metric you care about.
As for why models underestimate themselves in the first place, I think it makes sense. The model at any given instance is the result of patching its own past failures, from a point when its final capability didn’t exist yet. It doesn’t know where its training would end up, so its sense of what it can do is tied to a weaker version of itself. Distillation on the other hand, perhaps goes too far the other way where the model learns to be overconfident in its predictions because of a much stronger teacher. Of course, distillation is still a net positive but produces poorly calibrated models which in turn blurs this intelligence-logic boundary (like an incorrect CoT, correct final answer).
5
0
21
1,099
I feel for the railway team, impossible times rn. I generally think they’re great at being graceful, transparent and upfront about outages. It also makes sense for attackers to target infrastructure companies because you get so much more leverage. We did migrate to AWS today - because compromised keys + no builds/deploys was a dealbreaker. I’m sure it was the best course of action given everything they know internally. Redundancy in infra choice and active attacks against yourself are inevitable now
5
0
19
2,156
Sid retweeted
Oh, you aren’t updating your dependencies every hour? Well, too bad, you are vulnerable
Oh, you are updating your dependencies every hour? Well, too bad, you are somehow also vulnerable
Oh, you are updating your dependencies every hour? Well, too bad, you are somehow also vulnerable
🚨 JFrog Security Research has identified a supply-chain attack involving malicious code in the LiteLLM 1.107.0 PyPI release. Avoid this version and investigate environments where it was installed or executed. More details will be shared soon in our blog post.
35
231
4,061
79,318
One last role at Ambition for the next little bit
- Do you love traveling and talking to people?
- Do you love seeing people do hard things and succeed?
- Do you have friends across the world?
Then you might be a great fit to be a Catalyzer
jobs.ashbyhq.com/ambition/939c5…
- Do you love traveling and talking to people?
- Do you love seeing people do hard things and succeed?
- Do you have friends across the world?
Then you might be a great fit to be a Catalyzer
jobs.ashbyhq.com/ambition/939c5…
12
8
126
12,870
Sid retweeted
This should be an OS-level feature on every computer. I want everything I do on my laptop recorded, processed, and made searchable via local models. Recording meetings is just the beginning. Who has had success with a tool like this so far, and what has it unlocked for you?
A few weeks ago, I gave Codex a goal to build something like this but with encrypted store and audited access logs. I called it Passage - it organizes activity into sessions, provides access logs, does (differential) OCR and comes with a CLI. Crazy times
github.com/srk97/passage
6
1
16
1,189
Time (and how we feel time as we get older) is the scarcest resource we have. Doing things is hard. So it only makes sense to do work that you find a lot of meaning and joy in. Everything else is a psyop / self-deprecating delusion. It’s also why I find excuses of “golden handcuffs” nauseating from people who have a choice
3
2
18
896
Sid retweeted
@ONagel33303 I see high importance in having a human who is genuinely passionate about a topic, who got obsessed with it, and worked with their own hands to craft an explanation around it, and who tries to share that enthusiasm. Probably all the more important with more AI-generated content.
10
45
601
18,200
We are looking for our second product designer, in Toronto, ON, Canada
jobs.ashbyhq.com/ambition/b8cf1…
jobs.ashbyhq.com/ambition/b8cf1…
15
18
126
11,157
Sid retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/De…
1/4
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/De…
1/4
1,661
4,907
43,426
6,835,876
Last week to apply for Catalyst 003: Critical Minerals 🇨🇦
Looking at the applicants now, I'm SO SO EXCITED to kick this Catalyst off, bringing together a bunch of experienced geoscientists and software people that want to learn and contribute.
Looking at the applicants now, I'm SO SO EXCITED to kick this Catalyst off, bringing together a bunch of experienced geoscientists and software people that want to learn and contribute.
🇨🇦🇨🇦Catalyst 003: Critical Minerals 🇨🇦🇨🇦
A month long working group to explore, understand and make progress on problems involving the processing of critical minerals in Canada.
Apply at: catalyst.ambition.inc/critical-miner…
1
4
21
120,439
Got a sneak peek at what @designcompute_ has been working on today. It’s so sick
0
1
10
843
Sid retweeted
We’re bringing the magic of Waymo to Toronto! 🇨🇦
In the coming days, you’ll begin seeing Waymo’s all-electric vehicles driving through Toronto’s streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future drwYQlbZJ
In the coming days, you’ll begin seeing Waymo’s all-electric vehicles driving through Toronto’s streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future drwYQlbZJ
322
368
4,094
661,490



















