vpj

@vpj

Oct 23

jack

@jack

Oct 23

We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.

block.xyz/dessa

vpj

@vpj

Oct 23

humans&

@humansand

Oct 23

We accelerated one of the most optimized NVFP4 decode kernels while also making it more accurate

Fewer bits typically mean more speed but less accuracy. We did the opposite: more bits, but up to 10% faster generation

Already used in Ampersand’s RL and Boba serving

vpj

@vpj

Oct 22

humans&

@humansand

Oct 22

Introducing Boba - the social intelligence that understands your goals and your community. Text Boba today or message it online at boba.app

vpj

@vpj

Oct 20

humans&

@humansand

Oct 20

No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable ls
Available to everyone through our API

vpj

@vpj

Oct 11

james hong

@jhong

Oct 11

I make this same post like every 2-3 years, because there's always new data further supporting it.

Georges Harik (@gharik) is one of the most prescient people in AI, and definitely the least well known and underappreciated one.

In 1999 he joined Google as one of the first employees (like #7 or something) because he thought he could use his AI knowledge to build a better ad system. He and Salar Kamangar built Adwords (Salar later bought YouTube and Georges later was the force behind Google wanting to buy Andy rubin's company to build Android)

Georges left Google so he could focus on AI research, which he self funded. He did so with someone he worked with closely at Google, our friend Noam Shazeer (they built the AdSense targeting engine). When Google finally decided to work hard on AI around the time of the deepmind acquisition, they tried to get Georges and Noam to come back. Georges wanted to stay independent, and our friend Noam went on to do great things obviously. This was my good fortune because it led to Georges and I hanging out a lot, because he basically needed a new lunch buddy.

Georges at the time was working on language models and despite karpathy saying nobody saw LLMs being so big even in 2017, Georges was telling me they were on the cusp of changing everything in 2013. My wife and I literally changed some of our parenting priorities that year (less tigery) because georges convinced me the world was going to change so much.

Later on he and I got involved in a company making CV photo filters called Lambda. They needed to make money to survive so they started building ML boxes which Georges bought a bunch of. But his significant other was mad there were so many loud and hot boxes in the house, she started worrying about fire risk, so Georges told the lambda guys they should build a cloud because in the future, the power density of the systems was going to be too big for even most enterprises to handle. This was like 2017. That's how lambda cloud began, and the premise Georges had was that AI was going to change things so much, demand growth would outpace supply growth, and the value of compute was eventually going to rise and that lambda was effectively going to be a REIT... so we should own as many GPUs as possible. It took the world almost a decade to finally figure out what Georges saw then (see attached tweet).

About 5 years ago when pretraining was still the only thing people cared about, Georges started telling me inference was going to be super impt. He was working on research investigating post training as a way to improve reasoning. He found some other guys working on the same idea that had a paper known as the STAR paper, and they worked together to prove it even further with another paper called quietstar. (They all later cofounded the neolab humans&). Georges got some friends to build deepinfra.com to be one of the first inference clouds because he saw it all early. He and the humans& team also smartly used all their money to buy compute knowing it would go up in price and also get harder to obtain.

It's kind of amazing to me how he has gotten everything right so far ahead of time. It's also amazing to me how VCs don't really get or believe a lot of the things he says, they often tell me I am wrong whenever I parrot Georges' current thoughts... and then years later they all put out charts and papers reaffirming those same things as if they knew it all along.

The founders of Google recognize Georges' brilliance and still talk to him a lot. (Besides being smart, he is a lot of fun to hang out with.) I'm just wondering when the rest of the world figures it out

Eric Zelikman

@ericzelikman

Sep 27

Deedy

@deedydas

Sep 26

The economics of a Neolab.

A neolab is loosely defined as a startup of AI researchers who raises a lot of money pre-production to be able to finance GPU compute to take on a large AI problem.

To buy 1000 GB300s or ~14 NVL72 racks will set you back $125-150M for 3yrs with 15-30% upfront. That’s about ~2-2.5MW.

Thats about enough to do 10^25 flops a quarter and get to a GPT-4 level model which is 1-2 OOMs off frontier for pretraining.

If you post-train on a great open source model, you have a better chance of getting to frontier. The risks are a) you need to spend millions on RL environments too and b) being lapped by another model release while being tied to a base model.

For this to payback, you need to give your customers a better and ideally cheaper inference service than a base model and serve them for long enough to recoup your large investment. Even at 50% margin on inference, to recoup $10M in training means serving ~10T tokens (!) if you price like Fable / Astra given a standard cache read / input / output split ($2/M blended). And you have to justify being better than a release like Opus 5.5 which is even cheaper. Often, you end up charging your customers a huge premium in terms of platform fees and compute fees on top of pure inference.

Meanwhile, every hour you’re not utilizing your GPUs you are burning money so you typically resell this compute back to a broker or run inference for open models / resell spot instances. At below a ~60% utilization on spot, you will still lose money.

Add to that insane cost of talent.

So what can you do with the compute?
- Not play the model game at all.
- Play an entirely different model game (Jev, World Labs) that if big labs played, would either a) cannibalize their business or b) be incrementally not significant revenue c) would cause too much distraction from the main main thing
- Acquire a proprietary data set (Peridodic Labs) in enough volume in a domain of usefulness to eclipse frontier quality. Often happens in robotics, biology, chemistry.

If you do overcome the challenge of building a model that is useful and well priced beyond big labs models, given the huge price of compute, you still need to play in an area where the revenue / compute ratio is signficant and market demand is large enough to payback your compute spend.

It is a difficult game.

Alexis Ross

@alexisjross

Sep 10

humans&

@humansand

Sep 10

For AI to work with us, it needs to understand us

Today, we're introducing Persimmon, the first large-scale model designed to realistically simulate how people talk and interact

Manya Bansal

@314Bansal

Sep 10

humans&

@humansand

Sep 10

For AI to work with us, it needs to understand us

Today, we're introducing Persimmon, the first large-scale model designed to realistically simulate how people talk and interact

vpj

@vpj

Sep 11

humans&

@humansand

Sep 10

For AI to work with us, it needs to understand us

Today, we're introducing Persimmon, the first large-scale model designed to realistically simulate how people talk and interact