We’re going to see a bunch of neolab to neocloud pivots next year
Heard from a neolab friend:
“Even if we fail to train the best model, we locked in GPU capacity early. H100/B200/B300 rental prices have since gone up 2–3x. We can just resell the compute and print money.”
OK, so that’s the business model.
15
11
205
15,483
Deep and fun conversation on @latentspacepod about the cloud market, why sandbox demand is exploding, why code reviews suck, and much more. Thanks for having me on!
Last week, @bernhardsson (Founder & CEO, @modal) announced a $750M Series D. I sat down with him the prior week to discuss a ton of stuff about the business.
We got into:
-why open-source models made fine-tuning a lot less necessary
-how Modal's team grew 4x in a year and could hit why the frontier labs have the same business model as music artists
and much, much, more…
Enjoy!
Timestamps:
0:00 What Modal does
1:25 From @Spotify to building @modal
4:19 Founding before the generative AI boom
7:43 First customers and fundraising
10:00 Building for inference and autoscaling
13:21 Where Modal gets GPU capacity
15:39 Renting vs. owning infrastructure
19:45 The risk of GPU prices falling
24:18 Is AI demand infinite?
28:25 What Modal can learn from neoclouds
30:26 GPUs as an asset class
31:55 Modal’s products: functions and sandboxes
36:03 The case for smaller training workloads
41:21 How agent sandboxes are changing
46:48 @AnthropicAI and sandbox security
49:29 Using agents to test security
52:51 Developer experience in the age of agents
57:45 What Modal’s forward-deployed engineers do
59:55 Modal’s approach to product management
1:03:29 Optional code reviews and engineering culture
1:05:56 Cloud, on-prem and bring-your-own-cloud
1:09:25 Capacity, fundraising and growth
1:13:07 Scaling the team
1:17:28 Erik’s tweets: ephemeral sandboxes
1:18:31 The problem with rectangular GPU contracts
1:21:20 Inference providers vs. model builders
1:23:25 Where models improve next
4
4
38
7,407
It’s going to be a long winter for every security team out there. Be nice to them.
18
41
484
19,745
Think we’ll come out of the winter with really good defensive use of AI for security though.
2
0
12
1,473
Erik Bernhardsson retweeted
Some projects are only done for shareholder value. This one was done for the love of the game, literally! My favorite online game, Curvytron, is now backed by the Sticky Sessions primitive my team recently shipped :)
curvytron.modal.dev/#/
Shoutout to @siwonpark_, @ksinpoint1, and @kathirmeyyappan for embodying the @modal spirit and shipping this with speed.
curvytron.modal.dev/#/
Shoutout to @siwonpark_, @ksinpoint1, and @kathirmeyyappan for embodying the @modal spirit and shipping this with speed.
5
9
53
3,794
It can be simultaneously true that we’re going to need a shit ton of GPUs and that the equity return of owning GPUs isn’t going to be super high. I’m biased but I think most value will accrue in the software layer – applications and infrastructure.
41
23
522
33,826
Erik Bernhardsson retweeted
The secret to Modal's speed is our custom lazily loaded container filesystem, ImageFS. ImageFS is built on a massive tiered distributed filesystem, layering local NVMe disks, regional and p2p caches, and global blob stores.
When we launched VM Sandboxes this month, we wanted to ke advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.
When we launched VM Sandboxes this month, we wanted to ke advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.
14
30
543
52,438
I met approximately 1e3 neolabs in London started by ex-DeepMind researchers. Quite bullish on the city.
27
10
410
33,235
Erik Bernhardsson retweeted
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and odal. There's still so much more to build, and we can't think of a better time to be building software.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and odal. There's still so much more to build, and we can't think of a better time to be building software.
85
105
1,238
308,822
Erik Bernhardsson retweeted
we have a lot of work to do.
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and odal. There's still so much more to build, and we can't think of a better time to be building software.
7
5
133
8,310
These businesses are cash machines!
I think an interesting question is what's the right capital structure for them. Think there's a decent argument they should not get stuck on the venture capital treadmill. At some point just get to positive cash flow and issue dividends.
I think an interesting question is what's the right capital structure for them. Think there's a decent argument they should not get stuck on the venture capital treadmill. At some point just get to positive cash flow and issue dividends.
I’ve had 3 diff people tell me they are 100% sure this “fastest to $1B annualized revenue” is 3 diff companies.
Very likely more than 1 data co has recently hit that mark, many of which are on Pavlov’s List.
Many more will get there in the next 6mos.
Total data spend is >$15B.
18
6
404
83,110
We're really good at scaling and also running bus shelter ads
Jev launched on a Thursday. By Sunday, the @typesafeai team had served a trillion tokens with no signs of slowing down.
Learn how they scaled Jev on Modal.
6
10
146
19,106
The math results yesterday are truly incredible.
This one for instance. I spent a ton of time implementing different integer multiplication algos back in school. The time complexity of this new algorithm is so deeply psychotic and unsettling lol.
This one for instance. I spent a ton of time implementing different integer multiplication algos back in school. The time complexity of this new algorithm is so deeply psychotic and unsettling lol.
9
10
362
27,592
Like, for this to matter, you have to multiply numbers that are something like 2^2^182 _digits_. So insane and impressive though.
0
0
25
3,755
6th place is pretty cool I guess, but next year we'll be top 5!
Every startup says they have the best team. We looked at the data.
@paraformtalent’s Talent Density Index ranks which companies attract, develop and retain the people the market values most.
The latest edition is live.
New Top 50: paraform.com/talent-density…
7
6
155
18,164