Who Wins the Inference Market

It’s so exciting to be here. I mean, I’ve listened to so many episodes. It’s it’s kind of surreal to be inside of it right now.

I started coding 1992 or something like that. I was eight. I just loved it, you know? It’s like one of those things, like, I realized you can control computers. I had a Mac Plus, you know, one megabyte of RAM or whatever. And, yeah, I started making games and, like... I mean, they were terrible. But, you know, it’s just... I think back at, like... In a way, I’m, like, so grateful. I look back at, like, my childhood. My friends, they were, like, collecting hockey cards or whatever.

That was like their hobby. My hobby was programming, and now I make a lot of money from it. Isn’t that kinda create... Like, you you know when you think about it? Like, the fact that I can make a lot of money, like, doing what what was my hobby since I was eight? Kinda nuts. So I I... I’m just so grateful about, you know, the fact that I fell in love with technology and coding, and then I started doing programming competitions when I was 16, 17, and so much fun.

Modal

I think the consistent pattern in my life is like I never had quite enough confidence in myself. And a lot of it that was missing was just the confidence, a little bit the money too. So it wasn’t until I was like ’thirty six, ’thirty seven I started Modal. And I think, I don’t think I’ve ever wanted to necessarily start a company. It was like one of those things where like, I just saw this big opportunity in the market. I almost thought to myself, like, why is no one else doing this?

Such an obvious idea. And that led me to think, like, maybe I should start a company. And then, you know, five years later, here we are. It’s gone pretty well. I think in hindsight, maybe I should have started much earlier. Do you wish you had? I look at some of these, like, Stanford kids, and they’re, like, 21, and, like, I kinda feel they maybe have too much confidence. You know what I mean? Like, they’re, like, you know, think they’re gonna take over the world, and they’re 21.

I think the ideal age is maybe, like, I don’t know, 32 or something like that. You have a little bit more experience. You know how to navigate people. You know how to hire people. You know how to fire people, like all these things. I don’t know if I would have been a good founder, like, in my twenties.

Dude, hate that stuff. It’s like, I don’t get it. Like, you’re eating glass, like, every day. Like, why are you doing it then? Like, I don’t know. You expect some, like, magic reward, like some big cookie in the end? Like, I never understood that mindset. I think it’s like this mindset that a lot of people have. I’m gonna suffer pain, and then there’s gonna be this big reward in the end. I don’t know. I don’t think about it that way. I enjoy every day I go to work.

Of course, there’s, like, hard times. Like, there’s, you know, you have to have hard conversations, maybe fire people, maybe you, like, do hard things. But, like, in the grand scheme of things, like, I mean, such is life, and I enjoy life. I enjoy building a company. And I don’t think of it as like one day we’re gonna flip this company and be... I hope to work on this for twenty, thirty, forty, fifty years. I’m having so much fun. I don’t wanna sell this company because I would just go and start model two.

You really wouldn’t. I don’t think money necessarily makes you happy. I mean, little bit. Like, it’s good to have some money, but, you know, be on a point. Like, what... You know? I like building stuff. I love technology. I love coding. So much fun. You’re such a sweet... Is that Swedish is that Swedish mentality? Maybe it is. Yeah. Dude, never has

before and said it’s not about money. I mean, I do care about money a little bit, but maybe you’re right. Because I do think in Swedish mentality, it’s like self actualization. Like, doing what you enjoy is kind of the highest form of prestige in a certain way. And it could be music, it could be poetry, whatever. It almost doesn’t matter. What do you do that you don’t enjoy? Yeah. I kinda hate fundraising, honestly. I mean, it’s fine, I don’t know. I enjoy it when there’s good people, but it’s kinda challenging.

You have to say no to a lot people. I guess that’s a good problem to have. But in general, it’s

Know, I know. That’s a horrible thing. Oh, you can only have 10,000,000 not $50,000,000

a deck, have you? I’ve never made a pitch deck, yeah. Every round we’ve done so far, I like to get to know people well in advance of raising money. So every round we’ve done, I mean most recently we announced a d round yesterday with Accel. I’ve been speaking to Rich and Gonzalo at Accel for I think at least a year. And at some point, we’re like, actually, you know what? We could take some money. Like, what are the terms? Like, let’s discuss terms. And then it happened very quickly because we already had that relationship.

No. I don’t think so. I I think maybe in later stages, it becomes a little bit more of a commodity, like the money itself. But I I think in the early days, like, I mean, for for several reasons. Like, one is that there’s a misalignment in Senate. Like, VCs always wanna put in way more money than a company needs. And that, you know, you just take on a lot of dilution. It’s not good as a shareholder. But the other thing is, like, you actually wanna hold back, I think, and and it’s often better to raise a lot less money at a lower valuation and then keep the VCs hungry for the next round.

I think there are times in the market where you are capital constrained. We have not been capital constrained for most of the time of the company. We’ve been extremely capital efficient. We burn only a few million a month. I mean, like, know, it’s a lot of money, respect for the money. You

No, I mean, up until like last year. Now we have to spend a lot of money on capacity. How

A lot. I mean, we actually are profitable on a P and L basis, but cash flow is quite negative, because we have to spend a lot of money on getting GPUs and building out capacity. And so that is a different period in the company and how we think about capital. Capital is I don’t wanna call it a moat because, you know, I feel like people throw around that word, like, to mean things that kick Mote to me has, like, a very strict definition. But it certainly helps.

Capital helps at this point.

I mean, moat to me is like you go read like Seven Powers or like one of those

It’s a good book. But I think, you know, then people read that and they get delusional, like... And you know, they’re like, yeah, we’re starting this company. It has all these moats. You never had moats in the early days. Like, you know, like Google could do anything they wanted to. Google could do, yeah, but it’s also like, you know, like what are the seven, I don’t even remember. It’s like cornering resources, like market power, like whatever. It’s like none of those things happen until you’re like ten years in, you know?

There’s an old blog post, I always come back to it by Ben Thompson. It talks about Snap and the like, the gingerbread man strategy. Do you know this thing? And it’s like, the whole point is like, Snap doesn’t have a moat, they just run faster. Like, they’re the gingerbread man. And I actually always come back to that. I think that’s so true, like, for almost every startup these days. It’s like, you just have to, like, move faster than everyone else.

Yeah, I mean, again, I don’t know if I would... But I kind of agree with your point. I think brand, I don’t wanna call it a moat, but it’s actually, I think the value of brand has gone up a lot. Can I be a dick? Yeah, please. Do you think Modal have done well with brand? I think pretty well. I think we have, it’s coupled with the developer experience, and the product, and the website, and the visual identity, and all these things. We just did an event in San Francisco, and I thought a lot about that, sort of how we package it, the event, and the visual look and feel and But like, do you

I think we have that. I think we have like really, really good vibes. Like, I’m terminally online. Like, people tweet, like, way more about Modal. I think there’s, a little bit more of a community. I think there’s, like, much better vibes around us. Like... But, like, I am paranoid about those companies. Like, I am paranoid about competition. Of course, I track it. Like, I of course, like, I I see everything that gets tweeted and about our competitors, and this is important. Like, everyone wants to be a part of the community.

Everyone... You know what I think is, like, people misunderstand is I I love TypeSafe with JEV. Yeah. Because, like, everyone wants to clone JEV. No one wants to get up on stage and wear a fucking pink fur coat just, like, talk about the Jevolution in an unhinged way. That is what brand to me is. Like, it’s... Like, that guy is, like, he’s playing the game on a whole different level. And and I think it’s just, like, so, like, true about, like, companies today. Like, you have to think about that.

I think so. I think you just have to create a personality. You kind of have to, like, figure out, like, what is the personality of your company, and just be consistent, and create, like, almost like a sense of movement, a sense of, like, community. Like this is something cool, and people want to use you because you’re one of the cool products out there. They’re like,

started with the developer experience. So that was... And we had a very clear sort of PLG product market fit, product led growth, in the sense that people came bottoms up, they just sign up on a product. Like, I think the consumer background of coming from Spotify is very important in how I thought about building what’s now a B2B company, selling tech effectively. In a way, it’s like a very boring category, but I think, you know, like, having personality and having, you know, a little bit of an edge is a big part of how you create that sort of movement and that sort of like sense of excitement and the feeling that you’re different.

And I wanted to start the moment you go to a website, and the moment you try the product and you just feel that it’s different, that magic moment.

Of I I think inference is gonna be an enormously large market. It’s like the new cloud, basically. And... But I don’t think inference in itself is maybe, like, the only thing you need. One of the things that I feel very strongly about is, like, it has to come with, like, a lot of other products. It has to come with post training, sandboxes, agentic stuff, all kinds of, you know, other types of inference, like low latency inference, batch inference. So building that platform, by the way, we’ve been working on this for the last five years.

I think that to me is, you know, not a moat, but it’s a very hard product to build, that infrastructure.

And like, if you’re just selling, like, tokens for dollars, there’s no switching cost, you know? People can just switch to a different provider. It’s just a race to the bottom. But if you couple it with all these other things around it, sandboxes and post training and reinforcement learning and all these things, like, you can actually create a whole set of tools. Like, one wants to build a, you know, buy a single trick pony. People want to buy a portfolio of different things. Like, they want to buy, you know, a whole suite of tools that basically lets them build amazing AI applications on top of it.

Right? And the infrastructure underneath is a lot of differentiation, a lot of, you staying power in the markets. It’s very hard to copy like the infrastructure required to build all of these products.

One thing that people don’t see about the inference market is, I actually think in the long term, there is so much gross margin opportunity in this space, because you can build all these things around it. All the sandboxes, all the agentic stuff, workflows, post training reinforcement learning. If you do all those things, there’s actually so much opportunity to make money.

I think basically almost no company today has software margins anymore. Like what you used to see in Silicon Valley ten years ago, with this 90% margins plus, I think that’s basically gone. And the reason is actually almost topological. It’s like you have this like stack of vendors, right? Like today you have applications, you have models, you have infrastructure, you have cloud, you have NVIDIA. And so when a customer pays a dollar at the top of the stack, it has to flow through a lot of different every layer, there’s going to be, you know, 20%, 30% gross margins.

So I think there’s a lot of margin opportunity, and there’s a lot of opportunity to make a lot of money. I mean, the space is so big, but you’re also splitting it across a lot of layers in the stack. So, I mean, you know, like the 90% software margin businesses are gone. I think Modal has an opportunity long term to get to 40%, 50%, 60% margins, but we’re never going to get to software margins.

I one prediction about the inference market is gonna split into a lot of different things. Because the market got so big in the last... I don’t think anyone expected, me included, I don’t think anyone expected it to be, you know, so big. It’s like 10x or 100x bigger than I ever thought. And that to me means there’s gonna be so much room for specialization, low latency, high latency, like various types of fine tuning and post training. I think there’s gonna be some wacky new technologies that let you do it with way less resources, new hardware, like all kinds of different things.

Specialization by product. I think it’s gonna fragment into a lot of different products. Now we can build a lot of those products and host it all under Modal. And that to me is very exciting because a lot of different, you know, there’s gonna be a lot of different products that companies are gonna need. For one product, they’re gonna need low latency inference. For one product, they’re gonna need batch inference. For one product, they’re gonna need sandboxes, which we have. For one product, they’re gonna do pre training their own models, you know, maybe it’s, you know, certain specialized, you know, audio models or whatever.

Like we do a lot of stuff besides LLM inference. And so that to me, that creates a lot of opportunity to basically kind of rethink what a cloud looks like in the era of AI, and kind of deconstruct the cloud and rebuild it for this new era of software.

So it is pretty crazy. I spent some time in March at GTC NVIDIA’s big conference in San Jose. I would describe that as like an open air drug market, but GPUs instead of drugs. You’re running around, you know, talking to 500 different cloud vendors. You’re like, do you have GPUs? Like, please get... And like, know, in the end, you come out, like you do a lot of deals, and that’s something we’ve had to develop the muscle. I think part of the value prop of Modal is like, our customers don’t have to think about that.

We do the dirty work of doing all the GPU deals under the hood. We work with 25 different neoclouds. Or, you know, right now we control like, you know, 0.1% or whatever, I don’t even know the number, of like the world’s compute. But like, my like crazy, like, you know, how I think about the long term goal is like, as we control more and more of the world’s compute, you and aggregate demand on the other side, and manage that capacity and demand extremely efficiently. So that to me is like something we actually think we can do, we can fill a big gap.

Well, so we built a lot of our infrastructure ourselves. Ourselves. We don’t use Kubernetes and Docker and things like that. We built our own file system, we built our own container runtime, we built our own scheduler, we built our own, basically how we manage the capacity. And that enables us to basically treat like all of the world’s compute as one big pool of capacity. And so if you think about the fact that there’s... We already have many thousand paying customers, and over time it just grows and grows.

Each of those customers have like very sort of variable requirements, like their workloads kind of vary over the day, and they have maybe bursts, and each individually, they’re like very sort of unpredictable. But if you pull all of those customers together, you end up with a very sort of smooth overall demand. And that is something we can manage extremely efficiently. And so one of the valid props of Modal besides, like, we can scale up with you is also, you don’t even have to do capacity planning.

Just come to us, we’ll give you 100 GPUs. I think it’s so amazing going back to TypeSay, for instance, like they, we had talked to them, I think a long time ago before they launched, but they launched overnight on Modal and we weren’t even aware of it to just like scale up. And we gave them a ton of GPUs and it just happened automatically. So that magic, I think, is something that is people really value with Modal is like we can give them a lot of GPUs on very short notice.

And we can handle that sort of variable. Like if you go to us, you don’t have to lock up, you know, like a rectangular block of GPUs over three years, you and have to know the exact number. It’s pay as you go. You scale up and down. You pay per GPU second.

lead times right now is about three to four months. So we basically do a straight line or exponential line extrapolation, and we look at an aggregate across all our customers. We have a very diversified set of customers. You can actually look at the demand in aggregate pretty well and predict based on monthly growth rates, which have been about 30, 40% month to month for the last twelve months. And you just project that, you know, three to four months into the future, you end up with like two to 3x.

You go out and, you know, buy that capacity and make sure to ramp up, and sometimes But

You can. Yeah. I mean, I think there’s a little bit of like a misconception that like we’re out of capacity. I mean, NVIDIA is like shipping like an insane amount of GPUs like every month. I don’t know how many 100,000 black holes are getting installed. Prices are up for sure. You have to pay more to get the same amount of GPUs. And I don’t wanna say it’s like an efficient market, but if you’re willing to pay more than other people, you can get a lot of GPUs.

Is there a price at which it doesn’t make sense for

I don’t think so. I think, in fact, like, we’re actually more long than we’re short GPUs because we we can charge a higher price to our customers. Like, there’s effectively a market price, and it’s, like, different dynamics because we lock up longer term reservations and and rent them out on a slightly shorter duration. But, you know, if if the underlying price goes up, people are also willing to pay more.

idea. Actually, it’s funny. Well, dude, if you have no idea, what am I meant to have? Did you see Jensen yesterday? He was in London, actually. Asked him this question. He’s like, well, obviously, like, supply is gonna catch up with demand eventually, and then we’ll see what happens with pricing. It might happen in a year. It might happen within five years. He said exactly the same thing I always tell people. Like, gravity. Hardware prices always go down in the very long run. Right now, we’re in a very...

You know, there’s a supply shortage, and, you know, that means that the prices have gone up. They may continue to go up for another year, two years, five years. I don’t know. But eventually, they will come down.

I think we’re very early. It’s... I think you’re saying the same thing. I mean, yeah, Javan’s power, whatever. Like, I think there’s just so much latent demand for these things. And, you know, I I think, you know, eventually we’re gonna start to run up against these capital constraints. Right? Like, there’s rates that are going up. Like, we just can’t build data centers fast enough. Maybe that’s good. I don’t know. Like, I think in a way, like, it’s a guarantee against like overbuild, but I also don’t know if we can overbuild these things.

There’s just so much demand for it.

No. I think what could happen is like people realize Instinct is a good example. Like, allegedly, they don’t charge or they don’t make revenue yet. So I think it’s a good example. But, like, clearly Instinct can make a ton of money in the future. But so, you know, that’s a dynamic in the market. There’s a lot of companies that are not yet making money yet, and they’re using a lot of GPUs. They just have to eventually start to make money. But, like, that that lag could take a while.

Right? And and that... We might run up against that.

I think you’re... Try to phrase it as, like, one versus the other. My my view is, like... I think Erik Fisher was actually... It’s funny. Like, it’s the same thing. Like, all of these things are gonna grow. Like, I think NVIDIA is gonna keep growing. They’re gonna keep shipping amazing chips. I think there’s also gonna be, like, Cerebras and all these etched and all these other ones, like, shipping specialized chips, they’re also gonna grow. So I I think there’s gonna be markets for all of it.

So why is Cerebras down 30%? I don’t know. I don’t follow, really, the stock market. You don’t follow the stock market? No. I I mean, like, a very passive, you know, buy the index, mostly because of my own time. Like, I don’t feel like I have the energy to follow the stock market. To what extent is

I actually, I think it’s a little overstated. I don’t think CUDA itself is the moat. CUDA is actually kind of annoying to work with. It’s like a pretty low programming model. But I do think all the NVIDIA, the ecosystem is. Like, I think all the kernel optimizations and all the stuff that, like Torch always supports NVIDIA much better than anything else. And the open source engines like VLM and SD LAN because of that, they always support it much better. So like everything is NVIDIA first today.

And that’s probably going to continue. It’s like funny for the last like fifteen years I’ve been saying, you know, like next year is the year of non NVIDIA accelerators. It never keeps happening. I’m always wrong. NVIDIA has just, you know, shown to to be incredibly good at having that staying power and just dominating the market. We’ll see. I mean, there’s, like, Tranium, there’s TPUs, there’s AMD. There’s a lot of other stuff that does look quite good, and they’re catching up. But, you know, I still think they’re quite behind.

in StartUp Planet is like a friend and a foe at the same time. They have this, like, weird sort of, you know, relationship with everyone. I I think some of the top neoclouds, Nebius included, they are trying to move up the stack. I’m personally somewhat skeptical. I think it’s, like, very hard to be good at, like, your core business and also trying to move into different market at the same time. Why would it be hard for them? Like, what’s the bare case on them being able to move into the inference layer efficiently?

I mean, I’m biased, but, like, we spent the last five years building an incredibly complex infrastructure stack. I think it’s very, very hard to replicate what we built. What was the hardest thing? All the code and everything around the file system, the container runtime, the scheduler, like building the system that can place millions of sandboxes every minute, scale up and down many, many thousands of GPUs, you know, power hundreds of thousands of tasks running concurrently. These are these are very hard engineering problems. I I mean, I’m I’m an infra geek.

Like, I love that shit. It’s so much fun to work on it, but, like, these are hard problems. Is Nebius a better business than CoreWeave? I actually don’t know. Like, I I don’t know. I I think they’re both, like... They’re they’re executed extremely well, both of them. I have a tremendous respect for what we’ve done. I think at the at the end of the day, like, the thing that I have the most respect for is actually more on the financial engineering side. They’re very good at that of stuff.

Because I think a lot of the constraint for building a neocloud very quickly is on the financial side. You need to be very good at... You go lock in like a long term customer contract, then you go raise a bunch of money against it, then you go buy the chips, then you deliver the chips, and then you pay off the debt, like over the next few years. Doing that sort of dance with the financials, like it’s kind of an interesting game. And CoreWeave has done it exceptionally well.

It’s a capital game. It’s a financial game.

I think open source is always gonna be... There’s always gonna be great models coming out. Let’s also remember, like the open source models, they’re all moving into licensing models. So like, they’re getting better at capturing the value of those models. And I think that means they’re gonna have more funding to train better models. And I really hope there’s gonna be European and US models because I don’t think it’s a good situation that we have so many Chinese models. I mean, they’re they’re obviously... I have tremendous respect for all the Chinese labs.

They’re extremely good at what they’re doing. But I think it’s good for the world if there’s, you know, more power spread out. But anyway, going back to your question, I do think that open source, there’s always going to be people who want to fine tune, want to have more control over their models and sort of tweak to parameters, like go low latency, high throughput, like whatever it is. And open source is gonna play a big role in that. It’s like Windows versus Linux. With the... I am sorry, with the greatest of respect, do

Yeah. I mean, I love Europe. I grew up in Europe. Grew up in Sweden, but I do think that Europe does struggle with accumulating large amounts of capital. I think both of those things are things that Europe struggles with. And and that is something that China has done really well. They’ve accumulated massive amounts of human capital. And and US obviously has very good, you know, very good at, like, raising enormous amounts of money and, like, training very large models. I don’t know if Europe has the...

I think NVIDIA is very smart to focus on the ecosystem, building up an ecosystem. And I think that includes like propping up a lot of open source models. And I think they probably saw an opportunity to pick up open source models. These companies are very resource intensive, right, very CapEx intensive. And I think it’s smart of NVIDIA to like buy some of those and sponsor them effectively in order to create American built models. By the way, I wanna take back one thing I said. Europe does have an insane amount of human capital.

I do think what Europe is very bad at is aggregating them and putting all of them at the same company. That is something I think US is very good at.

Yeah, yeah, yeah. I mean, it’s almost like too much on the other side to that point, but Oh, when you were four, you were encouraged to start, leave school, drop out, kindergarten wasn’t worth it.

I think so, and I think that’s a function of what I talked about earlier, where the market for inference is so large, and that when a market is a billion dollars, you’re not gonna have too many submarkets. But when a market is starting to get 10,000,000,000, 100,000,000,000, whatever, you’re gonna have a lot of different submarkets. And JEF does classification and things like that, regression type things very, very well. And there’s a large enough market for that.

I told you before, but like, I think the brand and the community and the movement around, you know, just this sort of everything they’re doing is so incredible. And that to me is like a massive, massive asset in this new age. That’s a $100,000,000,000 company. I think there is an opportunity to create a $100,000,000,000 company with a lot of specialized models and inference. You would have done it though at 8,500,000,000. I would have done it at 8,500,000,000. I I think there’s a pretty big outcome here.

Could be an exit, could be, you know, like a public company, who knows? But but I do think there’s a lot of upside here.

Yeah. Fantastic guy. Love that guy. Is he great? I mean, you know, his charisma, his like... You have to create like a personality, like, around your brand. You have to create this like visual identity. It’s obviously not the visual identity I would have picked. I hear you, but I think then you should be more loud, if I’m just honest,

I think that’s a fair point of criticism. I think I could be out there even more. I tried to tweet. I tried blogging. I’ve been, like, kinda bad at that recently. No one gives

Yeah. Maybe, right? Maybe I just need to, like, go more unhinged on Twitter. It’s like fund related marketing, basically. Like, in the early days of of Modal, like, that was a big part of our growth. Right? Like me just like tweeting all the time about what we’re building. And I think that’s really important. I mean, every company has to find its own sort of, you know, go to market strategy. But in the early days, like we spent a lot of time just talking to people on Twitter, just like hitting people up, try the product, what do you think?

Of them ended up angel investing. You know, they’re now supporters of Modal. And, you know, you have to lean into what you’re good at. And in the early days, I did a lot of shit posting.

It’s It’s pretty smart. I love what you’ve been doing in terms of building up this media brand. It’s fantastic.

I think the biggest thing we need is we need way more products in in Modal. Like, we wanna create a busy comprehensive experience of everything you need to do in order to build an AI application should be a product inside Modal. The other thing we can do is hire way more FTEs and way more sales reps. I will say we’ve been behind on that, but we should hire like way more people that reach out to you personally and say, Hey, I saw you get started in Modal.

I saw you’re working in the bio space. How can we help you? Here’s some bio examples, you know, protein folding, whatever. We work with a bunch of other similar companies. You seem skeptical.

the B2B space, so I think it’s like, you know... But

a lot of them. But but I mean, I I sort of agree with your higher level premise. Like, if there’s anything that’s, like, super critical and never goes away, it’s prioritization and resource allocation. Right? Like you can’t boil the water and try to do everything at the same time. You need to be extremely ruthless. And having a multiproduct strategy, think is a very hard balance. Like when do you branch out into a second product or a third product? Right? Like it took us four years to see success with a second product, which was sandboxes.

Yeah. So we launched sandboxes in 2023. Didn’t see a lot of success in the early days. I mean, there was like some success, you know, it wasn’t horrible, but, you know, it was a relatively insignificant part of our revenue up until summer of twenty twenty five when models, I think like Sony 4.5, one of those, like, just got really good at generating code and building whole apps. And then it just exploded. I mean, sandbox is today is a 300,000,000 run rate business, about a third of our revenue.

And so that’s been a kind of a crazy journey. And now we’re thinking about, you know, what’s the third and the fourth. We just launched a training product. I’m super excited about that. Working on batch products, working on reinforcement learning. We’re working on all kinds of other stuff. And so that evolution into becoming a platform, it’s actually there. Like if you go back to the very early decks of Modal, you know, I put together some just for my own thinking. It was all there.

It’s like we had this like market map and all these, you know, it ended up being very different products than we thought, but it’s so important for me to create, like, you know, to focus on a category and focus on a set of solutions, not just, like, be a single single widget vendor.

It actually is like one of those products. I I think one of the benefits of being in infrastructure, but also one of the drawbacks in a weird way is, like, you kind of just have to trust your gut on, like, what are, like, important primitives and just build, like, amazing building blocks. I think we’re in the space of, like, providing building blocks. We try to look at, like, what do engineers need and not, like, listen too carefully because, like, they’ll ask for, like, all kinds of random stuff.

Distill it down into like, what are some of the building blocks? And we put out a serverless function product in 2021. It saw success in 2023 with gen AI inference. We put out a sandbox product in 2023 because we saw a lot of people using LLM to generate code. And then it saw success in 2025, like I said. So a lot of like how we think about building products has always been kind of like, let’s just put out like good building blocks and then it exploded.

But like to answer your question, it was reinforcement learning, it was vibe coding, background agents, and maybe some other stuff too in the future. Those were things we actually didn’t anticipate. So I think infrastructure is often like that. Like we didn’t anticipate stable fusion when we started building our serverless function product. It just happened

Sometimes, I think that’s right. I mean, we’ve definitely been extremely lucky in certain ways with timing. Like we put out products like right before we started the company in 2021, and then Stable Fusion and Chat came out 2022. We had really good timing with that, right? It just created so much demand and we’re very well positioned. But some of it is also like with infrastructure, like, again, like, I think it’s about like building products that, you know, you anticipate will be useful in the future. And from your own experience are things that you always wanted to see.

I think you have to kinda trust your gut in that.

I think we should have allocated more resources towards LLM inference like earlier. I think we’re a little bit on the sidelines for a while. We didn’t realize how big open source inference would be, and we thought it would be kind of a race to the bottom, just a commodity space. And then Deep Sea came out, we’re like, fuck. Like, we gotta move very fast. And so we staffed up a team immediately, you know, went from, like, you know, one or two people working on this to, like, 30 people twelve months later, and and now we’re one of the top three providers.

I think that’s a little bit of a fixed pie size mental... Like, I I I think, you know, with with all due respect, I do think both spaces will grow five x, 10 x, 20 x over the next twelve months and and beyond. Like... And and so, like, it’s hard to argue, like, you know, which one’s gonna be bigger than the other, but I think both will grow, like, in, you know, orders of magnitude over the next few years.

It feels pretty cool. It’s, you know, like, I I don’t know. Like, I’m just like, you know, Swedish guy that, you know, grew up in a random. For me, it’s like these numbers are absurd, but like, yeah, it feels great. It’s so much money.

don’t know.

I don’t know. What I can say is we’ve been growing... We grew 15x in the last twelve months. So you can extrapolate that. I mean, I don’t know. Like at some point we start... Markets just get very big and they start to slow down a little bit. But with the growth rate right now, like the growth rates are not slowing down. Like we’re growing 30, 40 So you percent a

I hope to be way beyond that.

I think it’s cheap. I mean, like, I wouldn’t do this if I didn’t think we could grow and become, like, a generational company. I think there’s so much upside. I think this is one of the largest markets opportunities that’s ever existed. I think that the demand for compute, the demand for AI applications, it’s gonna be enormous, and they’re all gonna need a place to run. And I hope a lot of them are gonna run a model.

That is a good question. I think training is something we just launched. I think that could be very large. I hope that could get to a 100,000,000 run rate in the next six to twelve months. Post training, similarly, working on batch jobs. I’m very excited also about a lot of like sort of traditional, like data warehousing stuff, databases, workflow orchestration, feature stores, like vector database. And there’s just so many parts of this. And they all kind of fit together very nicely. And that to me is like what I’m so excited about.

It’s like, once you start building in this space, there’s so many natural adjacencies. Like this is like a problem with me. It’s like, I get excited about, so I have like no lack of ideas. Like at Modal, like we have so many ideas of things we wanna go. I would love to get into continuous integration. That’s like a very interesting space. Agent harnesses. Maybe we should train models. Maybe, you know, there’s just 5,000 directions, you know, and that’s like my ADD brain, like wants to run-in all of those directions.

You have to be obviously ruthless with prioritization and only take on, like, one or two things at at the right time, but but there’s so many different directions we can go on. It’s amazing that your wife calls you for decisions.

I just wanted some curtains done. Yeah. You know, with the curtains, I just like you pick one of them. I’ve seen Simpsons and, like, Rainy Wolf Castle becomes the president, and they’re like, which one do you wanna pick? You know, one to seven. He’s like, I was born to lead, not to read. Number four.

I don’t know. I think post training works because there’s a lot of tasks that people do in real life that have fairly finite sort of feedback mechanisms. So coding is like a great example, right? Like you can do coding with reinforcement learning pretty well because, you know, you can run code in sandboxes. And a lot of labs are running modal sandboxes for that exact purpose. Start to look at cyber security is another area. Math, obviously, with lean. I mean, seen some fantastic product. I think it’s kind of incredible, all these like OpenAI proofs the other day.

So law, maybe to some extent, you can, you know, do like eval. So I think I think post training right now, like, there’s like such incredible focus on it because you have all these tasks that traditionally only humans can do, but actually well within reach for for agents and and for LMs. But like, I don’t know if it extends beyond that. Like, I don’t know. Like there’s a lot of tasks out there that are like have way longer feedback loops that I don’t know if like post training quite yet are is within reach.

A lot of people ask me like, what’s going happen with software engineers in the long term? And I think, you know, we’re probably gonna automate everything that an agent can do within twenty four hours or a week or something like that. But I kind of feel like, you know, engineers, like they work on a longer time scale and they have feedback loops that are longer. So I have a feeling like reinforcement learning, we’re gonna start to run into some limitations around that. But I’ve been wrong for so long, so maybe I’ll be wrong and just keeps going.

And eventually we can do reinforcement learning on things that take six months for a human. I don’t know. What does the day look like in three years for an engineer? I mean, I’m extremely bullish on software engineering. I’ve been coding for thirty years, and I’ve seen so many different changes with like programming languages and the cloud and so many different frameworks. And every single time we make programmers more efficient, it just increases demand for software. There’s so many parts of the world that desperately need software.

And so I’m very, very bullish on software and the need for software engineering. And I think one of the biggest misconceptions from the outside is that developer is about like translating from English to code. Like that has never been in the job of software engineers. It’s like to understand the requirements in the first place and be kind of an interface between the humans. You’re building for humans at the end of the day and like trying to figure out like, what does that even mean in terms of processes and workflows?

That continuously moves up in the stack, you know. Back in the sixties, we used to write assembly and punch cards, and then we moved to COBOL, and then to C plus plus and then to Python, and now we’re using agents. And that’s just like one it’s definitely like the biggest thing that I’ve seen in the last thirty years. But I do think we have many, many analogs and many parallels throughout the history of software and computers. To me, that makes me feel like, you know, software engineers are gonna be fine for the next ten, twenty years.

think a 500,000 or something like that. How many engineers do have? A 150. Dude, that’s so worrying, you know. You wanna... Should be token maxing more?

Yeah, we’re not quite there yet. But I will say, our software engineers are extremely agent plus, including myself. I mean, I write code with agents all the time. I

just gonna eat more. Mean, like, again, like we’ve seen this so many times, like you make something more efficient, you make something more powerful, it just unlocks like five new use cases, then and demand just goes up in the end. Like, that’s that’s just the history of like the last fifty years of computers and software. But like, I wouldn’t be here if I wasn’t such an optimist. That’s why I started a company. Know, like, I’m just like the biggest, like, optimist that there is in the world about software and computers, and I just love this shit.

And I think it’s just gonna keep growing.

Yeah, it’s incredible. If I was 20, and I just wanted to make a lot of money, I would go out and start an environment company and just sell data to frontier labs. There is so much money to be made in that space, it’s kind of incredible. Let’s go a layer deeper. What would you do next? Do you specialize on a specific market? I mean, the coding space is where you see the most demand right now. If you can sell code environments to the labs, you can make a lot of money.

It’s a big space in that. Still? Still, yeah. Yeah, I meet startups all the time. I angel invested in a bunch of that. Mean, they’re incredible businesses. What I will say, my one piece of advice is make a lot of money today, but like, is this a venture fund? Do you wanna get on the venture treadmill? I’m not sure. Like, I think these are companies that are gonna make a lot of money today. I don’t know if they’re gonna make a lot of money in the very long run, but like who cares?

Like, take the money. It’s like great money. Like it’s a fantastic business today, and that’s all that matters.

For sure. Yeah. And I think it’s gonna be a tough business in the very long run, but today, you can have extremely good cash flow selling these data sets.

Yeah. For some stuff, it’s good to own your own intelligence. For other stuff, it’s not... Like, I don’t think it matters. Like, we have a lot of customers that own proprietary models on Model two, and their customers, know, they use a proprietary model. Like, I don’t don’t think it makes sense for everyone to own their own intelligence. Why does it make sense for you two versus not? I think for certain use cases, you can take an open source model and you can fine tune it, and you can get really good performance.

And the nice thing is you know what the fine tuned model is based on, it’s not gonna change, like, they’re not gonna take away the model from you. It’s nice to have that consistency and the ability to like, continuously like tweak it without, you know, the underlying model changing, which has happened in some cases with the frontier labs. That’s like, you know, that’s nice. Like, you know, I support that, but like, you know, I think there’s many other things that matter to people. Owning your intelligence is like just one thing out of many.

Like, I think latency matters, think costs matter, performance, I think hackability, I think, you know, having a platform of a lot of adjacent services, like, these things matter too. Owning your own intelligence, like, yeah, I’m supportive, but there’s many other things.

No, I don’t think so.

Well, I mean, not really. If you think about TypeSafe with Java, it’s a highly proprietary model. I want people to use Java. It runs on Modal. So we have many customers that themselves are proprietary model, that are hosting on Modal, that are serving their customers. And I’m very, very in favor of that too. Think the beauty of Modal is always that it’s like a general purpose platform, and we support so many different things. That to me is like, you know, the beauty. It’s like, you know, we can, if want you to use open source models, like, that’s great.

Like, we’ll help you. And there’s many customers that build incredible AI applications on top of open source models, fine tuning them, reinforcement learning, have a bunch of, you know, adjacent sandbox use cases or training, whatever. But there’s a lot of companies that have built highly successful businesses, you know, Suno, Runway, TypeSafe, and many others like running their own proprietary model on Modal, and their customers don’t care. And that I think is like, you know, puts us in a really good spot.

saying in UK?

I think it’s possible to build a successful business in model routing. I think you can do that today as a standalone product. I wonder if long term it has to be part of a more comprehensive platform. At the end of the day, think what these companies want, they want evals, they want ways to fine tune, they want more control over performance, like latency, throughput, like all these things. And I think it’s like one of those things that kind of makes sense today. Like these are very good businesses.

I invested in open router personally, and they had a great outcome. I actually think they could have done even better on a standalone basis. But I think in the very long term, it it feels like one of those categories, like, you know... And and... But that’s my view on everything in AI. It should really be a part of, you know, larger, more comprehensive set of tools.

OpenRouter, like the growth rate and the revenue, like this is... And and the brand, frankly, like, they have a really strong brand. Mean, I there’s a lot of, like, there’s 500 other competitors, but but the the space is growing so fast and there’s so much revenue I would have told them to keep going.

Well, it’s kinda bad if you wanna make money, right? But it’s great if you wanna develop a relationship with founders. Sure.

You’re against the relationship with founders? No,

I don’t know. I mean, think one thought I have is like, obviously at some point, this AI stuff is growing so fast, and we’re here, and we’re gonna go to here, and then there’s gonna be a pullback, you know, 20%, 30% in the stock market, everyone’s gonna freak out, and people are gonna say it’s the end of the world. I’m almost certain it’s gonna happen, but, like, I’m so bullish, because then we’re gonna keep going. And, like, you know, like, so that would be, like, one thing that I’m like, you know There’s gonna be a pullback.

I’m okay with that, because I think the long term opportunity is so fucking big. It’s, like, so large.

I mean, I personally don’t need a lot of money. I’m pretty happy with life, and I have enough. So personally, I actually support it for my employees, though. It’s it’s quite important, actually, as a founder that you make the equity worth as much as possible, and one of the way to do that and to tell future candidates that, No, this isn’t just paper money, this is real money, is to have a precedent of doing employee secondaries. Then you can point to last round, we let employees sell this much or whatever, and up to 10%.

Like, I think you have to have guardrails. Like you don’t want employees to sell too much. You want them to still be very, you know, have a lot of skin in the game and be very incentivized to make the company successful. I But think it’s a way to, you know, create a very liquid, hard currency out of the equity.

It’s nuts. It’s nuts. I mean, you know, like, the offers we see for... We’ve lost new grads going to some of the labs for, like, talking 20 plus million. I mean, these are exceptional people. Exceptional.

You know, like, just fucking sharp people. You know? These are like... I I know it when I see it. Like, they’re just, you know, they’re just extremely hungry, hardworking, I do everything about mean, they go to some of the big labs. Is that it? They go to OpenAI and Anthropic? Yeah, the big labs. Yeah, and they pay a lot. But is that where the talent comes? There’s like a few other ones, but I mean, mainly those people are known for paying a lot of money. And you won’t pay that much?

We pay a lot. We can go very aggressive. We wanna be out there paying 95... Fifth percentile for... In the market. And there’s like a few companies that I know, like, they’re probably gonna pay a little bit more. I think we have way more upside in the equity. And going back to what I said about secondaries, I think it’s important that for that reason also like to create like a real tangible value and the currency in the equity that makes us, you know, our offers seem very, very strong.

Plus the fact that we have a lot of upside. I mean, you said it, right? Like our stock seems cheap. Like I kind of agree, there’s a lot of upside in our stock. So that enables us to not necessarily have to match with the labs, but I think, you know, create a compelling sales pitch for why our comp package is, like, basically better than the labs. And we’re willing to go very far to get the best talent.

I love Turbo Puffer. I know Simon really well. He’s a Danish guy. He’s got this Scandinavian connection. Just a fantastic entrepreneur. I just love the business. How big can that business be? That’s a good question. I mean, it is, like, somewhat specialized. They’re doing vector databases. I think it could be a very large category. If I ran that company, I would have 500 different ideas of how to expand into many adjacent things around databases, data warehouses, other types of querying system. Like there’s... It could be a large category.

Probably the magnitude of the sandbox space. I think it’s kind of insane. Like, I look at the revenue, and it just keeps going up. And I’m like, well, clearly has to stop right now. And I think I’m, like, doing some sort of reverse psychology on the revenue because it just keeps going up. I’m like, what is going on? It just keeps growing and growing. And, like, I’ve been wrong so many times. I’m like, you know, in a good way, like, you know, and and so I...

It’s incredible how large that space has gotten.

Oh, boy. I I don’t know. I I don’t think anything is overheated. I will say, like, one of the things that I’m a little nervous about is companies that are dependent on continual infusion of equity capital. I would not invest in them. And there’s a lot of them that I... I would be nervous about anything where, like, if there’s a venture capital pullback, they will struggle to raise more money. I think it’s important you raise the the... Every round as if it’s your last round, and there’s, like, a real path towards commercialization and revenue.

I mean, sometimes I feel like it’s kinda similar. Like, you mean, like, in terms of, like, how to balance it, or do you mean in terms of just, like, being a good dad? What

Setting them up for success in life. And I always, like, think about, like, the right amount to push the kids. I’m not a tiger dad, but, like, I think a little bit you need to push the kids a little bit to, like, get them exposure. I don’t want to create, you know, in the future a situation where my kids come back to me and say, Hey, dad, I wish you’d made me do X, or I wish you’d made me do Y. But I also don’t want to create a situation where my kids come back to me when they’re 20 and they say, Hey, dad, why did you make us do X?

Or why did you make us do Y? So a little bit of pushing. I make them play piano. I don’t know why. But like, I think it’s good way to, like, learn something that I hope, you know, when they’re 20, 30, they will actually appreciate. They don’t like it that much. But I think stuff like that, you gotta push them a little bit, but not too much. My my parents are very lax, and, you know, I discover math and and programming. And they were like, Erik, why are you doing so much math?

Like, you should go out and play like the other kids. So in a way, I actually wish they would have pushed me even harder, but I pushed myself so hard anyway, so it doesn’t matter. So so you let... That, like, right amount of push.

But I I don’t know if that’s Like, I think there’s a lot of misconceptions to Scandinavian culture. I do think there’s a lot of, like, respect for people who follow their dream and, like, do things. And, like, whether, like I said, you know, it’s music or writing books or, you know, making furniture or art, like... Or being an entrepreneur. Like, I actually think there’s a lot of respect for I think this, like, self actualization and following your dreams is, like, kind of highly regarded. And I think, you know, like, that’s the flip side of having a very lax culture is, like, you kind of pursue people to follow their dreams.

And I don’t know, my dream was, like, doing math and programming. I love that shit, so I did that.

Yeah, I mean, Norway has some issues.

Maybe it’s like the first post economic society, I don’t know.

In

discovery is one of the applications I think is so incredibly cool. I can’t wait to take AI generated drugs. You know, I’m like old and my skin is getting bad and like, I don’t know, like, you know, every time I drink a beer, I feel like horrible the day after. Like, I don’t know, give me all the AI generated drugs I can take. Like, I want to live long, want to live a healthy life. You know, and I think it has Do you think that is actually a realistic future?

I think so. But I think with the FDA and other things, it’s gonna take a while. But I think there’s so much incredible progress today in AI aided drug design. I think there’s so many cool stuff, so many things.

Thank you so much for having me. This is a lot of fun.