Raising $500M To Compete in the Race for AGI

with me, Harry Stebbings, and there could not be a better time this episode. Just last week, Poolside announced their series b, a $500,000,000 round, valuing the company at $3,000,000,000. Today, we’re joined by their cofounder and CEO, Eiso Kant. This is an incredible episode on the future of LLMs, the race for AGI, how the chip and compute layer evolves, and so much more. But before we dive in, this episode is presented by Brex, the financial stack founders can bank on. Brex knows that nearly 40% of startups fail because they run out of cash, so they built a banking experience that takes every dollar further.

It’s a stark difference from traditional banking options that leave your cash sitting idle while chipping away at it with fees. To help you protect your cash and extend your runway, Brex combined the best things about checking, treasury, and FDIC insurance in one powerhouse account. You can send and receive money worldwide at lightning speed. You can get 20 x the standard FDIC protection through program banks, and you can earn industry leading yield from your first dollar while still being able to access your funds anytime. Brex is a top choice for startups.

In fact, it’s used by one in every three startups in The US. To join them, visit brex.com/startups. And finally, let’s talk about Squarespace. Squarespace is the all in one website platform for entrepreneurs to stand out and succeed online. Whether you’re just starting out or managing a growing brand, Squarespace makes it easy to create a beautiful website, engage with your audience, and sell anything from products to content all in one place, all on your terms. What’s blown me away is the Squarespace blueprint AI and SEO tools.

It’s like crafting your site with a guided system, ensuring it not only reflects your unique style, but also ranks well on search engines. Plus, their flexible payment options cater to every customer’s needs, making transactions smooth and hassle free. And the Squarespace AI? It’s a content wizard helping you whip up text that truly resonates with your brand voice. So if you’re ready to get started, head to squarespace.com for a free trial. And when you’re ready to launch, go to squarespace.com/20vc to save 10% off your first purchase of a website or domain.

And finally, before we dive into the show, I

Well, thank you, Harry. It’s a pleasure to be here, and it’s glad that we finally met in person. It’s been a minute since we’ve known each other.

Poolside’s in the race towards AGI. We think the future is gonna play out, that the gap between machine intelligence and human level capabilities is gonna continue to decrease. But are the path towards that, in our opinion, is by focusing on building the most capable AI for software development. And all of this comes back to a set of foundational beliefs that we have that I would say are different than some of the other companies in this space in terms of where both research is heading and where capabilities are heading.

The term AGI is a loaded term. And the way that I like to take the definition that is most commonly used is that at some point we are going to be in a world where across all sets of capabilities that we have as human beings, machine intelligence is going to be as capable and if not more capable than us and surpass us.

Now, our point of view is is that that world is still quite a bit out and that we are actually going to end up in a place before that where we see human level capabilities in areas that are massively economically valuable and can drive abundance in the world for all of us, that are not gonna be equally distributed, not for every single thing. And what I mean by that is that if you think about foundation models today, and I have a kind of simple mental model about them, which is that we are taking large web skill data and compressing it into a neural net, and we’re forcing generalization and learning.

And this has led to things like incredible language understanding in these models. But it’s also led to things where we look at and we say, these models are kind of dumb. Why aren’t they able to do x, y, or z? And our point of view is that the reason why they’re not able to do x, y, or z has to do with how they learn. The most important part, I think, of what I said is the scale of data. When we have web scale data, we can get language understanding.

But when we have areas where we have very little data, models really struggle to learn truly more capable areas. And I mean improvements in reasoning, improvements in planning capabilities, improvement in deep understanding of things. While as humans, we don’t require so much data, the way to think about models is that they require magnitudes order more data to learn the same thing. Our focus is on software development and coding, and it’s for a very specific reason. The world has already generated an incredibly large dataset of code.

To put a little bit into context, like usable code for training, what we refer to as about 3,000,000,000,000 tokens. And if you look at kind of usable language in English on the Internet for training, we’re talking about anywhere between ten and fifteen training tokens. There’s a massive amount of code that the world has developed. Over 400,000,000 code bases are publicly on the internet. So why don’t we have this incredible AI that’s able to already, you know, do everything in coding? It’s because coding is not just about the output of the work.

Right? The code that we have online represents the final product, but it doesn’t represent all of the thinking and actions that we took to get there. And that’s the missing dataset. The missing dataset in the world to go from where models are today to being as capable as humans at building software is the data set that represents being given the task, all of your intermediate reasoning and thinking, the steps that you do, the code that you write and try to run and then it fills and you learn from that those interactions, and all the way to kind of getting that final product.

And that intermediate dataset, that’s what Poolside exists on creating.

This is the right question. The way I think about the world is that there are problems that we cannot simulate. The real world is impossible to perfectly simulate. It’s messy. It’s multivariable. How do we deal with the real world when we’re trying to close the gap between human capabilities and AI? We have to gather data. The best example of this is Elon and Tesla. Elon has put millions of cars on the road that are actually capturing every single engagement and disengagement with autopilot and every single scenario, and extending that back to Tesla to train increasingly more capable AI.

And if you look at how full self driving got more capable over the years, it’s directly relational to becoming more and more learning from the data instead of rule based, and more and more cars on the road. And so to me, Elon has one full self driving. It’s inevitable that the most capable AI for full self driving is coming out of Tesla because they’ve been gathering and building up this dataset. And he needs to gather data because it’s non simulatable. Now this is the head fake behind Poolside.

You think about AlphaGo being deterministic. You think about the other end, the real world being nondeterministic. Where does code sit? Code sits a lot closer to being deterministic. Follows a set of rules. Every time it runs, it runs in exactly the same way. And so this is what we call execution feedback. What we’re really known for is our work in reinforcement learning from code execution feedback. It’s the way where we then take a model that we’ve trained from the ground up.

We put it in an environment, say it’s an environment with 130,000 real world code bases, several orders of magnitude, the largest environment in the world, and we send the model off to explore different solutions to sets of tasks and learn from when it passes the test versus when it doesn’t. There’s a lot more details behind this, but the way to think about it is if you can simulate it, you can actually build an extremely large dataset. And part of the things that we synthetically generate is not just the output code, but it’s the intermediate thinking and reasoning to get to that output code.

Because models today, you can try this yourself by going online and chatting to any model, can actually produce their thinking. They’re not very good at it yet. So what do you do when your thinking is not very good? You need feedback. In our case, deterministic feedback, code execution feedback.

We are making in our space, especially I think post the chat GPT moment, like incredible advancements in the algorithms that are making learning more efficient. Internally, I I have this thing that I say to the team, and they’re probably tired of me hearing because I say it every single day, say all the work we do on foundation models, on one hand, is improving their compute efficiency for training or running them, or on the other hand, improving data. Now, the way to think about the algorithms and the improvement of compute efficiencies, that’s table stakes.

All of us, OpenAI, Anthropic, Google, etcetera, are doing this, and we’re just constantly improving here. And it’s engineering and research combined. But the real differentiation between two models is the data. Compute matters tremendously for data. Because if you think about Poolside, and we spoke about how do we get this data, and I mentioned the word synthetic, it means that we’re generating it. It means that we’re using models to generate data, to then actually use models to evaluate it, to then run it. And so compute usually matters on the side of the generation of data.

But once we have all of this data, where we started today, we spoke about neural nets essentially being compression of data that forces and generalizes learning. Now, when we have small models, we are taking huge amounts of data, and we’re forcing this generalization of learning to happen in a very small space. And this is why we essentially see these difference in capabilities. Larger models require essentially, it’s easier for them to generalize because we’re not forcing so much data into such a small compression space. And so my personal mental model of this is that the scale of your models this has been shown over and over again.

By the way, we owe a depth of gratitude to Google, to OpenAI proving out the scaling laws, which essentially say as we provide more data and more parameters, more skill, hence more compute for these models, we get more and more capable models. Now there is a limit to that most likely. If you think about it as a as the analogy to compression, your image that you had, you know, at high resolution compressed down to small resolution, the small models are the small resolution. We have generalization, but you’re losing things.

But in the infinite extreme, an infinitely large model wouldn’t be doing any compression. So there is definitely a limit at some point to model size. But what underpins all of this, to directly answer your question, is the compute. And the compute really, really matters. Your own proprietary advantages in your applied research to get great data or to gather it matter equally as much. But if you don’t have the compute, you’re not in the race.

We are horribly inefficient at learning today. If you think about what drives efficiency of learning, it’s the algorithms and it’s the hardware itself. We’ve got probably decades, if not, you know, hundreds of years of of improvement still left there and different forms of it over time. If we look very practically, in the coming years, we are gonna see increasing advantages on the hardware. I’m gonna see increasing advantages on the algorithms. But I hope everyone takes away that this is table stakes. This is something that you have to do to be in this space and you have to be excellent at it.

It’s not what differentiates you, it’s what allows you to keep up with everyone else.

I think the biggest cognitive dissonance that people have around synthetic data is a model is generating data to then actually become smarter itself. Right? It feels like the snake eating itself. There’s something that doesn’t make sense in it. Now, the way that you need to look at that is that there’s actually another step in that loop. There’s something that determines if from all the data that the model generated in my domain and software development, I have a task in a code base, and the model generates a 100 different solutions.

If I would just feed those 100 different solutions back to the model and its training, the model won’t get smarter. That’s the snake eating itself. But if you have something that can determine an oracle of truth that can help say, this is better and this is worse, or this is correct and this is wrong, that’s when you can actually use synthetic data.

We are starting to understand that the scaling the first version of the scaling laws that came out spoke about the amount of data we provided during training and the size of the model. More data, longer training, and size of the model larger requires more compute. And so we often say, hey. The scaling laws are about applying more compute. And it’s actually more correct than we initially realized because the importance of synthetic data for models to get better is another form of using compute. But we’re using it at inference time.

We’re running these models to generate these 100 solutions, generate a thousand or a 100 or 50. I think we have a lot of room still for scaling up models. We can do this by scaling up data, and we can do this by scaling up the size of the model. Now our opinion is that there’s a lot of room to scale the number of parameters and size of models still. But there’s something that we don’t really talk about in our industry as much. We’re training extremely large models.

And by the way, we until very recently weren’t even capable of doing so because we didn’t have the compute and the capital. This is why our fundraise has been so important to us so that we can have the capital to scale up. But what everyone does is that extremely large models, we can’t run cost efficiently for our end users. You have a multi trillion parameter model that is what we often architect as an MOE, meaning that not all of those parameters activate during inference time, but are still very large.

It’s too expensive. Every request that you make to that model is not a couple of cents. And so you have to find a way to actually build models that you can actually run for customers. So And what happens in our in our in our industry, and and this is our path as well, is you train a very large model where you can clearly see that there’s more capabilities in the model. And then what we call, we distill it down to a smaller model. And because this is the thing, learning from data models are really inefficient, but learning from data in combination with learning from a smarter, larger model is actually quite efficient.

We make really big things that become really smart. We then teach the smaller models to try to match as much of that intelligence as possible, which we can then economically viable put in the market and make revenue from.

We should separate the price and the cost of models. Okay. If you look at what’s happening in the world of general purpose LLMs, at LLMs for everything, it’s an incredibly competitive price war that’s happening. And it’s happening between the large hyperscalers, and it’s happening between kind of referred as the escape velocity AI companies, Anthropic and OpenAI. And then you throw in the mix the the vendors that are are putting up the open source models from Meta and such. And I often think about, you know, what sits in that stack of costs?

Well, what sits in the stack at a cost is a server or a box, the networking around it, a data center, the chips, right, the GPUs, and then the energy that goes into that. And everything after that is marginal cost or variable cost of the running of the models. So we have to think about who has the lowest cost profile in the space, right? Who has the cheapest first principles CapEx that they’re doing to run these models? Well, that’s the people who have as much of that vertically integrated and who have as much of that infrastructure already online and brought into the world.

And this really is the hyperscalers. This is Amazon in number one, Microsoft in number two, Google in number three. But there’s something interesting about all of those. Each of those at different moments in time understood that they couldn’t be reliant on hardware built by Nvidia or AMD by someone else, that they also built their own. First along of that has been Google with their TPUs. I think it’s currently their fifth generation. They started early on this, and they’ve been improving it ever since. Then you have Amazon who’s been working on their training and and inferential chips, their neuron course for some time.

And Amazon has an incredible background, by the way, in manufacturing chips. And this is something I don’t think people take enough credit to because while Google has to work with Broadcom to be able to bring TPUs into the world, Amazon is working with the fabs directly. And they have an incredible skill. Right? They’ve done an amazing job on in the cloud. And Microsoft’s still earlier in their own chip journey. Now I know this is again a preamble to your question, and I’m sorry for for bringing that this there all the time.

But I think it’s an important thing to understand because when you are buying Nvidia hardware and you’re putting it in a data center and you’re working with an Oracle or you’re working with whoever it is in the space or even a Google or Amazon or Microsoft, you are taking that margin of that chip, right, the h 100, the h 200, the Blackwell generation coming up, and that has to be baked into your costs. The way that I think about this is that at the extreme end of it, Amazon and Google and and Microsoft as well as they come out with their own silicon, have a lot more margin to play with.

And now then it comes down to business decisions. And right now, since this is a war, and it’s a total you know, like, it’s it’s an incredible race that’s happening. Often refer to it as a drunken bar fight that’s happening in our industry, is that we are you know, all these companies are massively incentivized to drop the cost of their models as quickly as possible. And they do that in two ways. They do that by cutting more and more of their margin down to their actual cost, right, their hardware.

And then you can see a big difference between, you know, what an Amazon is able to do and a Google and a Microsoft and an OpenAI and Anthropic. But they also do it on the intelligence layer. We spoke earlier about large capable models that distill down into smaller models. If you have the most intelligent largest model, you can distill it down into a smaller model, and you can have advantages at that layer as well. My view is, though, that in the extreme of it, the compute margin, like the hardware margin, really matters as this gets lower and lower in price.

And this is what we’ve seen in cloud computing as well.

This is my personal view of how the world plays out. It’s really easy to to stay focused on the tactics and and and things that matter in this moment. And they’re exactly the right questions of what matter right now in the moment. But we are moving towards a place where we are closing this gap between human intelligence and machine intelligence. And I think it’s gonna be an incredible amount of problems and challenges and places where we wanna apply this intelligence. If I have a view on on modern history, and and my view on modern history is that if you look at what happened from the printing press onwards, is that what we’ve done is we’ve connected more and more people around intelligence.

We went from, you know, the telephone to personal computer to the Internet to the mobile phone. Fundamentally, we’ve been able to do is we’ve been able to take hard challenges in the world. That’s cancer research, or if that’s even, you know, building a business, a SaaS company, anything. We’ve And been able to connect more and more people together to direct resources to those things. What we’re fundamentally doing is we’re bundling intelligence. Right? More and more people got connected together. And I think that’s the true underlying thing that has underpinned this technological exponential curve we’re on.

Right? If you think back about a hundred years ago or fifty years ago, you can truly see it’s an exponential. I don’t think we wanna live in any other moment in time. And the reason I mentioned this to your question is that I think we are now gonna go from a world where human intelligence and the amount of humans we had was the entire bottleneck, to now we are having machine intelligence. And so we can pair investments in energy and chips and compute together with humans and have explosion on this exponential of all of the places in the world where we wanna direct it to.

My point is that I think there is a huge amount of places where this is gonna be valuable. We will figure out the compute efficiency along the way. The hardware will get more efficient because that’s capitalism. As the opportunity is big, we’ll direct things to make it more efficient.

about this is there’s things in the world that today we consider economically valuable, and and that ranges from scientific progress to very mundane things. Office buildings full of things that we look at today and say, why can’t that be automated? So if we take what’s economically valuable, the next thing we need to ask ourselves, what’s the gap between models today and human level capabilities, and how large is that gap? And in some cases, the gap is actually not that large anymore. And we were talking earlier about speech recognition.

Models today are pretty much there. Maybe there’s a tiny bit left to say, but we’ve closed that gap, you know, to an incredible amount. In other areas, the gap felt like it was gonna be impossible to close, but we’re making a lot of progress. Come back to full self driving. If you’ve been in your latest, you know, Tesla FSD update, that gap feels getting closer and closer to be closed. Now there’s other areas where the gap is really large still. I think software development, our domain, we think the gap is still very large, right?

What models are able to do is they’re massively useful assistants and they drive massive economic value because of that. But between a model working with a developer today, there’s a huge, huge gap, and we wanna get to a world where developers can work with models that are as capable as them, and potentially even one day more capable. Now, the reason I mentioned this is so we’ve got the human capability aspect, Right? What’s the gap that’s there? How economically valuable is the domain? Then there’s a next area, I think, that you have to ask yourself is, how easy is it gonna be to close that gap?

And that comes down to data. Where can we get extremely large scale, web scale data to be able to close that gap in areas where the intelligence gap is really big? Because the bigger the gap in intelligence today, the more data that we need to close it. If you kind of use this as a where can we find the data size related to how large the gap is between human and machine intelligence, and how economically valuable it is in the real world already, I think the intersection between those is the places where companies like us get to exist.

GitHub today has this incredible dataset, almost all of the code in the world. GitLab is a player, but only in the private side, right, what sits behind the accounts of developers. GitHub is massive in public code and it’s massive in private code. But private code, no one’s allowed to train on. Not us, not OpenAI. So all of us have access to the same public data, and it’s the output data. And so there isn’t an inherent advantage from a capabilities race perspective. Another thing that we frame in our company over and over again is there’s a capabilities race in the world.

And to your point earlier, we said there’s four things that matter. Agree with you on the three, but I’m gonna add one. Compute. It’s data. It’s proprietary applied research. It’s the algorithms. And talent. Talent is absolutely key in this industry. Now in the go to market race, it’s talent first and foremost, but it’s also product and distribution. And distribution, Microsoft definitely has an incredible positioning in the world.

No. $600,000,000 that that we’ve raised till date and the latest $500,000,000 round translates to us being able to be an entrant into the race. And what that means is that the 10,000 GPUs that we’ve now brought online this summer, you know, that came from this capital, allow us to make incredible advancements in model capabilities because of our ability to take reinforcement learning from code execution feedback and generate extremely large amounts of data and then train very large models with it. It is enough for this moment in time, but over time, it won’t be enough.

need? It’s a very good question. There are real physical, real world constraints behind this. We’ve seen crazy numbers thrown out in our industry of, you know, compute cluster sizes and things like that, but the world actually still needs time to catch up with the real ability to do so. Today, interconnecting more than 32,000 GPUs is extremely challenging. We’re starting to be able to possibly interconnect 100,000. But right now, a million GPU cluster, a 10,000,000 GPU cluster for training of models has both true algorithmic things that we have to overcome to be able to do this, and also has actual, like, physical limitations still in the world.

So we’re not living in a world right now where, like, unlimited money can buy you unlimited advantages. It’s why we get to exist, right, with 10,000 GPUs.

I think, again, it depends on how much cash and how much compute. We about a year and a half ago when we started as a company, there was a true imbalance between supply and demand in the world that even as a frontier AI company starting this, everyone wants you to win. Nvidia is incentivized to hyperscale. Everyone is incentivized actually to make early stage companies succeed with compute. It’s a lot easier when you’re an early stage AI company to get compute than it is when you’re an enterprise, because they understand this is where the future is heading towards.

But even then, there was a real mismatch between demand and supply, and we had to do an incredible amount of work of understanding the market, building relationships, and then having plan A to Z to get there. In the last six months, the world has still a huge supply shortage, and we can see this. If you’re an early stage startup, there’s lots of paths for you. If you’re a frontier AI company, you need to make decisions about who do you partner with, who do you work with, how much do you do yourself.

You need to I’m making decisions today that will impact us on compute in twelve or eighteen months from now. It’s very rare to be at early stage companies where you have to make decisions right now that impact you, you know, on physical infrastructure a year and a year and a half later. Have we seen that demand supply imbalance change? The world has still far more demand for GPU and and and GPU like compute than, that supply that’s available.

If you want to become a hyperscaler that is able to put data centers all over the world with GPUs in it that is that are are going to allow you to serve these models to everyone, an infrastructure player, that’s probably it. And that’s probably just a starting point, right? If we look at the massive CapEx investments that all of the cloud companies are doing, they’re far above $100,000,000,000 when you look at them over the course of, you know, a couple of years.

Now, in the race towards more and more capable AI, closing that gap between human intelligence and machine intelligence, I think we are all pushing the frontier more and more possible, and we’re seeing how that gap closes as we’re scaling up our models and scaling up our data. I don’t think anyone has a definite answer of how many dollars is it gonna take from here to there. If we knew that, we knew the outcomes. We’re all on the frontier of what’s possible right now.

I think we need to separate spend from getting the world’s most possible capable closing this gap, getting to AGI, closing the gap between human intelligence and Are they the same

They’re not the same thing. Because if you look at these models as investments that we’re making to get intelligence out on the other end, that needs to be economically valuable to end users, right, with lots of layers and applications and things in between. The model the creation of models is CapEx. The operating of them, the inference to running of them is OpEx. But the OpEx to run them requires extremely large scale physical footprint in the world. Very simple, if we would spend, you know, a $100 making a model and it will ever return 2 or $3 in terms of value to the world, it makes no sense.

Right? The world will punish it. It won’t exist. The huge scale out that has to happen in the world for AI to become something that can tackle all of our world’s problems and can and can feed in all of from from our software to our, you know, to our daily lives requires huge footprint footprint to run these models. It requires massive inference, and that means it requires data centers all over the world, close to end users, latency matters, and that requires a massive build out. And I think this is the one of the largest build outs that we’ve seen in physical infrastructure since, you know, the the last couple of decades in in the cloud.

We’re in a world today where the amount of data centers that can hold and power and have enough energy to power increasingly magnitude order larges of clusters is a very small number. I think he’s absolutely right in this sense. Now, you know, the data centers from, you know, two years ago versus the data centers in terms of size and power requirement that we’re gonna see in the next two years look radically different, not just because the scale of number of servers and nodes that we’re interconnecting.

This is the difference between inference. Right? For inference, we don’t need all of the machines to be connected to each other in the same place. For training, we need them all to be connected to each other in the same room, in the same place. And so that massively changes what a data center looks like.

I think it’s a good question. When we’re scaling up the size of these models and we’re training them on more and more data, and we’re using more and more compute for it, at every single step that we’re taking in the learning, every, you know, set of samples of data that we show the model, we need them to communicate with each other and share what they’ve learned across the optimization landscape.

And so this means that if I would, you know, have two data centers that sit far away from each other, the amount of information that they have to share with each other, all of the different servers, and we’re talking here thousands and tens of thousands soon of servers, you know, would make it so slow that it wouldn’t be economically viable to train these models. Once I’m running a model, I’m using a lot less servers to run it. So think of it as having lots and lots of copies of the model during training over lots and lots of machines, that every time they see data to learn, they need to communicate with each other to continue to improve in their learning.

The dynamic that exists in the world today, we all owe a depth of gratitude to NVIDIA. When I started in this space in 2016, you know, we were stacking ten eighty Ti chips in racks of servers at our office. And Nvidia already back then understood that AI was going to be world changing. And no other company, the exception of maybe Google, had that deep of a realization. But Nvidia had massive conviction on this and continued to double down here, and has made more and more incredible, you know, hardware.

The company that fast followed on that was Google. That’s why we’re on the fifth generation of TPUs. And the company that fast followed on that was Amazon. The reason I mentioned those three specifically is that they are all building extremely large volume of chips and constantly iterating on faster and better and better generations of chips for training and for inference. They’re, for me, the three primary players in the race, just from sheer volume of what they’re producing in the fabs versus what they’re bringing online to end users.

Now we have some other companies in this space. AMD, right, competitor to Nvidia, doesn’t have their own cloud, right, is reliant on being in this competitive nature from a price perspective with Nvidia. And so their ramp up is entirely determined by the demand of the world wanting to use their chips. The demand of the world for AI, for Google and Amazon with their own silicon, is not the demand for chips, it’s the demand for AI.

The way I look at this is that we’re gonna be in a world where those three, and possibly new entrants or possibly, you know, AMD may be catching up, but I think really those three, and Microsoft with their own silicon one day, are going to be the driving force in this industry.

delayed.

Because I’m training on h two hundreds. And so the compute that I brought online in in at the end of August, these 10,000 h two hundreds, mean that the longer the next generation chips is delayed, it helps me in a competitive nature in the world. But also, there’s a lot of marketing around the next generation of chips. And, again, we have to separate training and inference. Pretty much what we’ve seen consistently with every two year generation of training from Nvidia is about a two x performance increase.

But training is about two x every two years. On inference, though, I think there’s a lot of hope on Blackwell because it looks like for inference, Blackwell might potentially unlock a much, much larger gain.

The way we think about this, and I think the way to think about it, is that these chips, when they become two times more efficient, the operations we’re doing on them is still the same. It’s matrix multiplications and additions and such. It’s it’s it’s it’s math that we’re doing on these chips. The Blackwell generation for us from a training perspective doesn’t unlock anything new. It just means that we have to we can do more with a certain set of chips. My h two hundreds become less valuable in the world, but it does not necessarily mean I have to go upgrade to the next generation.

GPT-five, whether it won’t deliver, isn’t a question we’re gonna look back on in a decade from now. In a decade from now, we’re gonna look back to this moment, and it’s similar, I think, how we look back to the early days of the computer, the early days of the Internet, the early days of Google and others, and realize that we didn’t fully internalize yet how much the world is gonna unlock in value and abundance. We wrote this blog post when the fundraising announcement came out. We said, look, probably in this century, there’s three mountains that humanity’s gonna climb.

AGI is one of the mountains, the other is energy, and the other is space. And so I think as we’re gonna keep progressing, we’re gonna keep looking at the next mountain. And we from the top of that mountain, we look back, and we’re gonna realize the ones before were exponentially smaller.

If we come back to the ingredients of the capabilities race, compute, talent, data, proprietary applied research, what we are gonna find is that for compute, dollars have a direct one on one effect. But when we look at data, when we look at proprietary applied research, and we look at talent, it is not as straightforward as dollars in, magic, you know, success out on the other end of it. I think we’ve had lots of examples in technology history already where we have seen these giants that seemed unbeatable, IBM in the early days of the personal computer.

And so if we live in a world where we could perfectly translate dollars to successful outcomes, whoever can put more there is going to win. In the race towards AGI, dollars are critical critical for compute. And remember, there’s still time constraints. There’s real world physical constraints of how large we can make these compute clusters for training. And it’s that time and physical constraint, right, the constraint of what the chip is able to do, what the networking is able to pass through, that allow companies like us to have time and to do things and build massive advantages on the data, on the talent, and on the proprietary applied research.

I think you’re fair to say that a lot of knowledge moves around.

If you look at if look at our capital raise, our last $500,000,000 round, you’ll see that there’s none of the the big hyperscalers, Google, Microsoft, you know, Amazon were part of the round. Was that deliberate? That was deliberate from us. Because I think right now, there’s a future we see ahead of us for the world. And right now we see a path towards that future that we can do as a standalone company. And we have to acknowledge the fact that, you know, we’re all in the same race.

And so to me, you can make strategic decisions along the way where you decide to say, hey, we’re partnering up together in in one way or another, like in an like you’re referring to equity relationships. It wasn’t something that we had to do at this point, and it’s something that, frankly, we very consciously decided not to do right now. There is one corporate that became part of our round, and that was very deliberate, was Nvidia. And it’s because we collaborate really closely with them. I think that the nature of of large technology companies choosing to invest, you know, in frontier AI companies is frankly the game theory optimal thing for them to do.

think there’s very few left to be acquired, to be very honest. Who is

One. Rica. Rica, r e k a. Small but very capable team from what I can see from the outside.

I believe they’re in Europe, Mistral. And I’m hard pressed to think of more companies. I mean, xAI is very unlikely to be acquired, but they’re a very capable player in this space. And look, when we and and I focus, of course, on on work in in large language models and and work towards AGI. So I think there’s a there’s very few companies that today are sufficiently far along and that are still left.

I would love to spend a day with the current leadership team of every single one of these and then make a decision. They each have inherent advantages to them. XAI has understood that compute mattered and built an incredible team there and did what no one really had done at that speed. They built a 100,000 GPU, three thirty two ks interconnected clusters in Tennessee. In the span of of months, xAI showed up with Elon’s strength, the ability to build physical infrastructure incredibly fast in the world. OpenAI had the incredible ChatGPT moment and has built this incredible business around both ChatGPT and the usage of their APIs, and is clearly ahead in revenue as others, like has publicly stated.

And then I think Anthropic has incredible, thoughtful researchers and a very rigorous approach in what they do, a very rigorous scientific approach in terms of moving things forward. And so while I can see strengths in all three of those, it would really be a day with the leadership team of each to determine where I would put my own money.

Look. I think Sam and OpenAI have understood the importance of compute and have understood the importance of data. What I imagine that $6,600,000,000 is is going towards is exactly those two things, where from the outside, I think it is tricky to be Sam today. Think I it’s tricky to be Sam today because general purpose models that aim to be everything for everyone is an incredibly competitive market, and you find yourself with incredible pressures from all sides. And you’re building a platform and a consumer product in exactly the same time.

And more so than that, you’re building a consumer product that from the outside is seeming to be for everyone. And I think that’s a really hard thing to do.

It’s one I think about a lot, to be very honest. And you’re getting me even with it. It’s probably one of only things you could have said that would have got me bit emotional because I think about it a lot. I saw it many years ago, and I think I understand what he means very well. It was earlier today with a with a founder I I really respect. We’re And funny enough talking a little bit about this, is that building what we’re building, it’s not a choice, it’s an obsession.

And you bring everything you’ve got to it. And we were talking about, hey, how do you deal with waking up at three in the morning and your mind doesn’t stop racing? Going back and forth and then sharing each of our techniques and probably going back home tonight and trying them. I think Elon is one of the most impressive examples of someone who has done this for such a prolonged amount of time at moments in time when the entire world, you know, refused to align around his view.

I think there’s there’s companies in the world that get built because they were at the right moment at the right time, and there’s companies in the world that get built that shouldn’t have the right to exist. Everything is against them. And I think Elon has done that not once, he’s done it multiple times. And and it brings me to another quote that I heard on the interview from him or about him was Peter Thiel. And Peter Thiel said, when we all worked with Elon, we thought he was crazy.

He would take so much risk. And then he went and started, you know, Tesla and SpaceX, and we thought he was even crazier. And if one of those two companies would have worked out, we would have said he’d gotten lucky. But both of those companies worked out and beyond and to extremes. But there’s something Elon understands about risk that the rest of us don’t. For And me, that’s the second quote that’s been on my mind most of this year.

When I was in high school in 2008, my very first startup was a virtual digital currency. My views on crypto have changed a lot over the years. The notion of decentralization and and what it can mean for the world is incredible are incredible ideals. The problem I think that we’ve seen in crypto is a a quote that I learned. It might have been my high school economics professor. And he said, bad money drives out good money. If you think about this in environments, when bad actors, you know, come in, it drives out the good actors.

Because we wanna be in environments with other good actors. And I think the promise of crypto started from great actors and found itself due to the incentives of the ability to make money very quickly in lots of distorted ways to bring a lot of bad actors to it. The bad actors have driven out a lot of the good actors over time. There are still a true amazing idealist in that space. Now the thing in AI is that we don’t have that.

We have a set of people around the world who all fundamentally might disagree on how to get there, but all see that in ten, fifteen years, we’re gonna look back and realize we had this incredible shift in the world by being able to close the gap between machine and human intelligence. And so while that might drive today some centralization because of the sheer amount of resources required that are scarce, Right? Capital is the least scarce part of this. Right? The talent is scarce. The proprietary, you know, applied points of view and research, those are scarce.

I think that does lead itself to a small number of companies. I think we’ve seen that over and over in history. We look at when the the massive boom of automobiles, right, the hundreds of automobile companies that started and how many actually survived. We’ve had this over and over again. And so I don’t think this is something new. What I would like to see is that it’s not just Google, Amazon, and Microsoft. It’s going to be an OpenAI and Anthropic, Poolside, and a set of companies who are able to get that massive escape velocity needed to sit alongside those companies and build the next, you know, generational businesses.

So I think it depends on the use case. There’s lots of experimental use cases still, and there’s use cases that are far past the experimental side. AI for software developers. I don’t think anyone in the world anymore questions that software development moving forward is gonna be a, for the foreseeable future, a developer led AI assisted world and increasingly AI assisted world.

I think there are use cases that are commoditizing very quickly. Speech recognition Mhmm. I think is one of them. Image generation is one already where we’re seeing more and more commoditization over time.

I wanna set the record straight. We’re an American company, and we’ve got incredible people from all the way from San Francisco to Israel. But a decision that we made early on, Jason and I, my co founder and I, we were planning on building this company in the Bay Area. And we did the work in the first days of the company, and the work was let’s make a list of everyone we think from both that we knew and also externally, like on research papers and in GitHub repos, that we think potentially could be great for us.

And the list ended up with about 3,300 people. A lot of work done. 3,300 people that we saw ranged from having experience on distributed training to GPU optimizations to work on data to reinforcement learning, experience with large language models. So, you know, the whole breadth of what it takes to build what we’re building from a model perspective. In that list was a location column. And as you can expect, the number one represented geo in that was the Bay Area, not even The United States, just purely the Bay Area.

But what really was striking for us is that there was a huge part of that list that was not in the Bay Area. It was spread out across Europe and Israel, from The UK to Switzerland to Tel Aviv to Amsterdam, you know, Paris, like all of these different places. While we couldn’t see a clear deep talent concentration in one place, probably UK actually being the one with the largest talent concentration, we didn’t realize that it was probably worth spending some time talking to people there. And so I went and had conversations, and we realized one thing.

We said there’s incredibly capable people here who wanna stay here geographically. They don’t wanna move to the Bay Area to join some of the other companies in the space, but they’re not finding massively ambitious young companies that have huge visions to join. And so we saw that as an advantage, right, to the four things in that capabilities race. We need to build unfair advantages for every single one of those. And so we said, great. Let’s build up talent here on this continent in Europe, as is we will in The United States.

And frankly, I’m very grateful that we did.

So

us is about 15 people.

The way to think about it is where has talent historically been even pre Chatuchupteen moment and talent in AI? And who helped build that talent in this space? The number one company we have to give credit to is DeepMind. DeepMind built an incredible talent base. They built it out of London. Meta did some work in building a very incredible talent base, and it did it between London and Paris. But in terms of when you look at it from a numbers perspective and sheer size of people, Google separately and DeepMind as part of Google, had made much larger investments.

And then there’s another talent pool that we do often talk about publicly that is just absolutely extraordinary, which is Yandex. Yandex built an incredible company in Russia with some of the world’s most capable researchers and engineers, many of which have since left Russia and have kind of become a diaspora all over Europe.

There was a tweet, and if I recall correctly, it’s from Aaron Levi from Box early on in in post chatty bitty moment, and he wrote something along the lines of, if you feel like you’re working extremely hard on reasonable hours as AI is now booming, you’re probably right to do so. Because it’s in these first years, and I’m probably going beyond what the tweet said, but it’s in these first years, it’s where the table gets set. Who has earned the right to be in the race to AGI?

The way that I’ve I’ve always looked at this is, from a personal perspective, and so has my co founder, and so has Margarita, has put us in a place where we’re gonna look back on this moment ten years from now, just like we would look back to the moment of mobile, Internet, and realize that that was the moment where the table got set. And you do not wanna look back on that moment and not have given it everything you’ve got because it’s a race. And look, most startups are not racist.

Most startups are against yourself. But AGI is a race, and so our view always has been the team that we build is a team that is deeply passionate to be in that race. And frankly, when you decide to join a race and you’re upfront about it, you decide to try to become the gold medalist in swimming, that means that there are sacrifices that come with that. You don’t get to have it all. And so that’s something that we’ve been super open about with with people from day zero.

It’s on our first intro call we talk about. Like, do you wanna join a race? And frankly, I have found no shortage of people in Europe that wanna do that. I think there’s a there’s a stereotype about Europe. But the fact of the matter is that people who wanna join races and do truly their life’s work, they’re built differently, and you can find them all over the world in every single country. You just gotta do the work to find them.

You know, history doesn’t repeat itself. It rhymes. I think it was Mark Twain. I think that might be the mistake that we’re possibly making looking at the past. And the reason that is is because we’re on an exponential in terms of technological progress. I think in 1996 with Netscape, if I’m getting the year right, there wasn’t this amount of people and capital that understood what the future might look like in the next ten years. And it took some time to get there. Now I could be wrong about this.

Another thing that I could could see as a possible avenue of why I tend to disagree is there’s a big difference between what was required to be built in 1996 versus what’s required to be built today. Bring it all the way back and try to steelman the opposite side of the argument is maybe it’s exactly that. And that the next couple of years are about these massive capabilities that were moving the world closer towards AGI. And then when you look at the following five or ten years, it’s true.

The huge economic value that’s gonna come from that will huge will, of course, surpass the economic value that we have. I think the economic value is gonna continue to surpass on the exponential that we’re on. But what I don’t agree with, the companies that are being built today will not have a set of companies amongst them that will become the giants of the future that have helped enable this.

So if if we take the battery analogy, I would actually think a little bit about BYD. Started as a battery company. There’s the largest volume of electric cars sold in the world. And I do think there is a lot to be said about deep vertical integration. Look at Poolside. We’re building foundation models with a mission towards AGI right now in the world focused on bringing more and more capabilities of AI to software development, building a truly end to end business. Because I agree with you that the value is not going to only accumulate at the model layer.

It’s gonna accumulate all the way to the end user. And so in our point of view, the way that we kind of, I think, get to avoid what the future plays out in in your hypothetical scenario is just by truly doing it end to end. But I still actually look at this thinking that there will be more value built on top of us in the future than what we can possibly unlock only ourselves.

They’re not. There’s a couple of interesting things that might not be as obvious unless you’re in our industry is. The research that still gets published openly, that doesn’t get held back, that is most interesting, is all coming out of China in vast spades of majority, something that wouldn’t necessarily be obvious. But if you think about the game theory optimal thing to do, because they’re not on the forefront of the world scene of AI, actually opening up some of that research is the game theory optional, know, optimal thing to do to be able to continue to attract talent.

Because that’s really what opening up your research does. Right? It attracts talent to you. I think China is at an incredible level of capabilities and in no way could be discarded or thought of as years behind, on AI or AGI progress. We’re working on technologies that we can see a massive societal impact. I think it’s really important to be good stewards of that technology and that progress. I think part of that is acknowledging that what we know about is the technology. What we know about is our users, our customers.

But we should be careful in terms of trying to know what’s best for the world and how to think about massive geopolitical conflicts and things like that. And so what I have said in the past is that the best thing that we can do as the West is to keep making it as attractive as possible where talent from China, we consider as a competitor, right, on a on a very large scale to come to our countries. The easier we make it for one of those four major ingredients in the capability race, frankly, of the most important ones to, you know, help us accelerate, I think is probably the the most practical advice I can give.

I think in the last twelve months, it’s a continued realization of the importance of scale of data and not only compute.

selling source to GitHub? It was probably the dumbest financial decision of my life considering it was an all stock offer, GitHub sold to Microsoft, I think, less than a year later. And it would have three x the price? Far higher than that. But I’m really grateful I didn’t.

I’m sitting here.

I think the question is, what could I have been able to do continuing on source mission? Because source mission was the mission we’re talking about today with Poolside. And back in 2016, there were very few people who believed it was ever possible for AI to write code. But no, don’t think there’s really no regrets there. I wouldn’t be sitting where I am today, and I don’t think I would have become the person that allows me to go build Poolside today. And frankly, I’m really grateful that that event did happen because that’s how I met my co founder.

That’s how I met Jason. He was the CTO at GitHub at the time, and it started this many year conversation on what the progress in AI looks like and and its applicability to software development.

that progress is going to halt.

would cause progress to halt? Global conflict that disrupts the supply chain of chips. That is fucking

Mark Zuckerberg.

I think we all should give a lot of credit to Mark Zuckerberg in terms of having built an incredible company with a lot of conviction on what the future would look like when most people didn’t agree with him. If look you at what he’s done on AR and VR in the last decade, right, from buying Oculus to where it is today, when most of the world just wanted him to stop, that required an incredible ability to have conviction for a future that’s gonna massively change the world.

To get to AGI requires an incredible conviction for a technology to change the world. He’s one of the people who has who has done that and and someone very different than the people I have currently on our board.

Regulation that fundamentally halts progress for small companies. The reality of regulation in many cases is that it becomes an expensive bureaucratic overhead, and that harms the most young startups. It doesn’t harm companies that have raised massive amounts of capital.

The world is finding a balance, and the balance that I’d like to see that the world is and I think it’s moving towards it, is to regulate the end user application of AI, just like we’ve regulated the end user applications of any type of technology before. It’s not the database that does harm, you know, how it’s used. And so for me, I would love us for us to to continue to hold companies massively accountable for the end use of their technology to to users, to to consumers, less so trying to put limitations on, you know, how much compute you’re allowed to train on.

We’re building tools that are closing this gap between human capabilities and machine intelligence. We are not building the Terminator. So what do you think of DST’s Yuri Milner? I really like Yuri. I I got to know him through several occasions over the last year, but I never got to understand him till I read his book. Most people don’t realize that Yuri has a book or a manifesto. You can find it online for free. And earlier you heard me reference looking from one top of the mountain to the next.

And that’s actually, it’s a metaphor that I took from Yuri. And because I think what Yuri has stated and said is that the sheer importance of scientific progress, what he often refers to as humanity’s story, right? We have this incredibly special thing on this tiny little blue marble in the universe that is so immensely large. And I think he really embodied the notion of we cannot let that flame die out. To him and to Elon’s credit, sharing the fact of the world is that one of the ways of doing that is to become a spacefaring civilization.

His whole view is that progress that’s happening towards AGI helps us to be able to take this special thing that we have, you know, humanity, and spread it across the universe. And there is people who say this, and there’s people who believe it and who understand how special this is and what it can mean. And Yuri is one of those people. But at the same time, he’s been an incredible capitalist. He’s been an investor that, from what I can see from the outside, that understood that huge technology waves were happening all over the world in different places, and it was a truly global investor.

What he saw was happening in The US, he would invest in India, he would invest in Indonesia, and across Asia, and all of these different places. And I think very few people from the investment landscape have taken strong conviction on what technology is gonna bring in the next ten years and then found a way to place bets all over the globe. Have people tried to buy

No

So I think we have to put some context here. Several years ago, me and my better half and our golden retriever, we were living in a fancy apartment. We decided to get an old sailing boat to fix up together. We put this old sailing boat in the harbor. Ceiling was falling apart. Something we did together and we were passionate about, and we loved being on the water, loved being on open ocean in particular. At some point, we were spending more evenings and nights on the boat where the ceiling was falling apart than the fancy apartment.

And we looked at each other and we said, why do we still have the fancy apartment? Now we got a slightly nicer boat after that, and home became a sailboat. And I’ve gotten this question from friends. Never I don’t even know how you know this, by the way. I’ve gotten a question From friends. From friends. For me, I think the same thing about living on a boat is exactly the same thing of wanting to move the world, you know, forward. And and and I think a part of it is the drive of of freedom.

Right? The freedom of options, the freedom of, like, what can what can adventure in the future hold? And then the simplicity of not of just being with, you know, the person you love and your dog in a small space and you’ve got your good internet connection and you do your work. You know, all the other stuff doesn’t matter. The watches, the cars, the houses, none of this stuff is ever gonna be the things you think back on. It’s the journey. And for me, a boat kind of allows for all of that.

It keeps life simple, and it gets me allows me to, you know, fully immerse myself in in Poolside. And and I have to admit, since I started this company, I’ve been home less than a month. And so it’s been it’s been a lot of travel, but it’s it’s the journey that matters. It’s not the material stuff.

I did. I was lucky. I went through it in my early twenties and a lot of stuff. And then I got rid of all of it. For me, the sheer realization is that it’s the it’s the journey with the people. The outcomes, you know, that we wanna see in the world, they’re part of the obsession. They’re the if you can see a future that looks like one that ends up being this incredible future, you wanna build it. But that moment once you reach that, which will always be an ever moving goal, is not the interesting moment.

The interesting moment is every single day with the people. And I think for me, I just learned more and more over the years is that I love people. And I love the people I work with, my team. I I think in every single person, there’s something incredible. And and if you cut all the stuff, you cut all the money, and and you pick the hardest, biggest thing you could possibly, you know, focus your life on and then do it with amazing people, you get to have this incredible experience.

We’re in a race. If we stumble, we we put our foot off the gas pedal, we lose. And I think in any race, there’s a 101 places you can stumble. We don’t get the luxury of stumbling on the capabilities race or to go to market race. We have to be excellent in both. If we make a misstep on either of those, we can fall behind. And if we fall too much behind, we’re no longer in the race, and we don’t matter.

I’m surprised how I usually motivates you? And people ask about the business. They ask about the outcomes. They ask about the future. See, very, very few people ask you about your why. And I think that’s probably the most important question. When we talked about earlier about looking at those three companies, my first question to all of them, you know, and I have asked this to some of them, is, like, is is is the why. When you

Why do you do what you do? I think that says a lot about a person, and I think and and at the end of the day, in any race, in any ambitious endeavor, it’s the people that make it happen. It’s not the dollars in the account. Those are the resources and inputs that you need, and it’s the people. And so for me, the why is a really important question to ask people.

I realize that I’m not calm or at peace when I’m not working on the hardest possible problem that lends to what I care about in the world.

Because my brain’s never gonna turn off. I’m always gonna wake up at four in the morning. It’s always going to be this constant going over and over of, like, what you care about. And when I’ve done that in the past were things that weren’t the hardest possible I mean, you say the most ambitious possible thing in terms of like what mattered in the world, the thing that mattered the most, I just didn’t feel at ease. And now probably never worked. I’ve always worked hard, but think I’ve ever worked as hard as I have at Poolside.

While it’s intense and it’s stressful, I I feel at peace.

My word. I love doing that show. If you wanna watch the full interview, you can find it on YouTube by searching for 20 VC where you can see ISO in the studio. It was a fantastic one. But before we leave you today,