Why Foundation Model Performance is Not Diminishing But Models Are Commoditising

Yeah. Thanks, Harry, for having me on it. I’ve got to watch some of your cool previous episodes, so it’s a it’s a real honor to be on here.

Yeah. I mean, Google Brain was and also now as part of DeepMind is a really magical place. I think during the peak days of AI progress on the research side, right, where every day there was a new paper that came out that would just change the world, that, like, 2012 to 2018 or so era, Google Brain was just, like, incredibly dominant. They did an amazing job picking talent. Like, the people who invented transformer, people who invented the diffusion model, people who did all of these new optimization techniques that we all take for granted today, they were all a brain at the same time, like, truly the Bell Labs of the era.

I learned a lot about how to make pure bottom up like, see what good pure bottom up basic research looks like at Google Brain.

So I have this worldview of AI progress as being a part of a couple different phases. Right? And I I like to think about pre 2012 as basically being prehistory. Of course, all of the, OGs in the field were probably not like that if I characterized it that way. But before 2012, like, most of the things we tried just didn’t really work. Right? Like, you you had things like, a sheep being identified as cats and dogs and chatbots that barely said anything coherent, etcetera. But I think, like, there was a period between 2012, like 2017 or 2018, where deep learning went from something that people didn’t believe in to being, like, the dominant paradigm in the field.

And so during that 2012, 2018 era, the way people made progress, what I mean by bottom up basic research is you hire the most brilliant scientists. They come to work every day with, like, no near term objective they’re being held accountable to. And they just work together and they think about, you know, like, I wonder what it’d be like if we could solve this, like, open technical problem in AI. Like, how do we create a model that better understands how to generate images? And they just go work on that of their own curiosity and drive and maybe some interest in glory and fame through papers.

And they do that for, like, six months or so, and then out pops out this like research paper that gets posted to archive and goes to a journal that like just solves the problem. That’s huge. Right? And so the reason I call it bottom up is because it’s just driven by the natural interactions between all all these researchers in a in a setting and they figure out what they wanna do.

Well, I think what happened was in 2017, the transformer came out. I was running engineering at OpenAI at the time, and I was working really closely with Ilia. And Ilia and I were just sitting around, and he was just like, look. This transformer thing is real. It’s gonna be the next most important thing. Let’s get all of our teams looking at how we can use this thing. The thing that most people in the general public don’t know about is we didn’t invent transformer at OpenAI. It was invented at Google.

But what transformer did, though, it was the first time you had a model that was generally applicable to any machine learning task. Back in the day, if you wanted to understand images, you used a convolutional neural network. If you wanted to generate text, you used RNNs. If you wanted to beat humans at Go, you used tree search or RL. Right? So you have these all these different models that you would use to solve problems in AI, and then transformer kinda became, like, the universal model and base element of AI that came out at that time.

And once transformer came out, in a weird way, you kinda stopped needing to make, like, super low level breakthroughs in modeling because it kinda just worked for everything. And then you get got to go take that thing to solve really, really big problems.

Yeah. No. That’s a really good question. It’s kinda like ChatGPT was like the frog that ultimately became boiled. Right? Like, transformer was a huge breakthrough. And every incremental year from 2017 to when ChatGPT came out, language models just got a little better and a little better and a little better. Like, I remember Alec Radford and a couple others, and I did GPT-two. And GPT-two came out in, I think, 2019. I just remembered, finally, you had this, like, pretty smart generalist model that you could just say, hey.

Like, write me a newspaper article about insert celebrity being arrested in LA, and it would just do a perfect job. It’d be like, oh, they were in, the Neiman Marcus store, etcetera. I thought that was so much fun. The thing is, like, two things had to happen. One, the models were getting increasingly smart, but there’s, like, a minimum viable smartness where you’re like, damn. This is a compelling experience. And the the second thing was it needed to be packaged up in a way that consumers could play with.

So if you go look at the lag, right, ChatGPT was really just GPT-three with instruction. It we’re basically, like, more chat tuning. But GPT-three API came out, I think, over a year before ChatGPT did, but only developers could play with it. So there’s no viral moment for consumers. So the packaging and the intelligence had to exist in order for that virality moment to happen with ChatGPT.

Well, the first one actually is and going back to the, like, eras of AI discussion we were having. Right? What OpenAI realized before basically everybody but DeepMind was that the next phase of AI after transformer was not going to be about research paper writing. It was going to be about let’s choose a major unsolved scientific problem and just try to solve it. And so, like, that led us to go build a culture of instead of, like, loose collections of federations of researchers, let’s put a giant team around how do you solve, like, robot hand control.

Let’s put a giant team around beating humans at, like, one of the most popular video games on the planet. Right? Let’s put a giant team around scaling GPT until this thing is, a generalist reasoning and chat engine. That’s just a totally different framework from, like, this very academic curiosity driven research, and I think that that’s the right framework. And I think that’s a big part of how we build the depth now as well.

Exactly. So it’s like the switch from, like, hiring, like, a thousand people to go sit around thinking about how to put together small rockets versus, like, creating, like, the Apollo project. It’s a lot better to say, hey. Our goal is to go to the moon, and we’re gonna hire however many people it takes to go solve going to the moon. It’s very different than just, like, a giant mass of people organically self organizing to do that.

I don’t think so. And here’s why I don’t think so. It all depends on what access you use. Right? The way that giant model scaling works, historically, if you go look all the way back to GPT two to GPT three to four, etcetera, the way it’s worked is that, let’s say, just to use a reductionist analogy, every incremental GPU you throw at the problem actually does have diminishing returns. But every doubling of GPUs you throw at the problem has very predictable, consistent returns. It’s kind of like a logarithmic curve versus a straight line, right, depending on what axis you use to go look at it.

So put another way, for just scaling up a base language model, you need to double the amount of compute for that language model for it to be predictably consistently smarter. Does that make sense?

So the better way to go think about it in my view is like, there’s two parts to model scaling with compute. One part to it is you simply make the model bigger, and then you throw more data and more GPUs at it. If we go look at CPUs and data centers, right, for a long time, we had Moore’s Law. Right? Every year, chips would get better at some predictable pace, and everybody’s, oh, Moore’s Law is gonna die. You know, we’re at three nanometers or whatever. There’s, like, no more nanometers left.

But what actually happened is you go look at the amount of compute available even for chips. It’s actually continued to trend up because now what we do is we build systems that have multiple chips in them. So we have, like, both the scale up of a single chip and the scale out. And as a result, every year, humanity has more and more compute available to it. It’s the same thing with giant model scaling. The base model itself, even if the base model itself stops scaling at some point as you throw more compute at it, there’s a whole new way to go make models smarter that is just being tapped right now.

And that whole new way of making models smarter is not just making the base model larger, but it’s by having the base model collect data for itself to learn how to get smarter. So let me give you a concrete example of this. Right? Concrete example of this is, right now, let’s say you wanna train an LLM to get better at solving math problems. The way you do it is you collect lots and lots of, like, positive solutions to hard math problems, and you throw it in the dataset.

Right? And you’re like, hey. Like, model could go get smarter at this thing. But a much better way to solve this problem is you give the model that you’re training access to a theorem proving and math environment. Right? Like, give it a Jupyter notebook, theorem proving library out there that a lot of people use as well. Give the model direct access to those tools and then say, hey. I want you to experiment. I want you to go try solving this problem and then reflect on it.

Like, did it did did you do a good job? Like, is this problem solved? If no, try again. So now what you get to do is you get to have the model play with the simulated world, basically, to collect positive and negative data how to solve math problems, and then that makes the model smarter. So the second way of improving model performance is just starting to be tapped now, and that’s also going to absorb a boatload of compute. Because of that, I actually am not worried about the diminishing returns to compute over time.

Everything’s an s curve. Right? The giant model scaling base model scaling s curve, for the last couple years we were here, we were at the sharpest point of improvement. You know, you could double the cost of your model of a $100,000,000 to $200,000,000, and that would be the fastest and easiest way to deliver a smarter thing to the world. And now if you’re getting to billion dollar training runs and $2,000,000,000 training runs and $4,000,000,000 training runs, it’s really freaking hard to go get more money to go make the base thing bigger.

And so because of that now, the critical path for model improvement is shifting over to this broader sort of simulation slash synthetic data slash, like, RL loop path. I think it’s just a natural consequence of the fact that it’s so expensive to just keep scaling.

I think a good way to think about it is, historically, for the last couple years as we scaled up LLMs, we’ve just been doing more unsupervised learning. You get more data, more smart journalists writing articles, feed it in there, and that makes it smarter. But the problem is, like, a model trained that way is only as good as the smartest data in the training set. Like, it cannot discover new knowledge because its job, the way the models are trained, is to do what a human would do in that situation.

But the underlying thing is if you wanna go solve, like, really big problems, like solve on like, prove unproven math theorems or, like, be able to, like, help you solve a creative problem at work, those problems are, by definition, things that are not in the training set because it’s either, like, a superhuman thing or it’s a novel situation.

But not in data. Yeah. I mean, I think that’s a key insight. I kinda think that chatbots, ChatGPT and stuff, and agents are kind of becoming different species of technology. Like, I think they’ll be useful in very different ways, and what they need to be used for is super different. Right? Like, just one concrete example is the hallucination problem. Having hallucinations in chatbots and in, like, image generators is, like, a really good thing. Right? Because it gives you, like, a starter tool for, like, getting to, like, solve a blank page problem.

Right? Like, gives you, like, little bits of novelty and creativity. But agents on the other hand, like, if you want something to go consistently, I don’t know, like, do your taxes for you or, handle all of your shipping containers or something like that, you do not want that thing to go randomly hallucinate and, like, make up stuff along the way. Right? And so, like, these things are speciating in an interesting way right now.

The coolest thing the reason why I love working in AI is that, like, for the first time as an engineer or researcher, it feels like you’re, like, uncovering, like, unknown secrets about how intelligence works every day. It’s, like, very different from programming. As a programmer, show up to work. I’m like, here’s the thing I wanna build. I know I can build it. I know if I am clever enough, I can solve a problem. And I know exactly the behavior of the system that I’ve built will be.

But the cool thing about AI is that every day you come to work and you make some tweaks to the model, and what you get on the other end is actually somewhat unpredictable. You kind of feel more like a gardener than an engineer. And I think what’s really cool about it is that as these AI systems have gotten bigger and as the architectures and datasets have improved, what the model is good or bad at, you can’t totally predict ahead of time. You have some estimates for things.

But just going back to the early days, right, when we were training GPT-two, we trained GPT-two at various different sizes. At the smallest size, the model was just, like, unable to do three digit arithmetic. But as the models got bigger and bigger and bigger, we didn’t change anything else. We just had to look at more data, and then we made the model bigger. And then, like, at a particular size, there’s just a moment where from, like where where where it went from not being able to do, like, three digit arithmetic to being, like, very good and predictably improving at getting three digit arithmetic better.

And that, like, moment, we couldn’t know about in advance. So that’s what I mean by, like, a minimum viable capabilities and how it’s a function of model scale. There are things that we really want these models to be able to do, like be really useful agents or to help us discover new things in science or whatever. But it’s hard today to say, hey, you know, if I just spend like $2,000,000,000 in compute on this model and have the right data, that’ll happen for sure. I think that’s what’s so cool to work in the field.

Reasoning is one of the problems in the field right now that I think a bunch of us sort of have similar ideas for how to solve, but it actually requires some new research to be done. So in a weird way, working in AI is pretty funny these days because the giant model scaling problem is so known, and it’s really a function of resources. And so you kinda don’t feel like you need to be a genius to go make new products on just pure model scaling. But I think pure model scaling does not deliver solutions to reasoning.

To me, the definition of reasoning is being able to, like, compose existing thoughts to discover some new thought. And I think to go do that, that’s not something that’s trained into the capabilities of LLMs by simply asking it to regurgitate the Internet’s worth of data. The way we’re gonna solve reasoning is back to what we were talking about earlier, taking theorem proving as an example. You wanna give the model access to a theorem proving environment and have it try things in the same way that, like, you know, a human mathematician would sit down and be like, well, you know, here’s the things I know to be true about the world.

How do I compose them such that I can prove the thing that I wanna prove?

I think the general capability of reasoning will need to be solved at the model provider level. And that’s because what you’re actually doing is you’re not just using the model to reason, you are trying to improve the model’s ability to reason, which means the model itself needs to change.

No. I actually think that, like, solving these reasoning skills are on the road map of every LLM player. I do think there will not be that many LLM players. I think there will probably be my guess is somewhere between five to seven long term steady state LLM providers at maximum scale just because of the costs involved. Reasoning is just another expensive thing that these companies have to get right, but I think they will all solve it because I think the way to solve reasoning is something that many of us in the field kind of have a pretty strong suspicion on.

Train a base model, give it access to a wide range of different environments to go solve hard problems in, and have the model try how to solve those problems, and use that and combine that with sort of human input on whether it’s doing a good or bad job, I think that will solve reason.

That’s a good question. Well, I think you can kinda think about memory as being two different things. Right? You kinda have short term working memory and then you have long term memory. I think people have made really good progress on short term working memory. Right? Like, if you go look at Gemini, Gemini’s context length is, a million it might even be more now. I just don’t quite remember. Like a like a million tokens long, which is so cool as you can feed it, like, giant snippets of video and be like, hey.

Like, write me a step by step of, like, every everything the person cooking on this on this in this particular video did, and it’ll do it. Like, that stuff is insane. That’s making good progress, and the reason that’s been hardest for computational reasons. But this sort of longer term memory problem, this goes back to, like, another thing that I believe, and that that’s why I’m slightly less excited about model building, slightly more excited about application developers. Because the underlying thing that everyone’s realizing now is that LLMs themselves are not a product.

Like, an actual product is this entire software system that uses LLMs in it. So for example, like, what we should be doing is we should be finding ways in which end application builders can be themselves responsible for how to build in long term memory about user preferences. Right? Like, I don’t know. Let’s say I’m building a company that’s working on a consumer travel assistant. Right? Like, I should just be able to tell that thing. Hey. Like, I I freaking hate this is a true story, by the way.

I hate aisle seats because once someone dropped a suitcase on my head in a flight and I got a concussion. Never book me an aisle seat again. That kind of, like, long term memory, I think application providers should be able to handle as part of a bigger system.

I think it’s a game of how much you existentially need to win. I think every tier one cloud provider existentially needs to win here. Let’s look at the dynamics involved. It’s it’s one where as these models get smarter and smarter, they kind of become the base computing primitive. Today, the base computing primitive is like nodes on EC two or, like, storage. Right? But in the future, when more and more software is just like, the logic of software is actually just handled by a by an LLM, nobody cares anymore about what the base computing primitive is.

All you need to do is access these models and compose these models to go solve things for customers. So then whoever controls the model layer controls all of the underlying compute. And so, like, right now, what’s happening, right, is, like, if you don’t have an offering here that is state of the art, then you’re just gonna be cut out of this particular game. I think this is also actually an area where I think it’s really important for companies like Nvidia to go up the stack. Right?

Like, Nvidia clearly killing it right now on on chips. But what’s happening is every one of the major clouds and every major LLM provider is working on a strategy to have their in house chips because that way they have better margins. And so then at the end of the day, if you’re like a developer or you’re like an end user talking to ChatGPTN from one of n different providers, do you care whether the back end is an Nvidia chip or an AMD chip or an in house chip from Google?

Right? You you don’t really care. And so, therefore, like, there’s, like, a really key point of the interface of the LLM gives you tremendous leverage on everything downstream.

That’s my expectation. To me, like, what’s interesting about AI from a business side, right, is, like, it forces the question of, like, what companies or offerings are gonna be bundled or or integrated and which ones are gonna get unbundled. And I actually think that there’s gonna be a really strong vertical integration pressure between model builders and chipmakers.

I I love the topic of chips. We can stay here for a while. It’s just it’s so much fun. It’s like the most interesting thing that’s happened in some time for that industry. Like so I we were just talking a minute ago about how important it is, right, for model makers to control their chips. Because that way, if it really is a scale and resources game, if company a, let’s say, choose Google with TPU TPUs are great. Right?

With TPU has a 20% cost advantage compared to company b using chip y, then Google will just be able to and just the the better cost of model training will let them go bigger, let them invest more in post training tricks like the ones we were talking about earlier and have an advantage. And so then company b is, like, gonna be really pressured to go find some way to go go do that themselves. Similarly, if you’re a chipmaker, it’s just too easy to be to be commoditized by these in house efforts if you don’t also own something at the model layer.

So I think that’s, like, vertical integration pressure

Okay. I actually think we’re saying the same thing. It is incredibly hard. Like, Nvidia is killing it. It is incredibly hard, but it is possible. And if the economic returns are high enough, people will do it. Right? So I just think Google TPU is a great example. Like, I’m a I’m a massive Nvidia fanboy. Jensen is incredible. And I think Nvidia has, like, executed so well here. I think we also have to give props, though, to, like I think the TPU team when I was at Google was, like, sub 500 people, and their budget was a shoestring budget.

And yet somehow every generation, they taped out quite good chips that were then used to train Gemini and and Palm and are used by third parties now and all of that stuff. And there is such a strong will to ensure that Google has its own first party chip. That’s the counterexample to, like, the perpetual chip dominance.

Okay. I’d like to think about AI capabilities in terms of actually, to start from the top on this one, thinking about the Apple think about, like, the Apple advantages in in this particular space. I think there’s, like, two areas of, like, extreme power and leverage that you get in machine learning right now. One is the ability to to run smart models for free at the edge, and the other one is to have the absolute smartest models possible. And so I think Apple has a massive advantage on the former.

And when we go think about, like, whether that’ll be enough, I think that’s a really hard question to think through because I kinda think about it as, like, concentric rings of model capability. Right? To give some concrete examples, like, 1,000,000,000 parameter model that is otherwise trained to be state of the art kinda has, like, this set of capabilities that it’s, like, perfect at, and then it’s, like, kinda okay at the next rung up. So, like, maybe the very minimal set of of capabilities is, like, is this tweet positive or negative.

Right? Like, you don’t need you don’t need GPT-ten to go tell you whether this tweet is positive or negative. Like, a pretty small model can do a perfect job at that. And so things like that will always run on the edge. And so then if you’re a giant frontier model provider, like, you’re just not gonna be able to monetize these, like, tiny skills that are just gonna live at the edge. But then conversely, a billion parameter model is probably for some time not going to solve, like, be able to create a three d part for me for my car.

Right? That’s probably gonna be the, like, GPT-ten problem. And so I think as a result, I think Apple is just going to completely crush at everything that looks like something that’s really private, something that’s that’s fine tuned on your own particular data, but doesn’t require massive reasoning capability. And that will all run at the edge.

I am extremely impressed with OpenAI, I think, in terms of their technical delivery. I think the degree to which GPT-four o was, like, I think, underhyped relative to what I think the true scientific improvements have been in that model. There’s a pretty big gap. Like, I think, like, we’re we’re moving towards a world where we’re gonna be training these, like, universal models that take any input in. Right? Audio, text, video, you name it, and then generate any output out. And all of humanity’s knowledge will be encoded in one of these models, and e g p four o is a much bigger step towards that than people realize.

So I think that Apple cutting that deal with OpenAI, I think at least part of it is a recognition that I think OpenAI is on a different trajectory compared to others on actual model progress. But at the same time, it also really strongly hints at a commoditized future. To the same extent today, as a consumer, I no longer care whether my computer, my desktop at home is powered by an AMD or Intel CPU. Trying to create a way in which Apple owns the interface and Apple owns the end customer, and then the, like, big brain LLM smarts is just like one hot swappable thing is brilliant for them.

I think what happens is all of the tier one clouds will have their own effort that will do well because it has to do well. And they will do whatever it takes to go ensure that they have the capital and data flywheel and talent to go do that. Then I think for the independent companies and I would say that Adept is very different because we’re what we do is we sell an actual end user facing agent to enterprises, which is a very different business model than selling models to developers.

But companies that sell models to developers will either need to effectively be the, like, first party effort of one of these big clouds, or they have a short window between now and commoditization to build such a big economic flywheel that they can afford to stay independent.

I think it would have to look like something like that. I think that right now that’s why I think of the companies of the independent foundation model companies besides Adept, I’m more excited about places like OpenAI because they have ChatGPT to help do that. Whereas, I think if you’re a pure play model seller, I I think it’s very difficult.

Yes. We’re really, really focused on this particular problem that we’re trying to solve. We’re trying to build an AI agent that you can delegate arbitrary work tasks to. And so that everything we do stems from that. So what we are not doing is we are not trying to just train foundation models to sell them to other people. What we’re doing is we’re building, like, a very vertically integrated stack. Going back to our previous discussion of where will vertical integration happen versus not, I do think that in the agent space, it’s extremely important that you own the entire stack from what is the end user interface.

Think, I like, as we’re talking about the Apple example earlier, owning the interface gives you tremendous leverage in this era of AI to how do you make agents that are reliable enough to be used at work, all the way down to what needs to happen to the foundation modeling layer to enable this whole end to end system to be maximally perform. That’s what we do. It says vertical slice.

Yes. That’s what our advantage is. It’s like, you know, we get this question all the time. Right? With Adept trying to like, we wanna be the system of record for workflows and enterprises. Like, any employee at any large company should be able to teach Adept, hey. Like, here’s how I do this particular thing. Right? Like, here’s how I handle fetching all the data for an insurance claim. Right? And this is really show Adept that, and then Adept should be able do it for them. That generalization, all of those edge cases and variability is why the only way to solve that is to have vertical integration of model with use case.

And it’s also why I think we’ll do better than companies that are just focused on a vertical like, a particular narrow problem. Because I was talking to Parag, who used to be the CEO of Twitter. We were just hanging out the other day, and he’s like, dude, every enterprise workflow is an edge case. And he’s absolutely right, and that’s why you need to control that.

I mean, just even looking at something as simple as I wanna add a new lead to Salesforce. Right? Let’s go outside and find 10 different companies who all use Salesforce and look at how they’ve got it configured, and it all looks completely different from each other.

Yeah.

Yeah. Totally. I mean, this is this is a good question. It’s actually a question that used to cause me a lot of heartburn because I found it so hard to explain to people why agents were gonna be different than RPA. Best analogy I’ve got is RPA is very useful for high volume tasks that always look the same. The analogy that I would give would be RPA is a little bit like you know when you go to a factory floor and there’s robots roaming around everywhere? What those robots do is there’s, like, a literally yellow line painted on the floor, And the robots, like, follow that line.

They go from cell to cell and station to station. They pick up stuff. But what agents are, agents are meant to be constantly thinking and reevaluating and planning at every step to solve your goal, and it’s much more like full self driving. The difference in utility between those two things is fairly large. Of course, there’s many areas where you don’t want something that can have variability, and therefore, you should use RPA. I just think in five to ten years, people are gonna use their computers by giving them high level goals.

I think so.

I think it’s just really fundamentally disruptive to their business model. Like, the way that a big corporation uses UiPath, right, is, like, there’s a big plans around a process transformation that needs to be done. Sometimes, like, an Accenture or something comes in and then maps out what the processes are like sometimes with a process discovery thing. And then RPA engineers go and build those workflows. And then six to nine months later, you hit play on this thing that then automates some invoice processing every night or something like that.

Right? Like, this new model of, like, you just put an agent in there and the agent observes what the the end user does to go do that job, and then that becomes, like, a, like, a thing that you can then just invoke with natural language. It’s, like, really disruptive to the business model. I think that the best way to run circles around incumbents is to do something that has a different business model than what they have.

Well, so basically, the way that we’re doing things is we are addressing use cases initially that are really painful that get our foot in the door. But we’re really focused on how do we make the end user be able to teach it any new capability. Like, I should be able to dump in my standard operating procedure for this new thing my team does, or I should be able to show a depth, like, 10 times and give it corrections on how I enroll a a new nurse into a health care portal in The US, and then the model should be able to do that for me.

And, basically, like, we’re working on something that’s ultimately very self serve over time.

I think in places we’re definitely gonna see that become true, but I actually think in knowledge work, the most valuable things to do will not be priced that way. And here’s why. I think that, like, the definition of price per work assumes repetitiveness, commoditization, cookie cutter, no creativity. I think what these AI systems are gonna do, especially AI agents are gonna do, we are basically gonna, like, give people the ability to go do new things that have way more leverage on their time and, like, give them more opportunities to be creative.

And so then, ultimately, what we’re building is, like, a copilot or a teammate. And copilots and teammates don’t charge you price per work. They like, you really pay them based on their ability to augment your ability to go do new things. Right? Like

I think both of these two things can be true. Like, I think Copilots are a great incumbent strategy because it lets them morph their existing software business model into something that kinda looks the same while getting in on the AI thing. But even separately from that, I just think, like, where are these systems going to be most useful? I just feel like everybody in this field has this vision, right, that, like, AI is gonna take all jobs. The, like, pricing by work thing is just a corollary of AI is gonna take all jobs.

Right? Because then it’s like, alright. Maybe you price by work on invoices, and then next month, you price by work on, like, consulting decks. And then before you know it, you price by work on, like, being AI CEO of, like, David Co. Or something like that. Right? Like, that’s not I don’t think this is how this is gonna play out. I think the way this is gonna play out is that what we’re gonna have is we’re gonna have humans fundamentally be the drivers of these agentic systems that basically give everybody tremendous amount of leverage on their own creativity.

And how can that be built without a copilot style approach? It’s like my question.

does that

Actually, this is something I’m gonna steal from our angel investor, Scott Belsky, who has just thought about this so much. He always calls it, like, this collapsing the talent stack thing. And the idea is basically that projects in teams where the same person is simultaneously the PM and the designer and or the engineer or the go to market person or the marketer or whatever, the more that, like, those different skill sets are smooshed in the same person, the faster that thing moves and the more effective the thing becomes.

So I think what it’s gonna do is it’s gonna make people, humans at work, much more like generalists, and it’s gonna have, like it giving people sort of, like, larger and larger scope over various different, like, areas that are different functions today while they ultimately supervise, like, a cohort of, like, AI copilots that are the specialists.

This is a very, very good question. And I Harry, we should rewatch this podcast in ten years and see how we feel. But I think, like, know, when we talk about AI, like, AI is so freaking broad. It’s a little bit like us asking maybe in the early days of the Internet, a generalized thing about the Internet as well. Like, it’s just I think there are some use cases that are clearly hitting PMF within an enterprise. But for the most part, like, just when we go to enterprises to go to go to go sell them stuff, like, they’ve got so much stuff that’s still on prem.

They’ve still got workflows running on mainframes, and it’s it’s 2024. And so I think even if technologies like cloud, which we’ve probably look at as from a startup lens as being so freaking mature, still doesn’t have full full adoption in enterprises is, like I I think that stuff is really is is really interesting. And I think as a result, like, we’re gonna be on this adoption curve for enterprise AI for a very, very long time. So we are still in the experimental budget phase? I think it’s I think the majority of it is, like, extremely experimental.

Like, one of things we do, for example, is we really try to not sign deals that are coming out of experiment budget because we quality revenue, basically.

Yeah. I think it’s true for most new technology, but definitely but definitely here.

So I feel like in self driving, what happened was there was an moment where you could get the thing to work at all. And then you’re like, okay. Well, now it works 60% of the time. How do we get this in 99.99999% of the time? And every day you show up to work and you just play whack a mole on what’s not working, and you just, like, hope and pray that this converges to that, like, ninety nine point nine nine nine nine nine thing. That’s not true for AI right now.

That’s oh, sorry. That’s not true for specifically, what I’m about to say is only applicable to building smarter and smarter models in agentic systems that ultimately help you do work. That’s that’s the thing that I’m trying to talk about. For building that, that’s not how the the underlying dynamics are right now. Like, every day, we go to work and there’s, like, actually brand new scientific things we wanna try that just dramatically improve the performance of the model. Some of those bets don’t work, and some of those bets really work.

The reasoning that we talked about earlier is an example of one. I think another example of one is, like, this, like, universal multimodality that GPT four o is. Those those breakthroughs are visible. And as a result, what I think has a hope of preventing this from just being a hype cycle that falls flat like AB is that those things are those shoes are yet to drop. And as they do, the capabilities of these models are gonna continue to improve. And on top of that, there’s not a technology that you don’t that you, like, need to get to that level of reliability before it can be deployed.

It’s already deployed

No. I don’t think so. Because I think the third bucket of, like, economic upside is still early, and I think that bucket is the companies that then turn the use cases that have product market fit into repeatable products. Right now, right, imagine you’re very large company x, right, and you need capability y, and then you’ve got the base model over here, right, that’s pretty darn smart, GPT-four or Gemini or whatever. Right? And there’s a giant gulf in the middle. In every one of these cases, the first people to go fill that gulf are, like, sort of consulting y service providers.

Right? But then the moment that gulf starts getting filled, start seeing, ah, okay. Like, this is the really useful thing for an enterprise, then people just go productize that thing, then that becomes a startup. And so then that becomes eventually a company that’s a conduit between the base intelligence and the customer. So today, that might be true for services, but I feel like a lot of these things will be turned into generalizable products. Then when they do, those companies will then be the real economic winners.

I think my main concern right now is actually one of regulatory capture. In the same vein that we’re talking about earlier about how there will be only a few sort of frontier model companies that can exist steady state, I think the the move to go pull up the ladder behind them is already beginning. Lawmakers don’t really understand this technology at all, and so their default instinct is sort of listen to the most credible source. And usually, those credible sources have alternate, ulterior motives here.

I think that what happens is that it becomes harder for the general field to go build on open source. It’ll become harder for new companies to get started that have new AI ideas they wanna go train and scale up. I think what really happens is just another concentration of power moment.

I think two things. One, I think the broader set of concerns about use and misuse and safety are extremely important. And I think that, like, what was a good thing about all of this is that people are having these discussions more more openly, which I I really strongly appreciate. I think with a lot of these systems, you can already see, like, clearer ways to go to go misuse them. Right? Like, spin up a bunch of servers, take the best code model you have out there, use them to go try to find vulnerabilities in software systems.

Like, that’s if that’s already happening, it’s gonna really start kicking into gear. So, like, things like that, I think, make me very concerned. At the same time, I think that, like, AGI is just a really difficult thing to reason about because the way that many people define it is, like, almost defining it as an infinity. And, like, reasoning about infinity is it’s really hard because you multiply infinity by point o o o o o o o 1%, and that’s still infinity. And so I think it’s a very brain breaking thing.

And so I think a better way to go look at it is to look at the path dependence. Like, how will this technology actually be developed in the next five years? And I think in next five years, open will always lag closed. And because open will always lag closed, because open just has fewer resources behind them and fewer incentives for people to go make things to be open as these things become more and more expensive, I view open really as a way for the rest of the field to keep up with the biggest incumbents, and therefore, think it’s actually pretty darn important.

Ah, k. So what I mean by that is that I personally find a world in which increasingly, generally, intelligent systems run around with their own agency and goals and not involve what humans most care about to be not a world that I really wanna live in. And I think because of that and this goes back to the what you were saying about, like, selling AI by work versus as a as a as a software tool. Right? Like, I would much rather live in a world where we have sort of these, like, AI teammates and assistants that we interact with instead.

And then I think the question becomes, how do you find the right interface between smarter and smarter AI systems and people? And how that interface is defined actually changes a lot about what training data you collect, how can humans align these systems towards the preferences of what humans want. Also, ultimately, like, how these models are even built and what their architectures are. And so in a weird way, the way the field is moving is let’s make models smarter, then let’s make use cases smarter, and then let’s go put them in people’s hands, and then let’s figure out what this means for people.

It’s kind of this waterfall sequential method, which I don’t think is a very good way to develop the technology. I think we should start back from ultimately how do how should humans use these things and then create the whole solution end to end that way. And so that’s why the HCI problem, like, people just aren’t spending enough time thinking about. Like, chat is obviously not is obviously not it.

Questions along the lines of, like, as these models get smarter and smarter and they sort of know more and more about the world and have more and more ability to do things in the world, how do you interact with them? How do you supervise them? How do you give them corrections and teach them to be more aligned with what you want? Questions like that.

It’s actually a little bit we were talking about earlier. I actually think agents and chatbots are gonna speciate and turn into two different products. How does that look? I think it’s gonna look like you’re gonna have these, like, rich interactions with these increasingly smart systems that can do things on your behalf, and then you’re gonna go have other systems that you talk to for, like, therapeutic or fun use cases.

Biggest misconception is that this is just gonna be something that at every step takes another human capability and fully automates it. Like, the implicit goal of AGI right now is replace human work. But, like, so much of human work, like, I just don’t think will be neatly captured by AI. And instead, it’s gonna be it’s just like AI will be a tool to level up human intelligence.

Agents in five years’ time, I mean, it’s kinda gonna be like a noninvasive, like, brain computer interface, basically. I think that’s what an agent will be. All of us are gonna be up leveled. It’s gonna feel like the same transition from, like, DOS slash command line to the GUI, but from GUI to agents. We’re gonna interact with them at a high level at the level of goals. And they’re basically just gonna let us, I think, have, like, basically, like, new kinds of thoughts. Like, the ability to go like, reason at one level of distraction beyond what we all do today.

I think one way it won’t happen is that fundamentally, like, are a reframing of where of, like, how software is bundled. Right? Like, today, we’ve we we bundle software in these, like, functional ways. Right? Like, you’ve got you’ve got Notion or Google Docs for for your docs, and then you’ve got Salesforce for sales, and then you’ve got, like, Workday for HR and all of the stuff. Right? But the work that we do fundamentally spans all of these different domains, and an agent should bridge those domains.

Otherwise, you can’t become a higher level thing. So if we’re locked into, like, end walled gardens by incumbents, then that vision will not happen.

I think the the question is what percentage of work done today is addressable by RPA? It’s very little. But what’s the percentage of work done today that’s addressable by agents? It’s like a thousand x that, 10,000 x that. I don’t know. Something in that order of magnitude. It’s just a it’s a very different market. It’s like a it’s like saying, should we work on self driving when self driving didn’t exist? And then looking at the market for those autonomous rovers and warehouses.

I mean, I feel like you did a great job of covering so many interesting things. I think you’ve got all the good stuff. Does that mean I’ve done my research well enough? You’ve done your research very well.

No. You’re you’re doing awesome. The fact that you’re able to do this after a wisdom tooth removal is insane. I had so much fun. I thought you asked great questions across business and tech. And, yeah, I’m excited to see how this all how this all plays out from here.