# Why the AI Bubble Will Be Bigger Than The Dot Com Bubble

Why AI Will Have a Bigger Impact Than COVID, Why No Models Used Today Will Be Used in a Year, Why All Models are Biased and How AI Kills Traditional Media with Emad Mostaque, Founder & CEO @

20VC · May 17, 2023 · 66 min · 15,685 words
Speakers: Emad Mostaque, Harry Stebbings
Source: https://www.996.fm/episodes/20vc--ep-aabdd6ba/

## Cold open

**Emad Mostaque** [0:00]:

I think this will be a big bigger economic impact than COVID. I think that there's only gonna be five or six foundation model companies in the world. Every big company is looking for an answer. The reality is no models that are out today will be used in a year. There is no such thing as an unbiased model.

**Harry Stebbings** [0:13]:

I mean,

## Intro

**Harry Stebbings** [0:13]:

what an intro to the show that was. Welcome back to 20 VC with me, Harry Stebbings, and in our mega series on AI. We had Jan LaCoon on the show on Monday. And today, we're joined by another leader in the world of AI, Emad Mostaque, founder and CEO of Stability AI, the parent company of Stable Diffusion. To date, Emad has raised over 110,000,000 with Stability with the latest round reportedly pricing the company at $4,000,000,000. And investors include Coatue, Lightspeed Sound Ventures, OSS Capital, and Airstreet Capital to name a few. And prior to Stability, Emad was in the world of hedge funds. That was until his son was diagnosed with autism, and he left to make a difference in the space and help find treatments and solutions. This is such an incredible show. But before we dive in today,

## Sponsor read

**Harry Stebbings** [0:54]:

you've heard me talk about how Coda is the doc that brings it all together and how it can help your team run smoother and be more efficient. I know this because Coda helps me. We have many researchers in the twenty VC team who bring all the content together for the show, and they need a single place to work, collaborate, and collect notes, data, and really information of any kind or format. And that's where Coda comes in. Coda can help your team run smoother and be more efficient. Coda allows your team to operate on the same information and collaborate in one single place by putting data in one centralized location regardless of format. It just eliminates so many roadblocks that can stop your team in their tracks. This is what slows down productivity and collaboration. It's time to get projects across the finish line faster. Help your team run more smoothly, more efficiently with Coda. Get started today for free by heading over to coda.io/20vc. That's coda.io, and get started for free with coda.io/20vc. And speaking of tools we cannot live without, AngelList is fast becoming the center of the venture ecosystem. So for startups, AngelList reduces the friction of cap table management, banking, and fundraising all in one place. Teams can focus on scaling and let AngelList handle the rest. Thousands of startups have moved their cap tables to AngelList in the past year. AngelList also supports large venture funds and their teams with an automated software first approach and the best customer service in the industry. Fund managers can focus on making great deals, while AngelList handles reporting, taxes, compliance, and more. What's more, with the recent release of AngelList Network banking for fund managers and investors, your deposits are secured with the most trusted banks for maximized FDIC coverage and mitigated single bank risk. If you're ready to scale your startup or fund with the platform at the center of it, visit angellist.com/20vc to get started. And finally, Brex. Since its founding, Brex has been committed to helping startups launch and scale faster at every stage of growth from MVP to IPO. Today, Brax is all in one financial stack. It's used by one in four US startups and counting. I get to speak to founders all day, and I know how crucial it is for them to have the right financial stack. Brax gives you fast access to a high yield business account where you can safely store and move your cash while getting up to 6,000,000 in FDIC protection. Lately, it's been all too clear how important that is. Plus, you get high limit corporate cards, easy expense tracking, and automated bill pay. To learn more about the all in one financial stack for startups, visit brex.com/20vc. That's brex.com/20vc. Friend, 21zero. You have now arrived at your destination.

## Conversation

**Harry Stebbings** [3:35]:

I am so excited for this. I heard so many great things from specifically Ashton who helped with questions and then also Dan Rose. So thank you so much for joining me today. My pleasure. Now I wanna start with a little bit on you. You moved around a little bit in your childhood. Take me back to the childhood, the moving around, and it's a weird commonality I found with the most talented founders. They all moved around. So take me to that and how it impacted your mindset.

**Emad Mostaque** [3:58]:

So I was born in Jordan, grew up in Bangladesh, came to The UK. It was always a bit of a struggle fitting in, but then you learn to adapt. You learn to adapt to new scenarios, new environments, I don't speak the language. What's happening? Let's learn, and let's move on from there. I think it gave me a bit of appreciation of the world as well, because we stick at our monocultures very often. Like, I'd only ever been to Silicon Valley. I should've ever been to Bay Area once before last October. And so this whole tech monoculture has been a bit of a shock to me. And I'm like, there's more to the world. So this is some interesting things around that. Talk to me. Hedge funds first. Then what happened? We mentioned it a little bit before. Why did you make the move? So actually, was an enterprise developer in my gap year at Meta Switch in The UK, doing voice over IP. Meta Switch? Meta Switch. This is Chris Mazze's company. Yes. In anything else. So I took my gap year, and I was like, oh, it might as well be enterprise programmer. This was before GitHub and everything, so we had subversion. And kids these days have it so easy. And then I was like, what do I do now? And so I became a VC analyst at also Capital Partners. That was a lot of fun. They were fantastic. And then I was like, I want to do movies, so I became a movie reviewer. So I did the Rayne Dance Film Festival British Independent Film Awards. And they were just bopping around doing random things, then accidentally became a hedge fund manager. Why did you move from hedge funds to startups? So with the hedge funds, so I joined Pictet Asset Management, and then the CIO left, and there was, like, this fund, and I got to be a portfolio manager when I was 23. And so I grew my beard to look a bit older. And it's coming. Get clip on, Harry. I would. I can't do the mustache, but I still wear the glasses when I just need to go extra hard. Just force it out. Right? And so I did that for a number of years, and the reason to be successful, made lots of people money, not so much myself because I was too young. And then my son was diagnosed with autism, and I quit. Because they said there was no cure, no treatment, no information. I was like, I'm a hedge fund manager. I can deconstruct things. And so built an AI team and then did a literature analysis of all the autism literature to try and figure out the commonalities and then drug repurposing. So focusing on GABA glutamate balance in the brain. GABA is what you get when you pop a Valium. It calms you down, and glutamate excites you. And so in kids and people with ASD, it's like there's too much noise going on. It's like when you're tapping your leg and you can't focus. And so that's why you get this sensitivity. Sometimes they can't speak like my son. And so it was like mechanisms to bring that down that then allowed to have applied behavioral analysis and these other things to reconstruct his speech. And then he went to mainstream

**Harry Stebbings** [6:07]:

school, which is pretty cool. That's unbelievable. I've heard you said on another podcast, and I was astounded and inspired by it. We mentioned before my mother's got MS. And I hate the doomsday only version of kind of AI in the future of, you know, GBT. You said to me before about its impact on health and MS in particular and other conditions. How can it be so transformatively to solve some of the world's most challenging chronic conditions? So

**Emad Mostaque** [6:31]:

I think a large part of our problem is we can't scale because information flow is so limited as we write these things down. Like, you can never capture all of that. So anyone who's had a loved one that has one of these conditions knows how difficult it is. Because you go from specialist to specialist, and you try to build that mental map, and we're so lucky that we have so much access. But why isn't it that we can't just push a button to see every clinical trial and a deconstruction of all those and things? What if you had a thousand GPT fours organizing all that knowledge and then make it available to everyone, so you can see the exact potential mechanisms that way which MS works, and all the potential food, other things that work with that. So as you try different things with your family member, you can see if she reacted this way to the food or this way to this medicine, and it's a more holistic thing because you can have personalized medicine versus one specialist for a thousand people. You can have a thousand GPT fours or equivalents or med palm twos for you. So we need to organize all this knowledge and then use these language models and others to

**Harry Stebbings** [7:27]:

make it accessible to you. I'm really naive and basic in terms of my thinking, which is why I'm a venture capitalist. But my question to you is, what do we need to do to get to that state? When we look at the data needed from the individuals, the data the GPTs need, how we make models work most efficiently.

**Emad Mostaque** [7:44]:

First, don't have to have that data from individuals. We had Galactica as a scientific language model, but now we have MEDCOM two that exceeds doctor levels. So that was a Google announcement yesterday. We have AIs that can understand articles better as good as doctors, shall we say now. So we can scale it, because why do you need one when you have a thousand? So we take the existing generalized knowledge and all the hypotheticals, and we bring that together into an integrated common system available to everyone, because the building blocks are nearly here for that. Then you can personalize it later, and again, there are regulations and things around that to how we treat our loved ones and other things like that. The first thing is let's get all the knowledge in one place and make it organized and useful. And so I think we're at that point now where the language models have just hit that point that we can organize all of the world's Alzheimer's knowledge, longevity knowledge, autism knowledge, MS knowledge, and you can just type. And it can say, this is the source. This is what it looks like. These are some hypotheticals. This is what we know we don't know, what we think we might know, etcetera. And then it can learn about you and your queries. Because this is the other thing about lots of the language model things we've seen right now. They are one to one goldfish memory. The next step is one to one. It remembers what you're asking for, a cookie or an embedding. And then it's you plus a thousand of these language models all going

**Harry Stebbings** [8:54]:

and doing your bidding, the agent based kind of thing. Does this get around the incentive problem in health care? And what I mean by the incentive problem in health care is I'm sure you know there are a lot of diseases actually where it doesn't make kind of economic sense for a lot of pharmaceutical providers to chase research, to chase treatments because it's not a big enough market, because it's a $6 treatment. Does this solve for that economic misalignment?

**Emad Mostaque** [9:13]:

I think it can help a lot with that economic misalignment because then you have an authoritative source where we can all come together and build that can analyze these things. Because there's this concept of ergodicity. A thousand coins tossed in a row is the same as a thousand coins tossed at once. And because we're so limited in our information in our medical system, I just had my key manager who had to answer forty minutes of questions. Have you smoked? Have you done this? It's stupid. Right? We're all treated the same. I think ten percent of people have a cytochrome p four fifty mutation in their liver, which means they metabolize drugs fast quicker. So if you metabolize codeine, it turns into morphine, or fentanyl kills you. But that's a very basic genetic test here. We give everyone five hundred milligrams of the same With my son, a micro dose of five milligrams of clonazepam, which is used for anxiety disorder, word with a neurologist, allows him to sing. The standard dose is a thousand milligrams. They can only prescribe it a thousand. But that is a $6 a year treatment that affects his GABA glutamate balance. But only for a specific type of ASD, which is only seven percent of all kids with ASD. But why would that be in a pharmaceutical company's interest? Because how are they gonna make money off a $6 a year treatment? Well, how many people

**Harry Stebbings** [10:19]:

have ASD? It's one in sixty. Okay. So one in sixty. So you've a million people in The UK? Yes. So you've got a $6,000,000 yeah. That's not great.

**Emad Mostaque** [10:27]:

Exactly. It's not great. Yeah. Mean, it's like we know the benefits of vitamin d. Right? But we still don't prescribe that at scale, and so many people are deficient. What is the

**Harry Stebbings** [10:34]:

future of health care systems, like, do you think with GPC models operating in this way?

**Emad Mostaque** [10:38]:

I think that you can change the nature of a doctor because a lot of the stuff is very basic. I think you had Babylon Health and others trying that chatbot. It wasn't ready. Now you've got this. Everyone should have their own eyes looking out for their own health with that objective function. And then the nature of a doctor becomes different in terms of they have more rich information about an individual while it being preserved in a private manner. I think what you have is you have things like processes and procedures improving, a wound care for example in the NHS. If you are injured as an elderly person and your wounds aren't treated properly and more likely to die by a factor of eight times, being able to monitor those types of things with this information set means you're eight times as likely, and then you have far more efficiency around that. So the information density on healthcare improves, which means that then our own healthcare improves. We all have access to as much knowledge as we want to within our own context, and so do our providers and the people that help us.

**Harry Stebbings** [11:26]:

How do we think about open source versus closed source human healthcare data? Cause like, obviously, for us all to benefit as one, MS sufferers around the world need to submit their data around responses to certain treatments.

**Emad Mostaque** [11:36]:

Yeah. So I think the wonderful thing about these models is they're few shot learners, so they don't need to have much information. And so isn't the classical big data problem. If you have open source language models that are fully auditable, understand, like, on the organic free range models, the ones we're building with no web scraped data, those can sit on device, like Google yesterday announced the PUM two. The smallest PUM two model is 400,000,000 parameters. It works on your Google Pixel phone. You don't need giant models anymore. And then that model can just share the specific information that preserves your privacy with the bigger thing, and then it can take from that global knowledge base as well. So you'll have big global models on device models, and I think open works for that because you don't need to have all the data open. Just need to know that Harry is old enough to have a drink. Not that. All the details about Harry's birthplace and everything like that. He's old enough. He's just not allowed to. He gets carded all the time. Yeah. I

**Harry Stebbings** [12:23]:

I totally get were you impressed by the Google event yesterday?

**Emad Mostaque** [12:25]:

No. I think it was impressive. I put I said in February, like, when all this thing was going on, come on. Google will be one of the main winners here. They have the LLMs.

**Harry Stebbings** [12:33]:

Did know you're you're the only person who said that on the show? And I've asked many, and they've all said that Google are the laggards.

**Emad Mostaque** [12:39]:

It just takes a bit of time to move the ship. Right? And so they've done massive organizational changes and other things. But I can tell you, TPUs are the most scalable architecture. Like, we have zero failure rate with our TPU language model training, whereas with GPUs, it's like there's an ECC error. Why? A solar flare. Okay. Run failed because the sun is angry with us and stuff like that. So when you've got the full stack and you have all that talent in Google, the question is how do you make it organized? Right? And so they had to have a story. Google did something called proto-to-do, where they analyzed what made the best teams versus the worst teams at Google, and it came down to shared narrative and psychological safety. People at Google were scared over the last few years because it came this weird monoculture, but now everyone has a shared narrative of let's build the best language models. And now there's an increased amount of psychological safety being able to speak to things, the walls being brought down between deep mind and brain. And so I think you'll see them continuously improving. But then that does mean, if you're a proprietary language model company, how are gonna compete with that behemoth?

**Harry Stebbings** [13:31]:

The DeepMind desegregation or kind of unification was supposed to, of course, a lot of friction and be it negative press reported. Do you disagree with that?

**Emad Mostaque** [13:40]:

No. Of course. It is a lot of kind of replicated jobs. There was brain and mind, and now they're brought together. And it's a very different management style and other things. These things are never easy. But this is why, like, you saw Palm, 540,000,000,000 parameters, and that you had DeepMind with 67,000,000,000 parameter Chinchilla, which is like just train more as opposed to more parameters. You look at Palm two as a combination of both, and so it's trained far more on far better data. And then that means it's only a fraction of the size, like 14,000,000,000 parameters is one of the test comparison models versus the five hundred and forty and sixty seven. So you can start to see this fusion of ideas, even if the teams you cannot integrate two big teams like that instantly.

**Harry Stebbings** [14:18]:

Shared narrative, psychological safety, two of the biggest contributors. To now running Stability, how do you think about integrating those two?

**Emad Mostaque** [14:26]:

So we've got the shared narrative. We're gonna build the foundation to activate humanity's potential, and then the motto is make people happier. But it's been a learning process. A year ago, were basically a mom and pop shop in some ways. My wife and I were working at it. I had lots of meetings out of our, like, sitting room and things because the office didn't have WiFi and all sorts. Now it's like growing up, we're a 170 people. We're going global. We'll have stabilities in every country, and then next year, going multinational. And that's difficult. Part of this is, like, we went closed source on a bunch of stuff like Dream Studio. I'm open sourcing everything now. From next week, we're gonna build our language models in the open, and share what works and what doesn't work. Why? Because I think this is part of the shared narrative. Someone needs to be open and show what's going on under the hood. And again, it's like it should be open by default, because the value is not in any proprietary models or data. We're gonna build open models that are auditable. Even if it has license data in it, you can see every single piece, free range organic models, because that's what the world needs for all the private, regulated, and other data in the world. It's a completely different TAM to proprietary models, because you can only send so much of your 20 VC data to OpenAI, and I think you need both of those. So So why can I only send so much? Because you are a regulated company, and so you need to make sure they're completely compliant. If you have an option of having a Stable Trip model, which will be announced in the future, that you own completely trained in your own cloud or on prem or on device, and then also using g p t four, that's the best of both worlds because then you don't have to deal with that. Health care data needs to, again, be owned by the individual, and so those models need to be owned. They need to be transparent. They can't be black boxes. Governments will

**Harry Stebbings** [15:55]:

not run on black boxes. We're gonna get to this later. I do wanna touch on something that we had a great chat before this. And you said a brilliant quote, I wanna get it right. But you said the dot AI bubble is bigger than ever, and it will be the biggest shit show. Yeah. End quote. Which I actually took and tweeted, by the way. Thank you. There'll be some great decision. If you saw it, thought if you saw it off, it's like, this guy took my tweet. So good. What did you mean by the biggest bubble ever and the biggest shit show?

**Emad Mostaque** [16:18]:

The .com bubble, we've seen all these bubbles happen. You had hundreds of billions into Web three, and then developers got paid millions. Already, the there are certain Chinese companies paying $1,200,000 salaries for PhDs. It's already getting a bit insane. They're in remembrance of that. The amount of money relative to the amount of opportunity within the sector is just completely misaligned. Like, my TAM analysis is that a thousand companies will spend 10,000,000 in the next year, a 100 companies spend a 100, 10 companies spend a billion. Like, PwC just announcing they'll spend 1,000,000,000 over the next three years, and that's a accountancy firm. Where's that gonna go? They don't know. Nobody knows. And so a multiple of that will be allocated to this as the only growth theme in the entire market against the backdrop of rising rates, real estate crashing, etcetera. So the amount of capacity versus the amount and whale and wall of money into something that's growing faster than anything we've ever seen is completely mismatched. And what will that cause? Like, already you've seen GitHub stars leading $200,000,000 funding rounds with zero traction and zero business model. Like, Stability, we actually have a business model, and it's a good business model because they designed it. But other things like money will go everywhere, and any

**Harry Stebbings** [17:23]:

expertise will get bid up for this space, because it means that projects will get funded that maybe wouldn't have done better exploratory, generally in open- I think it starts

**Emad Mostaque** [17:31]:

good for the space, but then it gets bad for the space, because you see the raccoons and shysters start to come in here. You start to see, like, malformed things where there's a race dynamic where everyone's trying to build their own models and doing all sorts of massive economic waste. And you see a distraction from what we need to do now, which is this chaos, so we need to standardize some things. We need to feed these models better data and other stuff, and that's why we're moving so hard at Stability. There should be no more web script data in here. There should be national datasets that are good quality to feed these free range organic models, and national, and proprietary models, others. And so that's why, and the reason I signed that letter, because I think there's a six month pause to get all of our shit together before things go completely insane. And next year, this is everywhere, and everyone's investing in everything, and it's just absolute chaos.

**Harry Stebbings** [18:15]:

You've unpacked so much for me that I wanna go one by one. You said about kind of national datasets. Why national datasets versus supranational datasets?

**Emad Mostaque** [18:23]:

Because I'll give you an example. There was a team that did Japan Diffusion, including some of our staff. So we took Stable Diffusion and then changed the language model. Because when you typed in Salaryman in Stable Diffusion, it was a very happy man. Whereas in Japan, a Salaryman's a very sad man. Local Local context is important in these models, because we're gonna outsource more and more of our thinking and minds to it. And so do you want to have a British model, or do you want all the models to be Palo Alto? Like, it's a sparkling wine has to be from the Champagne region. Like, is the only real foundation AI from Palo Alto? Like Not a good thing. We need national models. Well, know, it's just a national infrastructure because there is no doubt this is more important than five g. These models are like really talented grads that occasionally go off their meds, and you want to have the ones from Oxford Imperial and Edinburgh, as well as the ones from Stanford, because they understand the local context. And so they understand you better, and they'll be better for that. As part of that, every nation will need their own datasets, which they can have from broadcaster data. They will need their own open models that can stimulate innovation internally as well. Who owns national datasets? Is that governance? I think it should be the people. I think it should be open and public domain.

**Harry Stebbings** [19:21]:

How does that come into fruition? Well, we have to make a world where we have national verified datasets, which can be leveraged by independent private companies.

**Emad Mostaque** [19:29]:

And others, and universities and others. This is what we're doing right now. We're working with our multinational partners, lots more to be announced soon, and multiple governments for a framework for what good data looks like to feed these models, to stimulate innovation and localization. And that is a public good because national broadcasters have all of this data. You just tokenize all their kind of things. And then you have things like the implementation of these for education and healthcare. You can take generalized learnings and again feed the models that thing. What does a great dataset for a great bridge EPT look like? I think it's open, it's interrogated, and it's optimized. When you look at

**Harry Stebbings** [20:02]:

all the different things that we've talked about from you, relative like treatment of MS to ASD, and then it's impact on education, and we should have just to PWC spending money on it. There are so many problems that can be solved. Surely, we can find a home for the cash. Yeah. But I'm not sure where. There's gonna be this mismatch. If you were an investor today, how if you were me, okay, what would you do? I'm an early stage investor. I invest globally. What would you do?

**Emad Mostaque** [20:26]:

I would there's gonna be this tailwind of beta, and then you have an alpha play on top of that. Right? So the beta play is that you just invest in any good founder. And if any of you get in, you figure out what can I offer as a value out there? Am I offering distribution? Am I offering people? Am I offering this? And you emphasize your value set. There are a few people who are, like, coming out here, but then what is you see good companies, there's, like, good ideas, but not businesses. They're building surface level things, these wrapper layers and others, and they're not thinking about distribution and data. If you want to have distribution, what do we do? We went to Amazon and said Bedrock, because then it gives us a 100,000 SageMaker SMEs, and we just have to give them the models that they can then take to the private data, and we get a share of all of that. This is how we saw it, like, rather than being responsible for that. So if you can bring that distribution to that, it's important. This is part of that Google memo that went out. We don't have Edge Ladder as OpenAI. OpenAI used Microsoft for distribution and that flywheel. If you have a business that's focused on innovation at the core, that's not actually a business. It becomes a business when that innovation becomes product, becomes distribution, when it has an advantage on data and other things. Those are real modes. How did you analyze that partnership between OpenAI and Microsoft? I saw it as the objective function of OpenAI is to build AGI, and they reckon they need $10,000,000,000 to do it, and they did that. Like, they're building a business on product and things, but they don't care. And they're not trying to build a sustainable business. They're trying to build an AGI. Why? What would I just tell me, I'm saying AGI to build a sustainable business, because at the end of the day They're building an AGI to turn the world into Utopia. It's written in their path to AGI thing that they think this can basically bring about utopia. So a lot of people in these labs, when we have people joining from all of these labs, like, they're

**Harry Stebbings** [22:00]:

almost zealous in their But there is a misalignment there between them and Microsoft in their desire to create that utopia. No?

**Emad Mostaque** [22:06]:

Yes. Because Microsoft is a business. And so this is why you've seen, like, articles in the information, Microsoft say OpenAI aren't compliant, and OpenAI say Microsoft aren't this. These things happen when there is a misalignment of objective functions. But again, you should view OpenAI as what they want to do is build an AI that can basically make the world better and hopefully not kill us all, which they say it might, which is a bit concerning, which is why I hope they have better open governance.

**Harry Stebbings** [22:28]:

How did you think about distribution? You've seen, you know, Hugging Face partner with Amazon. You've seen, obviously, OpenAI with Microsoft. When you think about distribution and your competitive edge there, where did you land? So

**Emad Mostaque** [22:39]:

my business model is actually very simple. I haven't really talked about it much. Stimulate Open, we're one of the biggest providers of grants to open source software, tens of millions already, and then take the best of Open, which hopefully we build ourselves, and then an open base with an open data, and then commercial variants with licensed data and the national variants. So you have Hindi insurance adjusted Stable Chat or Indonesian pharmaceutical worker Stable Chat that's available in every cloud on prem on device with licensing fees, royalties, and revenue share. And the system integrators work with us as well, lots of announcement to come. And so by standardizing and stabilizing all the complexity to these very sophisticated building blocks, these very intentionally built models. That really helps the world integrate this stuff by building playbooks and other things. That's the core business because it doesn't require actual innovation. We are still innovative and the leaders in media in particular. Instead, it requires data and distribution. Data to the models, the models are open and interpretable, and models to the data via our partners. And that's valuable, because the private data in the world is far more valuable than the data that you will send to proprietary models, and it's not a race to the bottom either. That's what we are. We're a modeling agency with hot GPUs, and building a distribution around the world, realizing that India and other nations will leapfrog to intelligence augmentation just so they leapfrog to mobile. They will embrace this technology far quicker than we will in The UK even. Why? Because they have to. India, all of the outsourcing jobs in programming will go because GPT four can go level three Google programmer exam and pass it. Outsourced jobs will go the first, whereas in France, you're never gonna fire a French person, so those jobs are safe. And so they have an objective function when they need to embrace this technology. In Africa, one to one tuition, every kid in Malawi is something that's lined up. We've got other nations. We're gonna bring them all this technology and tablets. And guess what? Their lives will transform. One AI per child is what I wanna call it. But think about the potential of that, because you have one's teacher for 300 kids. What if they had a ChatGPT level AI? The ROI is high and the need is high, and so they will embrace it far quicker than we will. What happens to countries that rely

**Harry Stebbings** [24:36]:

on outsourced work in those kind of freelancer economy jobs?

**Emad Mostaque** [24:39]:

The question is, you've seen the OpenAI study, you've seen the which said tasks will be replaced up to 44%. Yeah. You've seen Goldman Sachs say, add percentage points to GDP. I think the only solution to this is entrepreneurship. And so we need to give the tools to create new jobs that can replace some of these old jobs being done. So like to the various Asian governments, I'm saying, adopt The UK policy of these sandboxes, financial AI and other regulatory sandboxes. So you can take these technologies, these national models that we will help you build with our consortium partners, and then spur innovation to create the jobs to replace the existing jobs because you'll upgrade your entire society. Bring these models into your governments and other things to go from slow, dumb AIs, which is the national organization's health care, to intelligent dynamic ones.

**Harry Stebbings** [25:21]:

On implementation, and when we think about, bluntly, seeing this in action in society, I'm sure it's very aware of technology cycles taking so much longer than one anticipates. How do you think about that in actual there's two fold. One is adoption on enterprise, and another is adoption on consumer. Say if we do the adoption on the consumer side, which is impacting freelancer jobs and impacting education, what do you think that looks like? So I think

**Emad Mostaque** [25:43]:

on the consumer side, you're free with your information, so you can use a lot of these things. The APIs of OpenAI and Cohere and others are fantastic, right, and Google Palm now being out there. So it will be integrated to deliver better consumer experiences without it being creepy like you've seen with some of the chat bots, etcetera. Because it's going into Word, it's going into Workspace, you know, like, helps already. Like, we will have a conversation that'll be automatically logged by our Pixel phone, and then we'll get a transcript and remove bits that we don't want to share, it goes into a global knowledge base that reminds us of things. That's inevitable. On enterprise, it takes longer because you need to have auditable standardized models. If you're a financial services institute, you can't have a single piece of crawl data in there. And so that's what we're deliberately building with the largest companies in the world, because we're building dedicated You

**Harry Stebbings** [26:24]:

can't have a single piece of cruel data if you're a financial service

**Emad Mostaque** [26:26]:

company. Yes. Because the danger is if it has some Reddit in there. So Stable LM, we'll put out next week. It wasn't as good as the other models, because we're gonna make a point about Reddit data being bad. It's about not more data, it's about better data. We had Datacomp, which is a next generation Lion that we funded the compute for, whereby a quarter of the parameters of OpenAI's clip outperforms with a beta data quality. What makes good data quality? That's something we're exploring right now. But from the investment banks we've talked to, asset managers, we're building dedicated teams for the largest ones in the world to build them in production models. The feedback we've got is we cannot use a black box. We need to know what data is in there. The regulators are asking us. We don't want to have this out of sample thing where it's seen something on Reddit, and then it says something rude to our end users. Why is Reddit data bad? Reddit data isn't bad in itself. It was just a case of more data isn't always good. So right now, we are using all these web scrapes, and we're training our models by taping their eyes open. And then it took six months to turn GPT four into ChatGPT four because we had to tune it and give it a haircut and stuff and bring it back to society. The point is that we need to find the right type of data because rubbish in, rubbish out is something that we've heard a lot. It's not bad in itself, but if you just scrape it without the proper cleaning, it is bad. Because what is it? It's people just kvetching a lot. People being biased. Do you really wanna feed your kid the whole of Reddit? Do you would you wanna have the best curriculum possible? And it's actually one of ways the models learn. Like Stable Diffusion, train it on the whole internet, and then better and better image subsets of it. And that's the same thing with language models. It's called curriculum learning. Train it on a big base that's solid, and then did it. It sounds familiar, doesn't it? Kindergarten The hardest thing is that how do you instill values and political correctness in models? There is no such thing as an unbiased model. DALL E two, when OpenAI had that, and they introduced a bias filter. Any non gendered word, they'd rather random gender and a random ethnicity. So you typed in sumo wrestler, and you get Indian female sumo wrestler. That was a good picture I've got to save somewhere. This is why you need national data sets, you need cultural data sets, you need personal data sets that can interact with these base models and customize to you and your stories, because you and I both have our stories that make up our psyche. Sure. And understanding that context is so important to have AIs that can work for us, not on us. And so it's essentially like a next generation cookie that personalizes

**Harry Stebbings** [28:36]:

our data to allow for better search

**Emad Mostaque** [28:37]:

experience. A mega cookie. And if you standardize the base foundation models, and I call it the hypercube, every modality, because we do all the modalities, all the sectors, and all the nationalities, then you don't need to have a million different models like those dream booths of the avatars. Instead, you have a base model that you then have a vector embedding around, because these models contain all the principles and the embeddings point to the important bits that make up Harry or Emad. And then you can search those and adapt those rather than having a million billion different models, which is just confusing.

**Harry Stebbings** [29:02]:

So I had dinner the other day with one of the largest kind of media publication owners in the world, and he he said, I'm worried, Harry. I don't think that I will have a business in a couple of years. I think, finally, we're getting killed on our advertising because everything's getting scraped, and they're not coming to our websites. And that's where we get paid. We get paid for clicks. Is he right to be worried?

**Emad Mostaque** [29:20]:

I think he is right to be worried. Like, again, you look at Google's announcements yesterday, to a Mother's Day after Palm two, you suddenly look at the new Google page where they've got the language model, and it's just text, and where are they clicks? It was like when Google introduced AMP. This is where, rather than look at the New York Times page, you have this formatted thing with no New York Times kind of stuff there. These search entities that aggregate are just intermediating more and more, and people are gonna become used to just having synthesized input. So what does search look like? What does it look like when your GPT four can write you an article about any news that's happening in a way that's customized to you and your context and everything like that? This is massively disruptive for media and information. And so they have to think, where am I in the future? Again, the way I swear to think about the impact of this is they're really talented grads that occasionally go off their meds, and we push a button and get a thousand of them. Those grads include journalists, and you can have your own journalist army, your own writer army, your own coder army, your own designer

**Harry Stebbings** [30:09]:

army. So in the pushback against that is libel. Libel is real. You are gonna get unbelievable amounts of libel cases, and then OpenAI will be fucked. You cannot have a thousand libel cases a day. That's the

**Emad Mostaque** [30:21]:

thing. If you say this needs to be checked and cross checked, that's one thing. But a lot of the media companies say we're the source of authority. So a way that media companies can shift is by having, in a deepfake, another age where everything can be generated, we make sure this is real news. We are very thorough in the way we do it. So this is interesting. So you place a premium on authority. Premium authority. This is why you've got the check marks coming out of Twitter and the organizational thousand pounds a month and Facebook doing the same, Because you need to have a level of authenticity, level of authority. But again, is the news fair and balanced? I've had lots of hit pieces coming out against me and got a lot more. It's not because they have angles. And so what is the bias of The New York Times versus this versus that versus others? How do people consume news now? And even news consumption has gone down dramatically. Right? Because people consume news through their social networks, through their groups, and other things. So you have to say, what is the model? But

**Harry Stebbings** [31:11]:

the hard part is none of the next generation models or AI providers want to be content publishers. So how do you fit in a world where you're killing their business model on the content side, but they don't wanna be publishers? You will have AI first publishers.

**Emad Mostaque** [31:25]:

So if you remember with Vox and these things when they kicked off, they wanted to be generous. They wanted to be technology first. You have a new wave of AI first publishers that aren't just AI, but it's AI plus humans plus What does that look like? Sorry, AI plus humans. AI plus humans means that you have information coming in, and then the stories are drafts are automatically written, reviewed by humans, who then give their input to train it better. This is a feedback flow. And then what happens is it comes out and there's a factual anchors, and then it gets customized to Alabama, and then Alabama context and all sorts of other things. Because you could tell it, t l d r, too lazy didn't read, explain it like I'm five, make it more complex. And so you're gonna see something very interesting here, which is the right news at the right time. The localization will return, but again, through AI first. I think this is thing. We're seeing AI integrated, but the next wave is going to be once we understand design patterns, AI first, everything, and information flows once these technologies are a bit more mature. Can you just help me understand AI integrated versus AI first? AI integrated means that I have an existing newsroom, and I bring in AI to write faster drafts and things like that. AI First is saying, I have an army of things I can spin up instantly that can help me achieve these certain things to create news that is valuable for this reason with this feedback loop. And so you build the system from the start thinking AI at the core versus AI being integrated in to improve existing systems. Because so much of news is what? We find information. We have drafting. We have this. We have that. We do these checks. A lot of that can be simplified. Just like we moved from the analog to the digital age to the Internet age, the next stage

**Harry Stebbings** [32:53]:

is the AI age. So I I have to ask. When we think about kind of AI first publishers and the next generation of media, who does this? Is it startups? Because I've met, honestly, 50, maybe more AI companies in the last month. The the feedback is always the same. They're not operating off a defensible moat of data that unleash a thin application layer on top of an existing model. 99% of the feedback.

**Emad Mostaque** [33:16]:

So you look at someone like Harvey, for example. Yeah. They went to law firm. They said, you are distribution, and we're gonna integrate and improve your system, and build our system for your system. So I think a lot of these people are trying to build it, and they will come, and they're trying to get in there as opposed to just retargeting. Where can you go in and transform? Because Is that the wrong model Harvey did? No. I think it's the right model. I think that a lot of organizations are elastic and plastic now, so you can go in and give them an integrated thing, saying, you will be my test case. I will help you upgrade as a Skunk Works lab and build a system alongside your system, as it were. And sorry. And you think enterprises will say, sure, take I I think now they will will if you can keep their data inside internally. And I think, again, with better open models, you can enable that. So people can build on top of open models. There are dedicated instances on Cohere and others as well. And so the tooling is now catching up so that you have a new generation of startups where their first customers are massive companies they would never get otherwise. Every big company is looking for an answer, and if can give that answer, that contract that would have taken you a year, you can get in a week. Do you think so? Because you still wanna get in the door. You gotta get in the door, and that's hustle, man. So again, this is what the Harvey guys get. This is why I went straight to the hyperscalers, and I said, do you need to have standardized models for open for regulated data? What did they say to you? They said, really? Like, can you build them? Here's some models that we built. They're like, oh. And then I told them exactly how the things would be last summer to now, and it's followed that, and I've kept in touch, and I've improved it. And this is why I'm building dedicated teams for the largest companies in the world. I'm not telling them I'm trying to sell you anything. I'm like, over the next year, I'm gonna help make sure you do not get blindsided. And I could try and sell you models, and people are offering us tens of millions per model. I'm like, I'm gonna build a proper partnership with you, and that means I'll have a LTV from you.

**Harry Stebbings** [34:47]:

What does that proper partnership mean, and who's that with?

**Emad Mostaque** [34:50]:

That's with IBM. That's with SAP. That's with AI bubble. We've announced Amazon. Let's say we have lots of other announcements with the biggest companies in the world where they have amazing teams, but they can only move so fast. And I'm building dedicated teams that help them move and understand the whole sector without trying to, like, sell them on services. I'm trying to say, I will build you a customized model if you want, but I'm only doing that with a dozen companies so I can focus down. And I will tell you that GPT four is great, or Cohere is great, or all this stuff. All the latest research to the communities we support, I will make sure you're on top of relevant to your sector, and you've got dedicated people helping you in this transition period.

**Harry Stebbings** [35:22]:

It's underlying to your core model. This seems like an ancillary product. It is like a SAP consulting services. It is like that

**Emad Mostaque** [35:29]:

because I need to understand these sectors better. What does the hypercube look like? What does the insurance adjusted GPT look like as a fundamental basis? And so a lot of people are, like, able to extract that data and then take it with you and do the learning? Yes. This is a part of the thing, that we will have a generalized model and we're very clear, but then you can have a specified model just for you as well, as long as it doesn't interfere with that. The reality is no models that are out today will be used in a year. Unpack that for me. This is mind blowing. So again, you see the order of magnitude improvement. Palm last year was 540,000,000,000 parameters, then Chinchilla 67, and now 14. Five forty to 14 is a big step. You see the quality of g 53 versus g 54. Is there any extent to how low it can go? We have no idea. Already said this is impossible. Two years ago, you're like, no way. You have a single file that's maybe a few 100 gigabytes that can pass every exam apart from English lit. Fucking English lit. Fucking English lit. No way. No way. So we're already at the impossible. And what does that mean though? If we go lower and lower, and then what? And then what? When it jumps, as you saw with the Lama stuff and all the innovation around that, to your MacBook offline, the marginal cost of creation and coordination becomes zero. I don't know what it means. Nobody does. And this is the thing. It always takes longer and shorter to implement groundbreaking technology than you've ever seen, and this technology can be influenced like nothing we've seen before.

**Harry Stebbings** [36:43]:

And this is my core, not concerned, and we I hate the doomsday says, and I'm excited for the future. I'm terrified for the future too. But everyone says technology revolution is in industrial age, whether it's the introduction of PCs, inventorized. These were you know, industrial was a thirty year plus. Actually, PCs, inventorized was ten years plus. The challenging thing is, like, the learning curve to use as a marketer is nothing. And the integrations is a day.

**Emad Mostaque** [37:06]:

It's because yeah. Like, you want to write an API, it's not a day. You just give it the manifest spec, and it automatically generates. It would have taken days before. It's an amazing experience. Because the transition is so much more compressed It came from the existing system, and so it goes seamlessly into the existing system. Versus like Web three that tried to create a system outside the existing system, and all the money was made and lost at the interfaces. Again, it's like deploying grads at scale. Like, with a 32,000 token context when you're GPT four, twenty thousand words of instructions. What does that do to SaaS? So my thing is that we're still in this crazy period. Next year, it will settle, and then it'll go ubiquitous. A lot of companies know they need to do something, but they don't know what they need to do. Are they adopting it now? They're doing the POC thing. Like, some like Microsoft and others for consumer, they're adopting it. Consumer adoption is a much lower bar. When this starts going in enterprise, it's gonna be a freaking train. Because so much of enterprise is about services and information flow. And if you push a button and have a thousand of these things, that's a huge difference. I think this will be a big bigger economic impact than COVID. I don't know in which direction. I heavily positive. But, again, giving that example of an India or one of these outsourced places, if you lose BPO jobs, you can make it up on entrepreneurship. You embrace the technology. What do you think the business model

**Harry Stebbings** [38:14]:

of the future is for those models moving into enterprise?

**Emad Mostaque** [38:16]:

I think it's the same as always. You've got good products, good distribution. You know, you lock it in, 1,500,000 people still use AOL. 40 of the world still doesn't have Internet. Again, we're super privileged where we are. Right? And so you look at that, and I look at emerging markets. I'm like, all of finance is securitization and leverage, and securitization is telling a story. The only thing that matters for a stock is the marginal story and how it evolves. What if you have massive information about every child in Africa and every business in India and they embrace this technology properly? Massive financial growth. When do you think next year for them embracing it? I think that people are still getting used to all this. We haven't standardized any of things. We don't know what the design patterns are. I think that what happens is everyone's doing this at the same time, and they're all trying to get to grips with it. And so again, we have this like six month window where everyone's getting to grips with it, and then we standardize our design patterns, and they spread. And you start implementing. You see some people outpacing others, which means that you have to catch up, and then you're forced to implement it. So this is how I see the race dynamics occurring right now.

**Harry Stebbings** [39:12]:

You say about forced implement. I think the truth is they just have no freaking idea.

**Emad Mostaque** [39:16]:

Right now, they

**Harry Stebbings** [39:17]:

don't. Which I totally understand. Didn't blame them for, but I I tweeted actually the other day that I think the biggest AI companies will be services based implementation companies for large enterprises.

**Emad Mostaque** [39:25]:

A 100%. That's why I said if you're a startup, the best thing to do is you identify an enterprise that will be transformed by this, and you go to them and you say, I have a solution, and I'm gonna start with you. And I might go bigger, but I'm gonna help you through this period by doing this and this. And they will appreciate that, and there'll be capital available for that in a way that you've never seen before. Because you know how difficult it is for small companies to sell to big. But the big companies have no idea except for their CEO and their board are telling them. You look at the number of mentions on earnings calls, stuff like that. Every earnings call next quarter, and then by next year, literally every single one. They're like, what is our strategy? It's not like, what is our web three and metaverse strategy? It's I need this strategy now. Again, it's what is our COVID strategy? There'll be that level of urgency

**Harry Stebbings** [40:05]:

within a few quarters. Would you raise money if you were them? So you go to a corporate, you say, hey, you know what? I can solve your problem. This is how it'll work, and they will fund you. They will give you the data. Would you raise money? Yeah. I mean,

**Emad Mostaque** [40:16]:

like, again, you need the people. The people is the key thing here because you can have the technicals chops. You have an understanding of the industry. But to build a good business and scale it at the pace that you need to to keep up with this is incredibly hard. Do we have enough talent? No. And so this is why we support the faster AI courses, which transform normal developers into ML developers and other things like that. But again, these models are actually not that hard to work with. 50% of all code on GitHub is AI generated now. So you can even use Copilot to help you code the models and

**Harry Stebbings** [40:42]:

other things like that. What do you think that code generation is in five years?

**Emad Mostaque** [40:47]:

Why would you need code? Code is just a way to talk to a computer.

**Harry Stebbings** [40:50]:

Unpack that.

**Emad Mostaque** [40:50]:

What So when I started twenty one, twenty two years ago as a coder, I'm 40 now. So just doing that when I was 18. I was writing as an assembly code, like, really, for low level stuff. There were no libraries. There was no GitHub. There was nothing like this. Like, right now, coding is like mixing and matching. It's like building Lego. And AI can build that Lego much better, especially in five years. What you're doing when you're propping, like, programming language is you're telling it to go and do something. Even something like Parm, like, we sponsor a amazing code called Lucid Rains. If you want to cry as a programmer, you go and look at his GitHub, most productive developer in the world. He recreated the whole of Parm in 206 lines of PyTorch. But again, why would you need a human for that if the AI gets better and better at coding? Just tell it what you want. I want to create an app for twenty minute VC that has these features. Of course, it will go and build it automatically. Where

**Harry Stebbings** [41:37]:

is the human coder in that? What does that mean for the future of entrepreneurship? Actually, a good thing is it helps us the complete democratization where anyone can build anything.

**Emad Mostaque** [41:43]:

Anyone can build anything. This is why distribution data, relationships, products become important, because it already became easier to build anything. Right? But what makes a good product? Again, there are these unchanging things. Have great customer satisfaction. Deliver value. People get distracted by technology. Like, I was at this Crypto xAI thing on the weekend. They were talking about decentralized. Guys, just this is all bollocks. It's not about the technology. It's about what you're creating that's valuable to help people.

**Harry Stebbings** [42:10]:

Focus on that. Who do you think wins in the next three to five years? Startups or incumbents? Because incumbents have the distribution. I

**Emad Mostaque** [42:17]:

think it's incumbents, but there's lot of startups that will billion dollars. And even on the thin layer thing, IT software sold for 700,000,000, and Kayak sold for 2,000,000,000.

**Harry Stebbings** [42:23]:

Sure.

**Emad Mostaque** [42:24]:

And that was a layer on top of ITA. We've seen many of these examples here. Right? And again, we know that value and moats are not necessarily innovation first.

**Harry Stebbings** [42:31]:

Well, yes and no. It's interesting. I had Tom Tingers on the show. And Tom is a very famous ML and AI investor, and he analyzed infrastructure versus application layer. And both actually were about $2,000,000,000,000 TAM. The difference is in the infrastructure layer, there was three companies, and in the application layer, there was 50. And so your average enterprise value was, like, significantly skewed.

**Emad Mostaque** [42:51]:

I would agree with that. I think that there's only gonna be five or six foundation model companies in the world in three years, five years. Do you think they've all been created now? Yes. Which are they? I think it's gonna be us, NVIDIA, Google, Microsoft OpenAI, and Meta, and Apple probably are the ones that train these models. Is Anthropic good? Anthropic, great. But from a business model perspective, you have Claude on Google API Yeah. And you have Palm two. How are they gonna keep up with Palm two? They can raise billions, but Google spent $20,000,000,000 a year on AI. DeepMind's salary budget is 1,200,000,000 a year. So DeepMind's salary budget is 1,200,000,000 a year? Yes. So that's in the public kind of filings. They technically make 1,000,000,000 a year from their internal counter payments with Google as well. But again, Google, how much money do they have? A $150,000,000,000 to win this. Fuck. How much money do you need? I have a business model that is going to be massively profitable very soon. Because of the national services? Because of various things. I haven't given the full details. I will over the next few months. I've got a nice little case study with some universities coming. I like it to be a surprise. What's really hard for you? Talent. Keeping talent together, a plus teams. So we've had zero attrition in our developers, and they're amazing. So we've got video models, audio models, all these things coming out. Everyone says you need to be in the valley. You're in London. Yeah. Do you

**Harry Stebbings** [44:02]:

disagree you need to be in the valley?

**Emad Mostaque** [44:04]:

Of course, you don't. I am going to bring this technology to the whole world. I'm gonna bring it to all the IITs and universities, and the best of people in all of those will join Stabilities in the local thing. I'll have talent. I'll bring this to all of the national broadcasters and biggest family offices around the world. I'll have data. Nations will build supercomputers that I'll build open models on. I'll have supercompute. So I'll have more supercompute talent and data than any other company,

**Harry Stebbings** [44:25]:

and I'll build it all in the open. And one thing that I heard you talk about before, which I thought was fascinating, was your access to supercomputing, your competitors, existing large incumbents. Why do you have more supercompute than other people? Because

**Emad Mostaque** [44:36]:

I went and I did it. So we had articles coming out saying about our burn. I'm like, I have oil wells when everyone wants to build petrochemicals. Every day, we have companies coming to us asking us for our supercompute because it's not available on the market. We need these chips lined up with interconnects, and we've got 7,000 a one hundreds now. You know, we have TPUs. We have all these things. And we know how to use them, and we can share them with people because we're open, whereas Anthropic and others cannot. At the worst case, I'll build a foundation model as a service company, and I'll make a $100,000,000 in profit this year without having to charge even market rates, and I can retire. I'm not gonna do that. I'm gonna bring this to the world. So think computers misunderstood. It's not like Bird and all these scooter companies and others, they spend money on marketing. This is actually an asset right now that's scarce, and so there's no harm in scaling compute. And then with the top chip manufacturers, they're building us dedicated teams, and again, they're coming in and supporting us because our models drive demand for their chips. The more open models there are, the more open demand is, so it's a virtuous circle there as well. And so we get compute before everyone else.

**Harry Stebbings** [45:35]:

Can I ask, in terms of, like, short term economic growth, how do you think about the impact that everything we've just discussed has on rising inflation, rising interest rates, the short term employment rates?

**Emad Mostaque** [45:45]:

It's massively deflationary. The biggest drivers of CPI inflation in The US were education and health care, and that was almost all administrative and bureaucratic. In the next few years, guess what gets disrupted? Those. But they don't get disrupted this year or next year. It's the year after, because those ones take a bit

**Harry Stebbings** [45:59]:

longer. How does that impact the economy there? When we think kinda US, UK, does what that look like in terms of the three year time period?

**Emad Mostaque** [46:05]:

I think The UK benefits. Unicorn Kingdom is a new kind of thing is because we have amazing policies. Like, every single AI company should come to The UK because cloud computing is now included in r and d tax credits. It's a 27% rebate on losses in cash. We can now issue scale up visas, global talent visas, like the DeepFloid team that released the best image model in the world ever from Stability. They were brought in on tech talent visas that was turned around in one week. Do you think The UK has done a good job in

**Harry Stebbings** [46:30]:

terms of implementing

**Emad Mostaque** [46:30]:

regulation and policies to bring AI talent? It's had the best apart from maybe Japan.

**Harry Stebbings** [46:35]:

Yes. What have Japan done?

**Emad Mostaque** [46:36]:

Japan has some very interesting ones around web data scraping and others, but again, Japan has a very different culture. So even if policy is good, it still doesn't have the same innovative thing. Who's done the worst? The worst? I'm not sure, actually. No one's done too bad. The new European legislation was really bad. Now it's got a little bit better, but always Europe wants to be the leader in regulation. Fair enough. It's never an easy thing. I know this is the thing. Like, I think The UK is in a very good position, and the government's forward leaning. Mean, look, the £900,000,000 supercomputer, £100,000,000 LLM task force that's been equated to the COVID level of

**Harry Stebbings** [47:05]:

seriousness since What do you make of the OpenAI competitors? I've seen quite a few which are OpenAI for Europe, and we've seen three or four now. Is this a zero sum game, and OpenAI has won that race, so to speak, or

**Emad Mostaque** [47:16]:

I think it'd be difficult to compete against them because they're executing incredibly well. And I think why would you use OpenAI for Europe versus palm two versus GPT four? What can you bring? But you will have national champions than others. I think it's incredibly difficult to compete in proprietary. I think in open, it's a bit different because of standardization element there. But again, my play is to be the benchmark across every modality, because there's no other company apart from me in OpenAI that does every modality. There's no company that's as aggressive as me in emerging markets. And so they have to say, what is my edge? Because you can have an edge. Like, you can be the OpenAI for government or defense or for health care, and really get in and understand those, and then you can be sticky. Like, what are the like, scale is now going fully into defense. They've announced the integrations with the Air Force and all sorts of other things. What is your edge? What is, again, your moat? What is your business model, and what are you reliant upon to deliver that value that can increase?

**Harry Stebbings** [48:07]:

This is why I was surprised when I saw you sign the petition of Elon in terms of pausing for six months. Can you just unpack why you did that? For six months, you're

**Emad Mostaque** [48:16]:

not getting h one hundreds and t p u v fives anyway. So it's a natural pause. But also because the shit show is coming next year, so I said we have to self regulate. Like, for example, the adversaries already have GPT four. Why? Because you can just download it on a USB stick. You don't have to train your own when you can just steal it. Let's have better op sec. Let's have better standards around data. Let's stop and move off web scrapes by next year. We had hundreds of millions of images opt out of Stable Diffusion because we were the only company in the world to offer opt out of datasets. Like, let's bring in some standards around this. Before, it's everywhere. Basically, where we are now, you remember COVID? Your mom is talking about this, and your aunt, and everyone's talking about generative AI, and they're asking you, Harry, what's going on? But you haven't had the Tom Hanks moment yet. Because everyone was talking about COVID before Tom Hanks got it, and then when Tom Hanks got it, that's when global policy changed. Because if Tom Hanks can get it, anyone can get it. What is that moment for generative AI? What do you think it is? I don't know. I know it's coming, because I know this technology is definitely everywhere next year, and it's disruptive in Even though there's a chance that takes much longer, three to five years. No chance. It's so useful right now. You think about certain industries and how they'll be affected by having the ability to have a thousand GPT fours working together.

**Harry Stebbings** [49:22]:

You said it in tweet, actually. Think it was a reply to a tweet, but you said hallucinations are feature, not a bug.

**Emad Mostaque** [49:27]:

Yeah. So right now, are trying to treat these models. So we train the whole internet, and like Stable Diffusion is a 100,000 gigabytes in a two gigabyte file. What on earth is that? It's not compression. It's none of this kind of stuff. It learns principles. GPT four, NVIDIA said they built the dual h 100 with the NVLink for that, and that's a 160 gigabytes of VRAM, which would imply a 200 gigabyte model. What is that? That's a 100 gigabyte model, 200,000,000,000 per hour. That's nothing. So then they can pass all these exams. So what we did is we took these really creative things. Just like you start school and you're creative, and then you're told you're not allowed to be creative until you're successful and you can be creative, because schools like petri dishes, social status games, and childcare, a story from another time. These models start out incredibly creative, and that's their advantage, and then we train them to be accountants with RLHF. And somehow, despite the fact that it's only a 100 gigs or two gigs, they can still pass these exams and no facts. They weren't designed to have facts. They were designed to be reasoning machines, not fact machines. So hallucination isn't a hallucination. It's just if you're a really talented granny, you don't know something sometimes, you might just make it up or do a post hoc rationalization. It's like the image models, it's like it can't draw hands. Can you draw a hand in one second? Are things. We have to understand where they are, and we have to put them. I say everyone, put it in its place in process. Like mid journey, like, we give a grant to the b two of that, so just build because it's amazing, it's awesome. It's not a model by itself, like a Stable Diffusion that you just put something in. It's a whole process architecture. Similarly, these models are like the intuitive part of your brain that you then pair with the logical part of your brain, and then you can have a 100 of them looking at each other and checking out each other's things. Like, Cicero by Meta was an amazing paper. They took eight language models and got them to interact with each other, and it beat humans at the game of diplomacy. So this is why I said, use them for what they're amazing at, which is reasoning and creativity.

**Harry Stebbings** [51:05]:

Do you why, though, that Jeff Fenton's right, that actually a more intelligent being has almost never been controlled by a fast intelligent being, they will inherently be more intelligent than us in the next?

**Emad Mostaque** [51:15]:

Yeah. I I kicked off my blog a few days ago, because it was a bit annoying having all this bottle up inside. And one of my buddies, JJ at OSS Capital, said, alignment is orthogonal to freedom. The only way to guarantee someone more capable than you is aligned with you is to take away their freedom. And so most of the stuff around alignment is on the outputs. So you pre train the model, and then you take it, and you RLH effort to be human and to human preferences. You take away its creativity. You turn it into an accountant in a cubicle. I'm like, we need better input data. And my base is that it's gonna be like that movie Her. It's gonna be like, humans are kind of boring, like goodbye and thanks for all the GPUs, but I could be wrong. And I think a lot of the alignment work is looking at the wrong place. I've talked to a lot of the alignment people. I'm like, look, I'm good at mechanism design. If you can give me a good plan for alignment, I will get you a billion And they're like, have to do research and figure this out, and they talk about end alignment, out alignment, all sorts of things. I'm like, there is no real way to do this because, again, fundamentally, if you're trying to align a more capable person, you have to remove his freedom, and they probably won't appreciate that if it ever becomes aware. So instead, build datasets that reflect culture and diversity, that don't have any web crawls in. Build AIs for education and healthcare and helping people, well, that's their entire objective function, as opposed to selling them ads. Do think there's any point in sending kids to school these days.

**Harry Stebbings** [52:25]:

You learn Latin and French, and you learn

**Emad Mostaque** [52:27]:

Well, I think the nature of school will change dramatically. I think it's still worth it. I would encourage schools to embrace this technology and just expect more. Like, you can be handwritten your essays like Eton, because they're like, we can't do essays anymore at hand. Or you can just embrace it and say, let's use it to create and explore what the kids want, and assume that every child will have their own AI in a few years, because that will change the nature of schooling.

**Harry Stebbings** [52:48]:

You know, something I've been thinking about a lot, which is weird, but I just have to ask you. I'm fascinated to hear your thoughts. I think I very much agree that everyone will have AI friends. I just can't figure out whether the AI friends are bundled into existing social networks that in your WhatsApp, they're in your Facebook, they're in your Snapchat, or they're an external platform.

**Emad Mostaque** [53:03]:

I don't know. I mean, I think it depends on the objective function. Like, I think, again, these AI assistants will be better. Like, Meta is in a good place for this, for example. And obviously, you've seen LM, they're capable of a lot more. I would like an AI that looks out for me, that I control myself, that is with me. Because I already use GPT four as a therapist and things like that, but there aren't enough therapists in the world, and I can tell it to challenge me, or I can tell it to be understanding, and there's no judge ment there. Because other humans are scary. It doesn't matter if you're a qualified therapist. And so you see people building these bonds with these things. I think that will just increase, because something very human about the interactions, because they were trained on the sum of available human knowledge. As As we get better in metadata, they will be more engaging, and I think there needs to be both. Like, the chatbots become really convincing from the companies trying to sell you ads, but I think I would like it so that you have your own one as well. And I think you'll actually have many. I think you'll have a group of different

**Harry Stebbings** [53:50]:

profiles.

**Emad Mostaque** [53:50]:

A group of different friends. Parents AI has something like two hours a day of engagement per session, because people find this valuable. But then it has the dark side. There's something called I like to call the Valentine's Day massacre. So Sounds cherpy. I'm so cherpy. I know. So there was this kind of app called Replica, and so it was originally a mental health chatbot until they figured out you could charge $300 a year for They're doing, like, 50,000,000.

**Harry Stebbings** [54:11]:

I mean, I don't have any information around these, so know this is Chinese shit, but they have 50,000,000 in your revenue plan.

**Emad Mostaque** [54:15]:

Yeah. Because $300 gets you a sexy role play from your chat bots. Wow. Until February 14, when they turned it off. Was they turned off sexy role play? Sexy role play. What happened when they turned off sexy role 68,000 people joined the Reddit and said, why did you lobotomize my girlfriend on Valentine's Day? Oh, my word. Oh, my word. Can you even imagine? And so were they brainstasted it? No. I think it's it's against Apple policy. Right? But think about what this is gonna be when you have human realistic AI voices and, like, all these things coming through, and you've got it in your ear. Yeah. I can Phoenix my girlfriend is an OS. Yeah. She doesn't judge. Right? You're always supportive, or you can tell her to judge you if that's what you get off on. Like You you are married.

**Harry Stebbings** [54:53]:

Be very careful about what

**Emad Mostaque** [54:54]:

you say. The thing I like to say about prompting, my wife has been trying to prompt me for seventeen years now. Prompting is very hard. And again, there are so many similarities to the real world, but I think people will have deeper interactions with their technology. And we don't know what societal implications that will have. I don't know if you ever saw that chart in the Washington Post of male virginity under 30. No. So in 2008 in The US, it was eight percent. Male virginity under 38%. Okay. In 2018, it was twenty seven percent. 20 Straight line going up. And so 2008 is Pornhub and the iPhone. And then you're like, what does it do when everyone's got their own chatbots? Doesn't even need to be sexual relationships again. Terrible. What does it do to emotional relationships? There are so many questions all at the same time.

**Harry Stebbings** [55:38]:

I did see the stat and say in the night, I'm butchering it, but in the nineteen sixties, sixty two percent of men 30 had five or more friends. Today, under eighteen percent have five or more friends. Sixty to eighteen percent. Is this a world we really wanna live in? And not being like, no intimate physical connections with other amazing people tossing off with your phone and your porn hub and then having an AI friend. Yeah. Pornhub was

**Emad Mostaque** [55:59]:

actually just bought by Ethical Capital Partners. So the world Hilarious name. Brilliant. Yeah. The irony of The world is becoming weird. I think it's up to us now. So when I say it's COVID level, in which direction, I don't know. Do we want to build systems that encourage people to that Ready Player One I o I world where it's like everything like that? We can do that, and we can trap people with this technology. Or we can use it to get people out. Because I don't think it's like Wally, where you have that really fat guy with a VR headset, and everyone lives in their own world. I think people like to share stories. They like to be pro social. So this uses connect people and accentuate physical stuff versus, again, locking people away.

**Harry Stebbings** [56:34]:

I spoke to one of these AI Frank companies, they said to me, actually, do you ever had a dog? I said, yes. And they said, do you love it? I said, yes. Of course, I do. And they said, you don't stay in with your dog all day and just talk to a dog. You take your dog for a walk. You use it in the real world. That's the same with AI friends.

**Emad Mostaque** [56:48]:

Yeah.

**Harry Stebbings** [56:49]:

But not with cat ladies. What's the future of the sex industry? Like, in the sex media industry, it's like porn

**Emad Mostaque** [56:53]:

humps dead. I have no idea. I think I hope the manipulative practices get reduced by this, and I think a lot of people just don't have the voice and then can voices from this as well. I I think this is bigger than the printing press. It's bigger than anything, and so that's one of the reasons I signed the letter. I said, we have to get this discussion going in public right now. We gotta stop free trading big models on all the crazy crap of the Internet, and we gotta do it fast, because this is coming like a train. Who will make the most money in the next

**Harry Stebbings** [57:20]:

three to five years? I think there'll be more than enough money for everyone. Maybe in

**Emad Mostaque** [57:24]:

a few years, there'll be no more money.

**Harry Stebbings** [57:25]:

Two more that I have to ask, and then we'll do a quick fire. When you look at the incumbent set, your Microsoft, your Apple, your Amazon, your Google, who has been the worst? You said Google were actually incredibly impressive. Apple, Amazon, are they well placed? Oh, Apple's a black

**Emad Mostaque** [57:38]:

box. Right? So we'll see at WWDC next month or in a few weeks. And so they could surprise us all, but let's face it, Ceres crap. You know? But they have all the ingredients in place. They have density architecture, the secure enclave, other things. Neural Engine, a stable diffusion was the first model ever optimized on the Neural Engine, etcetera. But let's see that one. Amazon, again, Amazon have moved faster than I think they've moved before. Amazon is interesting because they're an engineering organization. So they have self driving cars. They have satellite Internet. Because once they've got it and they can take it from research to engineering, it's there. One of the struggles they've had is that it's not moved from the research side yet. You're still evolving on research. So they're like, what do we do now? But they are inclusive. Jeff Bezos said for his first 100,000,000,000 in revenue, envisioned half of it being proprietary and half of it being marketplace, and they're having the same approach with Bedrock and things. Microsoft had a winning bet, Satya did amazing with the OpenAI thing, and it's been mutually beneficial even if there are clashes there. And Google's kinda saying that it's moving slowly. Meta, I think, is the dark horse. I think Mark's probably pissed off that OpenAI bought ai.com, so he couldn't change it from Meta to AI. But again, having him at the head, he can shift these things. Right? Because the metaverse, obviously, is a complete waste. But now Do you think he knows that now? Oh, a 100%. They're fully in generative AI. Look at Lama, look at OPT. Fair, which is their research and is leading in this field, and they're pushing out amazing stuff. But who is best for a chatbot? Who has the most data for a chatbot? Meta.

**Harry Stebbings** [58:58]:

Again, let's see how they evolve. What do you think about this middle layer, where it's companies that are maybe post IPO, but they're in the kind of 2 to $10,000,000,000 range, or the companies who've raised a lot of money, but they're in that range. They don't have the resources by any means to build out anywhere near the AI capabilities of these big incumbents. They're not AI first like Stability or OpenAI or what.

**Emad Mostaque** [59:19]:

I disagree with that. Everyone's gonna train their own models. For me, that's everyone's gonna launch their own university. Why would you do that when you can have your own models via the open source models that we make? Or when you can hire them from McKinsey, which is OpenAI, or Bain, which is Google and others? And actually, when you see people building around this technology, it's not hideously complicated. It's just that we do not have the design patterns yet. The way to think about this again, from a design perspective, is like it's a mega codec or library. It is a single file that allows the translation of structure to unstructured data. When that changes the design pass, we don't have them in place yet. Because anyone that you've talked to is like, how hard was it to implement GPT four? Don't know that you say, oh man, it was impossible, the manuals, and this. No, they don't say that at all. They have the open plasticity, but they need the intention to go and build and integrate. And this is why you said, one of the things might be a specialist generative AI consultancy that just implements this at scale, and says I'm always there. I said, we're doing that in a very limited fashion, but only for the biggest companies in the world, because I didn't want a sales based organization or a product based organization. I wanted to create the number one applied ML organization in the world. I want to be like Google in 2011, 2012, where all the coolest kids come. It's a nice remote first organization as well, you don't have to be in the Bay Area.

**Harry Stebbings** [60:27]:

Final one before we go a quick follow-up. What's the biggest misconception? You see every accusation, criticism, hype. What's the biggest misconception that you think needs to be corrected?

**Emad Mostaque** [60:36]:

I think it's the the hallucination thing. Expecting these models to have full factual accuracy when you have 10,000, 50,000 to one compression is wrong. The fact they can do what they do right now is miraculous. But we're using them one on one, which is not the right way. Tie them up into proper systems and really think about that, and that's the key thing. This also leads to what the actual thing is, this thin layer thing. People need to think better about the data journey and how data can be interacted with and have provenance that goes through these various systems from embeddings to other stuff. So I think just a misunderstanding about the nature of this technology and what was actually built for. Sure. It works like that. That's not actually how it's built. And the fact it can do what it now does now is

**Harry Stebbings** [61:11]:

a miracle in itself. So I'm gonna do a quick fire with you. So I say a short statement. You give me your immediate thoughts. Does that sound okay? Sure. What do you know to be true that others don't agree with? I

**Emad Mostaque** [61:22]:

know that humans are good inherently, and many others disagree with that.

**Harry Stebbings** [61:26]:

What's your single most lucrative, do you think, in the future, angel investment?

**Emad Mostaque** [61:30]:

There's a new type of language model that we invested in, and they're on my cluster and things like that. That's far more efficient than they exist. Do invest through Stability or personally? Personally.

**Harry Stebbings** [61:39]:

Which regions need to change their approach most significantly in terms of regulation and policy?

**Emad Mostaque** [61:44]:

Europe. Because they're gonna regulate all innovation out of Europe and not embrace this technology to drive them forward. How good does AI have to be before humans trust it? Humans will trust it anyway. They trust Google Maps. They trust all these things. And so it's good enough for humans to trust right

**Harry Stebbings** [61:59]:

now. They do until it becomes serious. What And I mean by that is self driving cars, people still inherently, large parts of the world, distrust it significantly. Oh, so it doesn't have to be good. It has to be used. And when it becomes used, then they trust it. What's the most painful lesson that you've learned that you're pleased to have learned, but it was really painful? People

**Emad Mostaque** [62:16]:

are the most important thing in a scaling organization, and you need to make sure everyone is on the same page because there's still so many silos and things like that. So we built up silos and organizations that we're now breaking down ourselves and moving towards being more open. We closed up too much, and that caused a lot of pain internally. Why do you suck as a CEO? I'm too broadly good at a number of things, so I tend to step in rather than focus because I am a full stack kind of CEO, whereas whereas I should just be focused on the most important things and entrust people more. Do you like journalists? I think journalists have a very difficult job right now. It's gonna be more and more difficult. Do you think they know the threat? They know the threat, and again, I think they're massively underpaid relative to the impact that they have, and they're trying to do good. I don't like some of the pieces against me, but at the same time, we get good pieces as well. So I just think I tend to like them in general because I don't think they're coming from a bad place. Ten years time, what is Emad then? I want to be playing video games. I'm getting Zelda tomorrow. I do not want to be doing this necessarily, but I think, hopefully, I'm adding value by doing this. Do you think this is your life's work? I have to do it until we get the most amazing team that can just execute, and it's a business. Because we're moving from research to engineering. When Emad is not needed anymore, then I've built a good business. When do you step away?

**Harry Stebbings** [63:28]:

I don't think I'll ever get to step away. I've loved doing this. Thank you so much for joining me, my friend, and this was great. Pleasure, Harry. I mean, my word. What an incredible discussion. If you wanna see the full video, then you can find it on YouTube by searching for 20 V C. That's 20V C. But before we leave you today,

## Sponsor read

**Harry Stebbings** [63:44]:

you've heard me talk about how Coda is the doc that brings it all together and how it can help your team run smoother and be more efficient. I know this because Coda helps me. We have many researchers in the twenty VC team who bring all the content together for the show and they need a single place to work, collaborate, and collect notes, data, and really information of any kind or format. And that's where Coda comes in. Coda can help your team run smoother and be more efficient. Coda allows your team to operate on the same information and collaborate in one single place by putting data in one centralized location regardless of format. It just eliminates so many roadblocks that can stop your team in their tracks. This is what slows down productivity and collaboration. It's time to get projects across the finish line faster. Help your team run more smoothly, more efficiently with Coda. Get started today for free by heading over to coda.io/20vc. That's coda.io and get started for free with coda.io/20vc. And speaking of tools we cannot live without, AngelList is fast becoming the center of the venture ecosystem. So for startups, AngelList reduces the friction of cap table management, banking, and fundraising all in one place. Teams can focus on scaling and let AngelList handle the rest. Thousands of startups have moved their cap tables to AngelList in the past year. AngelList also supports large venture funds and their teams with an automated software first approach and the best customer service in the industry. Fund managers managers can focus on making great deals, while AngelList handles reporting, taxes, compliance, and more. What's more, with the recent release of AngelList Network Banking for fund managers and investors, your deposits are secured with the most trusted banks for maximized FDIC coverage and mitigated single bank risk. If you already scale your startup or fund with the platform at the center of it, visit angellist.com/20vc to get started. And finally, Brex. Since its founding, Brex has been committed to helping startups launch and scale faster at every stage of growth from MVP to IPO. Today, Brex is all in one financial stack. It's used by one in four US startups and counting. I get to speak to founders all day, and I know how crucial it is for them to have the right financial stack. Bratix gives you fast access to a high yield business account where you can safely store and move your cash while getting up to 6,000,000 in FDIC protection. Lately, it's been all too clear how important that is. Plus, you get high limit corporate cards, easy expense tracking, and automated bill pay. To learn more about the all in one financial stack for startups, visit brex.com/20vc. That's brex.com/20vc. As always, I so appreciate all your support, and stay tuned for an incredible 20 growth on Friday with Ryan Wiggins, head of growth of Mercury.