Chips, Models or Applications; Where is the Value in AI

Yeah. I, I grew up in the middle of, you know, nowhere in Ontario. It was a big 100 acre lot, and it’s a maple forest. And so it was super cool to grow up in, like, the most Canadian environment ever. But it it was very distant from technology, for sure. But you loved gaming, didn’t you? I did love gaming. So I love technology from from scratch. It’s just it was really hard to access it. Like, we couldn’t get Internet. We could do dial up, but I had dial up for years after people had gotten high speed Internet.

And so all my friends, you know, they were online gaming, doing all this sort of stuff. And I was just so jealous. Or or not jealous, but just, like, missing out on this wave of technology, the Internet that was coming about and becoming popular. So it made me obsessed with tech. I would like sit at home with our computer with shitty dial up Internet, and I would just try to make it faster. I would try to make the most out of what I did have. And eventually, that led to wanting to learn how to code and understand how the web works and, you know, can I make this stuff faster?

Can I load the Internet faster? Because I was watching pixels go line by line. And that’s really what pushed me into CS, forced to learn how this tech works so that I could get more out of it.

I I think video games teach something to you. You’re much more willing to grind, to just do repetitive, difficult, painful things towards some broader goal. So that sort of resilience, I think, is important. Then also the the fact that you you can respond. Like you get to try again. You get a second attempt. That optimism or that that framing is really important. I think in a lot of cultures, you get one shot. You have a reputation, and if you fuck it, it’s done. It’s over for you.

Maybe what gaming can give people is a sense of you can fuck up and you can try again and you can get better. And the second time, you fuck up less than the first time. And the third time, you fuck up less than the second time. And so that notion of progress through failure, I think, is probably something very significant for fans.

Yeah. I mean, there’s analogies. So in in machine learning, that’s called curriculum learning. Like, that you want to start, k. Let’s first teach the model to do something very simple to to make it a little bit more complex and build on that knowledge. What’s funny is that curriculum learning has actually failed in machine learning. We we don’t really do curriculum learning. It’s just throw the hardest material and the easiest material all at the same time and let the model figure it out. But, yeah, for humans, it’s it’s so effective.

It’s such an important piece of how we learn. It’s interesting to see that hasn’t taken off.

It’s definitely true that if you throw more compute at the model, if you make the model bigger, it’ll get better. It’s kind of like it’s the most trustworthy way to improve models. It’s also the dumbest. Right? Like, if all else fails, just make it bigger. And so for folks who have a lot of money, that’s a really compelling strategy. Super low risk. You know it’s gonna get better. Just scale the model up, pay more money, pay pay for more compute, and go. I believe in it.

I just think it’s extremely inefficient. There are much better ways. If you look at the past, let’s say, like, year and a half. So between, I guess by now it would be like between ChatGPT coming out or GPT-four coming out and now. GPT-four, if it’s true what they say and it’s 1,700,000,000,000 parameters, this big MOE, we have models that are better than that model that are like 13,000,000,000 parameters. Right? And so the scale of change, like how quickly that became cheaper is absurd, kinda surreal. And so, yes, you you can achieve that quality of model just by scaling, but you probably shouldn’t.

It certainly requires exponential input. You know, you need to continuously be doubling your compute in order to sustain linear gains in in intelligence. But I think that probably goes on for a very, very, very long time. It’ll just keep getting smarter. But you run into, like, economic constraints. Right? Not a lot of people bought the original GPT-four, certainly not a lot of enterprises because it was huge. It was massive, super inefficient to serve, so costly, not smart enough to justify that cost. There’s a lot of pressure on making smaller, more efficient models smarter via data and algorithms, methods, rather than just scaling up due to market forces, just pressure on price.

be both. There’ll be both. The one pattern I think we’ve seen emerge over the past couple years is that people love prototyping with a generally smart model. They don’t wanna prototype with a specific model. They don’t wanna spend the time fine tuning a model to make it specifically good at the thing that they care about. What they wanna do is just grab an expensive big model, prototype with that, prove that it can be done, and then distill that into an efficient focus model at the specific thing they care about.

That pattern has really emerged. I think we’ll continuously exist in a world of multiple models, some focused and verticalized, others completely horizontal.

think if you’re just doing the scaling project, you you have to be one of those, or you have to be a subs an effective subsidiary of one of those companies. But there’s a lot more to be done. Like, if you if you’re not completely adherent to scale as the only path forward, if you believe that there are data innovations, there are model and method innovations. Can we just what are data

Yeah. So, pretty much all of the major gains that we’ve seen in the open source space have come from data improvements. Models getting much better by taking higher quality data from the Internet, better scraping algorithms, parsing those web pages, pulling out the right parts, upwaiting specific parts of the Internet. Because there’s lots of repetition and junk. Right? And so pulling out the most valuable knowledge rich parts of the Internet and emphasizing them to the model, synthetic data and the ability to create new data that is super scalable.

So you can get many, many billions of words or, you know, hundreds of millions of pages of this stuff, but it’s no humans involved, just written by models. Those innovations, the ability to increase the quality of data have led to most of the gains that we’re we’re seeing right now.

Yeah. So this is stuff like new RL algorithms. You know, there’s lots of rumors about Q Star and what that might be. Ideas around search, searching for the solution. So the status quo with models is I I ask you a question and the model is expected to respond immediately with the right answer. That’s an incredibly high burden to place on the model. Right? Like, you couldn’t do that to a human. You couldn’t ask a human a hard question and expect them to just regurgitate the answer immediately.

They need to work through it. They’ve never been in a board meeting. Yeah. Sometimes we do. Sometimes we do. Yeah. There’s like this very obvious next step for models, which is you need to let them think and work through problems. You need to let them fail. They need to try something, fail, understood why they failed, roll that back, and make another attempt. And so at present, there’s no notion of problem solving in models. When we say problem

I think it’s not that reasoning is hard. It’s that there’s not a lot of training data that demonstrates reasoning out on the Internet. The Internet is a lot of the output of a reasoning process. Like, you don’t show your work when you’re writing something on the web. You sort of present your conclusion, present your idea, which is the output of loads of thinking and and experience and discussion. So we just lack the training data. It’s just not freely available. You have to build it yourself. And so that’s what companies like Cohere and and OpenAI and Anthropic, etcetera.

That’s what we’re doing now is collecting data that demonstrates human reasoning. How do you think about competing against, like, OpenAI’s incredible UGC play? Yeah. No. That’s super difficult. And especially with enterprises, they never let you train on their data. And so we can’t train on any of our customers’ data super private. Their perspective is their data is their IP. There’s too many secrets in their IP, and so they’re just not willing to do it. And I I’m super empathetic to that. And so for us, our focus is synthetic data.

We push a lot on that as well as having, like, a human annotation force and scale as a partner for that. We have our own folks in house, but that’s the burden that’s placed on us because we’re not a consumer company. We have to generate this data ourselves. The benefit is we’re more focused, so we have less surface area to cover. So it’s not the entire world showing up and asking us to do potentially anything. It’s like enterprises with very clear patterns for the type of stuff they wanna do.

It’s like they wanna automate certain finance functions or they wanna automate HR functions. And so the scope is reduced dramatically, which lets us really focus in on on those pieces.

that the current LLM API market is dominated by synthetic data. That’s mostly what people are doing. They’re creating data from these big expensive models to fine tune smaller models that are more efficient. So they’re ostensibly, like, distilling the bigger models. I don’t know how sustainable that is as a market, but I definitely think there’s always gonna be a new task or new problem or new demand for data. And whether that comes from models or whether it comes from humans, we’re gonna have to to meet the demand.

For the next little while, it’s gonna be a really tricky game. It won’t be a small market. There will be a lot of

Who’s only selling models and who’s selling models and something else? I don’t wanna name names. But let’s say Cohere right now only sells models. We have an API and you can access our models through that API. I think that that will change soon. There are gonna be changes in in the product landscape and what we offer to sort of push, not away from that, but to add on to that picture and that product suite. But if you’re only selling models, it’s going to be difficult because it’s gonna be like a zero margin business.

Because there’s so much price dumping, people are giving away the model for free. It’ll still be a big business. It’ll still be a pretty high number because people need this tech. It’s growing very, very quickly. But the margins, at least now, are gonna be very, very tight. And so that’s why there is a lot of excitement at the application layer. And I think that discourse in the market is probably right to point out that value is occurring beneath, like at the chip layer, because everyone is spending insane amounts of money on chips to to build these models in the first place.

And then above at the application layer where you see stuff like ChatGPT, which is charged on a per user basis, you know, $20 a month type thing. That seems to be where at this phase value is accruing. I think that the model layer is an attractive business in the long term, but in the short term with the status quo, it is a very low margin commoditized business.

gotten way, way more. Yeah. So it’s a huge chunk of our spend now. Way too much. And you have a direct relationship with Nvidia? Yeah. Yeah. And loads of chip players. Like, we’re we’re close with Nvidia, AMD, in conversations with lots of startups that are building new chips. We also run on TPUs from from Google. And that’s because you don’t wanna have a single point of failure? It’s mostly because market demands it. Like, our customers wanna be able to run on tons of different platforms. They want optionality.

They don’t wanna get locked into one. And so we need to provide a really diverse base of platforms to run on. Similarly to how we’ve been very avoidant to get locked into one cloud, and we wanna be available on every cloud, it’s because market demands it. Like, customers want choice. They don’t wanna get verticalized lock in to one provider.

I think it will be. Right now, chips are just exceptionally high margin, and there’s very, very little choice in the market. That’s changing. I think it’s gonna change faster than other people think.

No. Yeah. The the shortage is going down. I think the it’s becoming clear there are going to be more options available. And not just on the inference side. Like everyone inference is already quite heterogeneous. You actually already have loads of options on the inference side, which is like not the training of the models, but the serving. On the training side, the picture has been, it’s essentially one company that creates the chips that you can use to train big models. That’s still true today, but actually it’s not true today.

There’s two companies. You can definitely train big models on TPUs. Those are actually now a usable platform for super large scale model training, and I think Google has proven that quite convincingly. And then there’s Nvidia. But I think soon AMD, Trainium, these platforms are gonna really be ready for prime time.

I mean, the supply chain thing is, like, really, really interesting. I I think Do you need to build out your own data centers? No. We partner with folks. Is there ever a time when that changes? You know what? We’re an economically rational actor. If it’s cheaper for us to build out our own data centers, we’ll go do that. We’ve run the numbers, and and we feel confident that the price we’re getting from our providers makes that not a really attractive path. The other reason we do it is if there were a chip to come out that was really attractive in its cost profile, but no provider would procure it for us.

We’ve been around for, like, five years now, and so it was well before the whole thing started popping off. So we we were lucky. We Did you expect it to pop off? I mean, I wouldn’t have started the company if I didn’t But, like, little bit, like, over. Not not in the way that it because it was a Totally. It happened later and much more suddenly than I expected.

No. Not not at that moment. In 2017, I was kinda like I was the intern on this transformer paper, and I thought, no. This is just research. You know? We just create new architectures, improve translation scores by 3%, and that’s what it is. I didn’t expect expect all that came of that that architecture, the transformer and the community’s love for it and and real, like, consolidation onto the transformer as a platform for building AI. That I didn’t expect. With language modeling and the whole scaling project, I thought the world would catch on way faster to that piece.

It started to become really obvious, but then it was two, three years before everyone woke up and it sort of hit the world. What was that turning point? Was it ChatGPT? It totally was. Yeah. It was ChatGPT. It was putting the technology directly in front of the user. You don’t have to explain it to your mom or dad or whatever. You can, like, sit down, talk to this thing. Experience what it’s like to talk to these models. Do you think chat is the best interface for consumers?

For some stuff. I think for other stuff, GUI, like a user interface, the traditional visual one is quite good. I think it really depends. Chat as an interface onto everything, I don’t think makes sense. I don’t want to have to type out explicitly my instructions to get stuff done. Like, sometimes I just wanna click some buttons and go through a GUI and get the job done. See, I I don’t think, like, GUIs are dead and that we should replace everything with a text box. But I do think it provides this really compelling interface.

Certainly voice does. Voice is magical. Certainly, it was magical the first time I saw a model write text back to me as compellingly as a human. That happened in 2017 shortly after we submitted the paper. We started training language models on Wikipedia and we sampled from those models and it could write Wikipedia pages as convincing as a human page. That was a very magical moment that computers kind of woke up and started speaking back to us. And then the next time was dialogue as an interface.

So not just I submit an instruction, the model returns a response, but having a conversation over chat with the model.

Absolutely. Like anyone who has tried having a voice based conversation with one of these models, it’s like a stunning experience. You’re kind of left in shock when you hear the model exhibiting emotion and inflection and you hear it breathe to inhale before it speaks, you hear its lip smacking. Like, there’s something so incredibly compelling about that experience. It’s hard to hard to describe until you try it for the first time. It’s such a compelling interface.

There’s like two two things happening. One, it’s getting harder to deliver gains in the models. It is getting more difficult, more arduous, more costly because there was a time where the models were dumb enough that I could pull say dumb enough, but the models were Not sophisticated enough. Yeah. Sufficiently unintelligent that I could pull anyone off the street. Any human was more intelligent than the model and had something to teach it. Right? I could just grab someone, say talk to this model, find errors, and they will, and improve it.

Eventually, the models, like, it was just kind of hard to get people, the average person to find knowledge gaps or that type of thing. And so you had to start going to domain experts. And initially, cheap kind of junior ones, like students of computer science could teach the model something, students of biology could teach the model something. And then the model started getting really good and kind of matching that level of knowledge. You’re just going into more specific and more scarce pools of talent to get them to teach the model their knowledge.

And so it gets more high friction, more expensive to teach the model the incremental new knowledge. When

Mhmm.

Yeah. I mean, fortunately, costs are falling super fast on on everything. Like, compute costs, dollars per FLOP, the scale of modeling Dollars per FLOP? Yeah. Like, how much a FLOP costs Oh, right. Or FLOPs per dollar. So a FLOP is a unit of compute and a model.

No. No. No. It’s a no. It’s a super old thing. It’s floating point operations. So it’s literally like one clock cycle of a That’s amazing. I’m glad I clarified. Yeah. Yeah. Yeah. No. And so if you have, like, 10,000,000,000 parameters, that basically equates to some number of flops. And if you 10 x that, if you have a 100,000,000,000 parameters, that equates to 10 x that number of flops approximately. So anyway, the the price for a flop goes down super, super quickly over time. And so that’s what’s unlocked much larger models today compared to 2017 and even two years ago.

So it becomes cheaper to build last year’s model by, like, a factor of 10 or a 100 each year. We just get better data, cheaper compute. So, yeah, it definitely lowers the barrier to the previous generation of models. The reality of the matter is nobody cares about the previous generation. Nobody wants them. There’s no market for last year’s model. It’s, like, useless in comparison to this year’s model. Any any sort of, like, technological development really makes the last generation obsolete super quickly.

I don’t know if it’s always the pattern that it’s cheaper to build the next generation. I I think with, like, chips, for example, and other very complicated pieces of technology, it does get more expensive to generate each new generation. And we still do it because it’s worth it.

Well, you were you were asking, do the improvements sustain? And I was saying it’s getting harder to improve these models. It’s getting higher friction. And the second the second weird effect is that as these models are getting smarter, each individual’s ability to distinguish between them becomes way harder. You can’t tell the difference between generations because you’re not enough of an expert in medicine, mathematics, physics to actually feel the change. The model is already kind of as good as it can get with all the basic level knowledge, which is what you and I have.

And so when we interact with it, we get the same experience between generations. But in reality, those generations are changing dramatically in much more specific capabilities or raw intelligence. And, yeah, you were asking, is it worth it? Is it worth it to keep spending so much money to push forward? And I think absolutely it is. Absolutely it is. It’s worth it to someone. Right? Why? Even if for you and I, like as consumers, when we’re using this stuff, like we don’t care if it knows C star algebras and like quantum physics.

It doesn’t matter to us. It has no impact on our experience with this technology. But that’s really helpful for a researcher in quantum physics. And so we’ll make more progress there by providing tools. It’s the same question around just technology in general. Like, we have abundant food. We have super cheap cars now, and we have phones in in all of our pockets. We’re kinda good. Like, we’ve got should we really invest in the next generation of technology that focuses creating a new material for a spaceship so that it can get up into orbit more efficiently?

Yeah, we should. And it might not matter to you. Like, you don’t give a shit if the spaceship gets up into orbit cheaper, but it matters to someone a lot and they’re willing to pay and there’s a market for it. And that’s how progress sustains itself.

Yeah. I think there will be a culling of the space. I think it’s already happening. I think a lot of the model builders that were

And I think there’s more more coming. There’s gonna be a consolidation in the space for sure. It’s really dangerous where you when you make yourself a subsidiary of your cloud provider. Why? Well, it’s it’s just not good business. So to raise money as a company, you need to go and convince some investors who they only care about ROI, on that capital. And they give you the money and you go create some value using it. But when you’re you’re doing this raising from cloud providers thing, the math is super different.

make

money. Do you think model for

No. I mean, I I think that everyone who was there at that point is still here. They’re still fighting. So our our first investor was, Radical Ventures, Jordan Jacobs there. He’s still on our board. He’s still I I call him, like, the fourth cofounder of Cohere. He’s built the company alongside us and is still very active and present in in building the company. So I I don’t regret it at all. What was the latest price? The media reports that it was a a little over 5.5.

It’s definitely pressure. It’s good pressure. Everyone gets into the revenue multiples game at some point. Some point, it converges to public market multiples. We are actually in a dramatically better position than a lot of our comparables because our valuation is not at the crazy state that a lot of others are. That’s my belief. We still have to grow into it, but I’m very confident that the market is strong. A lot of people need these models. On the margin side, it’s under pressure right now because of price dumping and because of free models being given out.

But that will change over time and then Cohere, our product stack will also evolve. Which one do you most respect? I would say OpenAI. They paved the way. Like, just sort of like a irrational conviction to this vision of scaling. I remember talking to Ilya about this stuff way before GPT-one, you know, like in the Transformer times around that time because he was big in the Toronto scene. He studied under Geoff. And this this idea of scaling, it was in his head back then years before he actually started pursuing it properly.

And that conviction led to the world that we live in today. This objectively magical technology that’s emerged and is now sitting available to everyone. I really admire earlier.

for consumers more broadly? I mean, I think lately or in the new OpenAI, they’re like a product company. They’re like hardcore building a consumer product. That is their objective, and it seems like it. And it’s working. People love that product. It’s a household name at this point. So I think in the consumer space, they’re going to be a product company. And I think that they have to become one in order to foot the bill to build what they wanna build. If you look at some of the departures, I I would say it seems like the AGI effort is starting to take a backseat to to product and to the the consumer offering.

Well, I think there’s two different camps right now. Some people are pricing the exact same with AI features and using it to drive expansion in their business. And then the other folks like Microsoft, like Salesforce, like Notion as well, they’re charging for the AI features and getting a bigger business as a product. Both of those strategies are are fine and super reasonable. For folks like Canva who are keeping the same price, I mean, I think it’s a good bet. They wanna grow their user base. They wanna expand their user set.

Just give them the most useful product possible. At the moment, don’t worry about margins because the cost of AI is falling super, super quickly. I think that’s reasonable.

It’s mostly trust in the technology, so security. Everyone is very sketched out by the current state of things. Who’s training on my data? Means concerned? Yeah. Yeah. Right. Not like a flop. Yeah. Well, they’re hoping that they don’t have a flop. So they’re they’re really scared that someone’s gonna take their data, train on it, and put them in some sort of, like, security vulnerability or that they’ll lose IP. I think that’s a very valid concern because people have been training on user data.

Yeah. So our our deployment model is set up to do that. We focus on private deployments, like inside their VPC, on prem. Like, what that means is just like it’s on their hardware completely privately. We’re not asking them to send data over to us. We’ll process it and give you back the response from the model. We’re saying we’ll bring our models to where your data is. We can’t see any of it.

the movement back to on prem in this new world? When I speak to folks, it’s super conflicted. In financial services, yeah, people are pulling away from cloud. They’re pulling away from cloud. They’re building out their own data center capacity. Everywhere else still seems to be, we need to migrate to cloud. It doesn’t make sense for us to have these data centers. I think that it probably depends on the vertical that you’re looking at.

There’s a lot of fear around AI being wrong. There’s a hallucination in in these models, and everyone views that as some sort of like, the technology is doomed. Know, sometimes it hallucinates. It doesn’t reflect reality. The models definitely do hallucinate. The hallucination rates have been dropping dramatically, but they’ll they’ll always have some chance of making stuff up or getting something wrong. But we exist in a world with humans, and humans hallucinate constantly. We get stuff wrong. We misremember things. And so we exist in a world that’s robust to error.

And so I I think We just have make a benchmark, so do we?

Yeah. Yeah. Like, VicTARA has one and there are other hallucination benchmarks. And we’re seeing them decrease at the same level as model progression? The same level I don’t know about, but definitely it’s been decreasing super fast. And with RAG, it’s like a step change. I’m sorry. If anyone doesn’t know RAG is? A retrieval augmented generation. So it’s the idea that you have a model. Oh, thank you for that description. It’s the idea that you have a model which can query out to a knowledge base, and that knowledge base might be your internal documents or a search engine.

It might write a query to a search engine, pull back the results, and then use that as part of its answer and cite cite its sources. So it’s saying, I’m making this claim because I read it over here. So now you can audit whether it’s correct. And it also, as a byproduct of the setup, really stops lying as much. It doesn’t have to make up as much because it has reference material it can draw from. And that’s a game changer. For for hallucination, definitely. And also just for, like, customizing the models.

Cause they’ve seen the public web, so they know a lot about public information. But for private stuff, I want my model to be able to answer questions about my email inbox, which is something only I have access to. And so the ability for the model to query my email inbox, pull back that information, it just makes it more knowledgeable about the stuff that I care about.

It it’s really started to shift. So last year, a 100%. It was, like, the year of the the proof of concept. Everyone was sort of testing it out, playing around with it. But recently, there’s been a big shift to urgency to get this tech into production. I think a lot of enterprises are scared of being caught flat footed. They’ve spent a year running POCs and testing stuff out. Now they’re sprinting towards, I wanna put this into production, transform my product, augment my workforce. What’s the number one use case for them in terms of what they need or want?

it’s employee augmentation. It’s these models becoming like a partner or a colleague to your entire workforce. That’s the most popular use case. I think Copilot is the right way to do that. I think Copilot is great. And it’s like the right idea of augmenting a workforce with an assistant, but it’s siloed again within an ecosystem. So it’s plugs into Office and, you know, the Microsoft suite of products. Enterprises don’t just use Microsoft. They use Microsoft for their email and docs and spreadsheets, and then they use Salesforce for their CRM.

They have SAP for their ERP. They have some HRM. They have internal software that they’ve built for themselves. And if you really wanna augment the workforce, you need to have a platform for developing these assistants, these agents that’s agnostic to a particular toolset and that prioritizes the tool sets rationally across what people actually use, what the market actually uses. So I I don’t think that that’s gonna be done by Copilot.

what it does to workflows? I mean, the hype is justified a 100%. That’s the promise of of AI. The promise of these models is that they would be able to carry out work by themselves. That just dramatically transforms productivity. Once you have a model that can go off and do things independently over a very long time horizon, so no longer like, I’m gonna do this one thing for you immediately and return and I’m done. But over the next six months, I’m going to be pumping deals into your top of funnel or something like that.

Right? Like doing outbound for you. It just completely transforms what an organization can do. The hype is justified. I think my critique would be, is that work gonna be most effectively done outside the model builders or within? Who’s gonna be best positioned to actually build that product? Why would it be best done within the models first? Completely depends on the quality of the model. It it entirely depends on the model. Like, the model is the reasoner behind the agent. And you have to be able to intervene at that level.

If you’re not able to actually transform the model to be better at the thing that you care about, if you’re not the one building the model, if you’re just a consumer of the model, you’re structurally disadvantaged to build that product.

Yeah. I wouldn’t either. He’s amazing. And I think he’s very cognizant of the threat against them, and I don’t think he’ll let it happen. So I I don’t see it going anywhere. The other thing is that you forget how sticky enterprise software is. There’s not a lot of, mass displacement of enterprise software. It kind of just stays for decades. It’s really hard to displace an enterprise software company.

There’s an opportunity for really transformative new consumer experiences. And consumers are far less loyal to one provider. They’re gonna go where their friends go. They’re gonna go where, you know, they get the best service, the best product. And so if someone can come along and provide something that is considerably better than what exists today using AI, consumers will move. Who do you think has the best researchers? Other than Cohere, I think it’s quite distributed. It used to be very concentrated. It used to be like Google Brain.

Google Brain, DeepMind. Why were they so behind then? Well, they weren’t in the sense that like two weeks after we released the transformer paper, we started training language models. So we like technologically and research wise, Brain was certainly not behind. I think what’s really important is product vision and the ability to imagine what could be with the technology. It’s not just the technological development itself, it’s the vision of what you can do with it. Even if you have people inside your organization who see that vision, are you equipped to enable them to execute on it, or do they have to go somewhere else to execute on it?

I think those are the questions that you have to ask. And then lastly, I mean, the scaling hypothesis, this idea that models will just continue to get better the more we pump into them and that we should be spending not just 10 times as much on building models, but a 100 times, a thousand times as much to build models. That’s like a super risky, uncertain, and a crazy bet to make. So I definitely don’t knock Google for the decisions it made. OpenAI made very made good bets.

What do you think OpenAI’s best bet was? The scale hypothesis for sure. Like, just that scaling is gonna sustain and that we should continue to 10 x, 10 x, 10 x, 10 x. So many people didn’t believe in that. There was so much pushback on it. Such a stupid, superfluous effort to go pursue, and they had the conviction to to push through.

are we all getting wrong? I think there’s there’s sort of like a meme that’s going around of people saying, we’ve plateaued, nothing’s coming, it’s slowing down. I I actually really think that’s wrong. And not just from like a we need to 10 x compute and and that type of thing perspective and trust me, it’ll get better. But from a methods perspective. So when I was talking about like reasoners and planners and models that can try things, fail, and recover from that failure and carry out tasks that take a long time to accomplish, these are like, for the technologists, obvious things that just don’t exist in the technology today.

We just haven’t had time to turn our focus there and add that capability into the model. For the past year plus, folks have been focusing on that and it will be ready for for production. So we’ll see that come out, and I think that will be a big change in terms of capability.

I think the the product space, the application space is still extremely attractive. There will be new products that come out of this technology that transform social media. People love talking to these models. The usage time is insane.

You might actually be wrong. I think you might want your children to be speaking to an extremely empathetic, extraordinarily intelligent and knowledgeable, safe intelligence that can teach them things and, have fun with them and doesn’t get tired of them, doesn’t snap at them, doesn’t bully them, doesn’t pick on them, imbue them with insecurities. There is no replacement to humans. There’s no replacement. There’s no world where suddenly we all start dating chatbots and human birth rates plummet. I I don’t think that happens. I wanna have a child.

I can’t do that with a chatbot. You know, like a human partner yet. A human partner is way more, infinitely more valuable to me than whatever it like, however compelling a chatbot is. A human is so much more valuable. It’s the same reason why I don’t think we’ll be able to replace humans in the workplace. It’ll be an augmentation. Humanity will become more productive and do more. It’s not that there will be less humans doing the work. You can’t replace humanity. Think about like sales. Right?

If I’m getting sold to by a bot, I’m not buying. It’s that simple. I don’t wanna talk to a machine. Like for certain simple purchases, maybe. But for the purchases that count, the ones that matter to me and my company, I would want a human accountable on the other side of that deal. When something goes wrong, I need someone, a human, who has authority to to be able to intervene. The fears around displacement and replacement, both on the consumer side where we’re all gonna get addicted to chatting to these chatbots and on the the workplace, the end of work, you know, there’s gonna be mass unemployment.

There will be localized displacement for sure. But in the aggregate, it’ll be growth, not displacement. So for sure, there are certain roles that are vulnerable to the technology. It’s kinda hard to come up with them. Like, customer support is definitely one. But at the end of the day, there still needs to be humans there to do that, just not as many as there are today. But customer support is a tough role, psychologically ugly. You get people screaming at you. Like the reality of it, if you’ve ever listened in on all recordings of of what it’s like, that’s a really emotionally taxing job.

Every day you wake up, you go to work, you get screamed at, and have to apologize for hours. That side of things, maybe we let the handle those conversations and the humans can come in and and help with, you know, the actual customer support conversations that humans would enjoy dealing with. They have a problem that needs solving, and they’re not angry about it. There’s just an opportunity to make this person’s life better.

I think robotics is, like, the place where there will be big breakthroughs. The cost needs to come down, but it’s been coming down. And then we need models that are much more robust just because a lot of the barriers have fallen away. Like before, reasoners and planners inside of these robots, like the software behind them, they were brittle and you had to program each task you wanted it to accomplish, and it was super hard coded to a specific environment. So you have to have a kitchen that is laid out exactly like this.

Exactly the same dimensions, nothing different. Yeah. So it’s very brittle. And on the research side, using foundation models, using language models, they’ve actually come up with much better planners that are more dynamic, that are able to reason more naturally around the world. I know this is already being worked on. There’s like 30 humanoid robotic startups and that type of thing. But soon, someone’s gonna crack the nut of general purpose humanoid robotics that are cheap and robust. And so that will be a that’ll be a big shift.

I don’t know if that comes in the next five years or ten years. It’s gonna be somewhere in that range.

Yeah. Yeah. Let’s do it.

I underrated it dramatically. I thought it was just scale. And a lot of proof points have happened internally at Cohere that have just transformed my understanding of what matters in building this technology. So now it’s the quality of data. Yeah. Quality. Like a single bad example, right, amongst like billions. It’s so sensitive. It is a bit surreal how sensitive the models are to their data. Everyone underwrites it.

about 1,000,000,000.

Yeah. It was kind of like a conversation, and here’s a few million bucks. Give it a try. So I think that one was probably the easiest. When you’re trying to raise $500,000,000, it’s a little bit more involved.

And and the interest on it is fantastic. Yeah. Yeah. I do pinch myself. Mean, trying my 25,000,000 a year. I don’t know what the specific number is, but it’s it’s a lot. It’s big number. My brain is broken. Cohere has broken my brain when it comes to economics and money. 500,000,000 does not feel like a lot. Like relative to my competitors, it’s not a lot. Does that worry? No. I mean, it’s part of our strategy. Like if we wanted to go take that deal, we could go take that deal.

But our strategy has been to pursue independence and and doing this ourselves.

have any board member in the world, who do you have among them? Mike Volpe and Jordan Jacobs. My existing board members.

Yeah. Mike’s Mike’s incredible. It kinda feels like he’s seen it all before. Like, I can come to him with virtually any problem, and he’s encountered that three times before. And the first time it went like this, the second time it went like that, third time it went like that. He just has such good experience and advice. Geoff Hinton, Yan Lakun. Which one’s your boy? Definitely more Geoff. I have a a closer personal relationship with him than Lakun, for sure. Do you think Yan is too optimistic?

No. I’m way more aligned with Yan and his beliefs about AI. Geoff is, like, very doomsday pilled, thinks that this technology is gonna destroy the world. Jan is much more optimistic, and I’m I’m aligned in that direction. I think that, unfortunately, Jan has kind of become a Elon reply guy. I I think Geoff is my co founder Nick, he’s super close with Geoff. They they play chess every Monday. Jeff is so so smart, so intelligent, and so thoughtful, such a deep thinker. I I admire him more than almost anyone in the field.

England stands out from the rest of Europe. There’s a technology optimism that exists here and a willingness to invest and make the changes necessary to support developing an ecosystem. In Europe proper, by the way, my mom is British, my dad is Spanish, and I have both citizenships. I’m also very much European, spent my summers there, like family is there. Unfortunately, the culture is just hostile towards tech. It’s hostile. Like the solution to tech is regulation in the European mind. I think there’s pressure to change, though.

And France is becoming much more ambitious and making a lot of noise on the European stage as well as the global stage about we need to be more progressive. It might take a decade, though. Impersonal remote. Cohere was like born in the pandemic, and so we’re totally remote. We’re all over the place. Not totally remote. We have offices in Toronto, London, New York SF. Those are definitely the centers of mass of the company. And people come in every day? Yeah. In person is just so much better.

You can’t even quantify the productivity lift from in person work.

Final one. I don’t think I get asked enough where do you want things to go? Like, I get asked a lot where will things go? I get asked a lot about the downside risks of the technology. There’s so much fear in people’s minds when they think about AI and so little discussion about the opportunities we have. And I don’t think people talk about that nearly enough.

I think that the world is super supply constrained. And pretty much every luxury we have today has come from technology that has developed to increase productivity, boost the supply of things, make them more abundant, make them cheaper. And so what I really care about with this technology is driving productivity for the world and making humans more effective, able to do more. I think it’s so unsexy. Productivity is just like so underhyped.

Like in in Canada, real GDP hasn’t really been increasing. Someone called it the, like, the lost decade. People aren’t getting wealthier. Things are not becoming more abundant. You can’t afford more for a decade. And so that sort of stagnation, you start to get a lot of social turmoil. Things start to be it’s not a growing pie. It’s a fixed pie that you have to fight for your slice of. And I think those dynamics really concern me. Our our priorities as a society right now should be on productivity and growth.