Anthropic CPO Mike Krieger: Where Will Value Be Created in a World of AI

Yeah, I think it’s an awesome question. I get a version of this question often from entrepreneurs who I went from purely building startups myself to now running a company that is partly enabling new startups to get created or helping boost their fortunes. The question I get often is, What can I build that is not going to be in the lane of an Anthropic or another one of these labs? I don’t have a perfect answer because it’s hard to have the crystal ball, but my sense of where it ends up being most valuable to exist is places where you have some differentiated go to market, some differentiated knowledge of some particular industry, or some special data that only you have access to.

Ideally, two or even three of those as well. So companies that are within a financial sector, within a legal sector, within healthcare. I mean, healthcare, I’ve gotten exposed to, and it is a tremendously complex ball of yarn. The work up front, it’s not the sexy work. It’s actually not the work that you’re gonna be able to really do in a accelerator or a short amount of time, but it is the work that the legwork that you’ve put in. I think those are durable places to generate value.

And then, you know, you can sit in a place where you can pull on what’s great from the foundation models. You can do your own fine tuning if you need it. You can do your own AI specialization if needed. But the thing that’s gonna give you legs and be durable over the long run is being able to sell into those places, have something that you understand about those places uniquely, and then get better for being deployed there over time.

That’s a great question. I think it can be both. At the highest level, the way I think about AI and product design is, you have to dance this very delicate dance of showing the future and dreaming up what the models are currently capable at their edges, you know, because you wanna design for where they’ll be three months from now, which is how quickly things are moving. But not over promise and under deliver because that’s like a very trust breaking piece. And now, if you’re a startup, you can do a little bit more of the over promising because people are kicking your tires.

They’re early adopters. They have a little bit more of that sort of willingness to engage. It’s much harder if you’re like an existing verticalized SaaS company and you say, we’ve added AI in and people try it and it’s like, it’s not that good. Or like, oh, I thought it was gonna do all these things. Or, you said it could do these 30 things. It does like two of them well. I think that, like, each of those two groups have, like, a very different challenge. On the former, it’s you have established products.

You have established behaviors. You wanna skate to where the puck is going without alienating your existing customers. I think we can dive in. I think there’s some good patterns for doing that. And on the startup front, you probably don’t yet have the data. And it’s like landing the initial sort of lighthouse customers, or you don’t have the relationship. But you have some hypothesis about where AI will have an impact on a given industry or given vertical. And then your differentiation is not the established relationships. It’s painting the future and finding ways of delivering that value quickly within a company that might be willing to take that bet on you.

A really good question. I’ve heard from multiple people that say like, my startup was not a startup until Claude thirty five Sonnet or the second Claude three five Sonnet. But I hear that from entrepreneurs that are, this company was not a company until this model breakthrough where now, you know, the accuracy went up, I don’t know, from ’95 to ’99, and now that’s, you know, close enough for this industry. Or from sometimes, it’s like from ’70 to ’90, sometimes you get those kind of generational leaps as well.

So how to figure out where where that is? Like, there’s times where entrepreneurs have been knocking their heads against the wall within a particular space where whether it’s helping people code, whether it’s helping with legal analysis, whether it’s I mentioned health care, there’s something in that space. And the lovingly assembled version of what they did, which probably involved multiple tools, was either price uncompetitive because it required an Opus class model that was not gonna be supported by the underlying business, is still worth doing because when the model arrives, you’re not starting from square zero.

And so often the companies that do benefit from those model generation shifts are not the ones that suddenly start that day and are like, gosh, you know, it sounds like Claude three seven Sonnet can do that. It’s the ones that have been beating against the wall. I take Cursor as an example. Somebody showed me a, list of Hacker News front page submissions from the Cursor founders over time, and it finally broke through. But that was not their first product or their first sort of, like, iteration on it.

They’ve been trying and and going, I don’t know exactly how long it was. But it was, you know, it was not just quickly enabled by the model. It came from that sort of, building context, building knowledge, building sort of experience about what has gone wrong or gone well with that space so that the model can unlock you. So I guess to be more succinct, don’t wait around for the models to be perfect. Be exploring in this space. Be frustrated by the current generation of the models and then be very aggressively trying the next ones that you can feel like you can now finally deliver on the thing that you saw in your head if only the models were just a bit more capable.

I think it’s a couple of different pieces. On the model layer and on the foundation model layer, especially, I think of three places where it’s worth investing for a long term place in the market. One is talent. And I know it’s hard to quantify exactly what does talent mean, what does talent density mean, but talent begets talent. And you become an attractor and especially talent around a cohesive mission or a story about why you’re building what you’re building. And I’ve absolutely seen that at Anthropic where I love our research team and I feel like monthly we get some new significant hire that has come from potentially another lab, potentially academia and has joined.

And so an advantage you have to cultivate and also maintain because people are obviously free agents and they can do what they want to do. So you have to maintain whatever was attractive in the first place. But that is important because to stay at the frontier, it requires more than just more of the same. It requires also figuring out what the right breakthroughs are. So that’s one. The second one is I think models over time get more different rather than more similar. Of course, there’s a lot of similar benchmarks that people are looking towards.

But there is something Claude y about Claude, and I think there is something GPT about GPT. And they have their pros and cons. And that’s both from a character and tone side of things, but then there’s also the places where those models really excel. And for us, it’s clearly been coding as one really big vertical that we’ve gone after. And it wasn’t an accident. And it’s also not a thing that we just say, great, it’s good at code. They need to be kind of good at code.

It’s seeing that traction and seeing how many companies are now relying on cloud models for code, for example, or for agentic planning inspires the next generation of what you want to do from a reinforcement learning perspective. So the first one is talent. The second one is focus and model characteristics over time that you develop deeper. And then the third one is, I got this question a bunch with DeepSeek when DeepSeek came out. Like, all right, what does DeepSeek mean for you? And I think there’s things that we learned from on the tech side, just looking at what they were doing.

But from a go to market and place in the market perspective, it has almost no impact. And that’s because the relationships we end up having with companies are not they set up for the API. They want to just exchange their input tokens for output tokens at some rate. It’s actually, hey, I want to be your long term AI partner. I want to help co design products with your applied AI team. I want dream big with you. I want to think about not just your API, but also Cloud for Work.

And so it looks more like being a company, which I know sounds trite, but is sort of what you’re providing people is AI partnership, not just AI models. I think the more you are just, maybe it’s good inverting that all to see what the failure mode looks like. I think it is resting on your laurels or not retaining your best people, just believing that making the models incrementally better in every benchmark is enough, and then treating the API as just a way of exchanging money for intelligence without figuring out how to be more of that AI partnership.

If you can’t do all three

It’s getting the environments by which the models get trained in to better and better match real world challenges that aren’t sort of single shot. I know Alex has been thinking about this problem as well because we talked about evals for agentic behavior as one sort of very specific version of the broader thing that I’m talking about, which is even within the realm of software engineering, the work of a software engineer is not just to produce code.

It’s to understand what needs to get produced, to work out the timelines with their product management counterparts, to deeply understand the requirements and deeply understand the user use case that they’re building for, and then also delivering whatever they built in a way that then can be tested and iterated on, and then has user feedback at the other end if they’re building some kind of public facing product. There’s no eval for that. It’s interesting that we call the sort of most common software engineering things SuiteBench. To actually be a Suite is a lot more than just, I looked at a pull request, I produced this pull request, put this to stiff, and then you’re going to accept it or not.

So building environments and evaluations that better mirror that. We think a lot about office professionals at Anthropic in terms of are the use cases that is going to be potentially really multiplied by these models in the future. Nobody’s really evaluating that well. There’s something around research that we’re starting to get a bit better on evaluations. There’s extremely convoluted, I mean that in the best way, evolves like Humanities last exam, which is very much like, Okay, multi step reasoning.

But there’s yet to be the sort of I show up to a new job, I quickly understand what my role is, who is who in the organization, what are the relationships that are being mapped, where to go find extra information if I need it, and then be in the sort of run loop of the functioning of the business. That’s a hard environment to sort of capture.

And so that to me is figuring out how we better either break that down into component parts, which is probably part of the story, but also think about it holistically as the biggest blocker to at least one slice of progress, which is how do models go from being extremely good at extreme slices of things to being more generally helpful collaborators.

I think for the models to improve, you do need a story around how do you perhaps seed it with original human data, but then can generate all these synthetic environments about which you can sort of path find and explore. Claude’s been having fun playing Pokemon this week, which has been a good but kind of funny distraction for our own research and engineering teams. I’m like, what is everybody doing? They’re like, oh, we’re watching the Claude Plays Pokemon livestream. But I think games are an interesting example where you can imagine a lot of different runs through the same game within some constraint and rules.

That gets a lot harder when the problem space is less well defined than did you make it out of the Viridian Forest. I never played Pokemon. I’m learning just watching this livestream. But it’s still important to be able to take golden paths, but also synthesize a variety of approaches through it so that you can still think about how the model can progress in the face of uncertainty. So I think it absolutely has to be a mix. And I think the best models will come from that combination of great, like for code, it’s having good foundational understanding of code and good examples, but then also being able to explore a really wide variety of paths through that.

The other part that is still, I think, underappreciated is how do you measure and evaluate and get data in for character? And I’m going to use a very loose word, is vibes. What is exactly the feel of using a model? We don’t really know until we actually sit down and play with it, which is in some ways kind of a nice property of it because it means it’s almost this very qualitative, human like aspect to it. But it also means that you don’t have good regression testing on it.

Sometimes we’ll go from Claude 3.5 to 3.7 and people will say, oh, Claude seems friendlier but more terse. Or Claude seems more willing to answer my questions, but I wish it was better at creative writing. These things are not easily evaluable. This goes to the data question. And so I think it is important to both be able to have the data in there around these more softer skills, but then also have the evaluations for them.

No, there’s a concept that I love from my background was in human computer interaction. And you might have heard this term of leaky abstractions, right? Which is, I guess, with software builders, we try to do perfect job of encapsulating all the complexity under some little shell. And then the users should not have to think about any of these things. And the reality is the current state of most AI product design is an extraordinarily leaky abstraction. But take having to choose the model, why should you choose between Opus, Haiku, or Sonnet?

Most people don’t understand the difference. Or if you go to the OpenAI, drop them and select them. There’s a lot of models in there. Every single one of them has a good reason for being there. And yet the overall experience is one of, why would I choose one over the other? Oh, this capability is available here, but not there. I mean, we suffer from this problem as well. So model selection. The second one is the once you understand how these models are built, they build up context.

They have turns. Every turn actually has the full context replayed to it. That’s how it’s able to make the next inference. What that leads to is this experience where every chat is different, which I always think of when you’re talking to a coworker, you might have different email threads, but it’s still one coworker behind all of that. And if you reference their favorite sports team or you reference a project you’ve worked on together, it’s not like, oh, I don’t know what you’re talking about, or I’m gonna have to go retrieve my memory.

It’s sort of like a shared underlying piece. That’s like another, it’s forcing people into a understanding of the models that I don’t feel like we should be having people meet there. And the last one is prompting, which is as much as things have evolved, and we’ve done a bunch of work around like how do we take simple human prompts and then translate them into the ones that are very model optimal, I want to make that absolutely transparent to people where it’s not something that they’re engaging with it.

And if the model has a lack of clarity on the problem or needs help understanding better, that then it engages in conversation rather than seeing the difference between somebody who’s an extremely good prompter versus not. Now that gap closes generation to generation, but I like, we need to collapse it even further.

You can’t separate the two anymore. And I think to be a UX designer I was just in a product review right before our call. And, you know, I was thinking about like Instagram product design sessions. It was pixels, some synthetic data, or maybe real data. You know, we took my feed and then we reformatted it to this UX that we’re proposing. But there’s not a lot of nondeterminism there. You’re going to put it out to the world and maybe people will use it in some ways.

But designers and product managers and definitely engineers today need to think, all right, what I’m actually doing is I’m designing a scaffold and a product around a fundamentally nondeterministic system, which means the evaluation, the model quality, the prompting on the back end, all is part of the product design, which is it’s going to have direct implications. So one example is you can prompt Claude to ask follow-up questions or not. And that might be what you want in one part of the product, but not another part of the product.

You might prompt Claude to want to go and think longer about a problem and do more reasoning or not. And again, these are all decisions that upfront you are making in product design. And they’re going to have this manifestation in the actual product. And then the other piece, we talked a little bit earlier about as a startup founder or as somebody who’s doing maybe classic B2B SaaS, you need to figure out, triangulate where the models are, where they’re going, what the user needs are together.

That’s going to be the case in your product design as well, where you’re doing the evaluations, hopefully upfront to see if what you’re doing is even possible with the car models or at least having an eye out for when they might be. But models change over time. Products change over time. If you don’t have a good framework around evaluation, even regression testing those evaluations, you might end up launching a product that three months later people are like, oh, the product used to be good, but something else has happened where it’s no longer serving that purpose.

And you’re like, but I’m not sure which of these three things change. Is it the model? Is it the product design? Is it the introduction of a different feature? The system prop got longer. It’s in many ways the most complex product development work I’ll ever do.

I think about this a lot, and especially because you have different surfaces and different audiences that have different both expectations of stability or sort of desire to be on the cutting edge. And so in an API product, like what people value is predictability and stability and the opt in of something that’s more future facing. Right? And so it can be a very opt in thing. So I remember we launched prompt caching, is a big cost savings for people. Initially we did that via like a beta header that you had to opt into.

And a lot of what we do on the API is in that bar. If you do that for our customer facing, like our more consumer stuff, that’s really lame to have to like have people opt in or like really, you wanna be able to sort of iteratively release and be experimental with folks. And you don’t want to totally break their experience, but you’ve got a little bit more of that permission. And then we have all these enterprise customers that are using Claude for Work in an enterprise.

Now, I think AI adoption in enterprise is still a early adopter product in the enterprise. So you can get away with more than if you’re, I don’t know how many releases Salesforce does a year, but I know a lot of these companies do like two, right, or three. And it’s usually oriented around some big event that they can do. We’re really far from that. We’re still launching pretty quickly, but we’re honestly still finding the balance there where, is it a monthly drop? Is it you ship as often as you can, but there’s an admin opt in on each kind of thing?

That adds complexity as well. And so it’s a great question. I would say it’s an active topic of conversation, how raw or how quickly we can ship knowing that want to bring things out to the world and you don’t know how they’re going be received and you want to learn. But as you accumulate sort of notoriety or people start depending on you for workflows, you can’t treat that completely of wantonly.

Yeah, I mean, it is much more complex. Instagram, the things that you had to watch out for the big rocks were very known in advance. Like don’t launch anything during WWC week. That’s going to be a flurry of announcements or the September iOS event. There might be some other big rock like holidays. Is so much easier from a product marketing perspective. Where here, it reminds me a little bit of Crossy Road where you’re like, Okay, the car’s going by. All right, there’s a gap in the car.

Launch tomorrow. We’re like, now it’s good. But oh, now we hear there’s a rumor. It’s so much hard. And I’ve heard from folks at other labs as well that everybody’s kind of trying to read the tea leaves and be like, all right, is anybody, is it quiet? Is it okay to launch now? Or like, I think we do it next Tuesday. So it’s much harder. Go, go, go. It requires a completely different approach. And, you know, I give credit to our, literally our product marketing team because they’ve had to orient from a point where, you know, we were Claude 3.7, we launched on Monday.

And we locked the blog post for that Sunday night at 9PM, which is not best practice from a marketing perspective. We were briefing press that day on Sunday. Thank you to folks that helped on the phone with us on Sunday. But that’s the point where everything is done and ready and locked and we can go. And so it does involve that sort of ability to react quickly and be nimble. I mean, even things like when we release a model, there’s a model card and there’s evaluations and a comparison table.

There are things in that comparison table that were released the week before, right? Grok three was just a week prior. So it involves a What whole

One of the things I try to do to support the team there is remind, model releases are going to happen and at any given point you are going to be, it’s so over, we’re so back. Like that cycle, it’s like you have to live that in AI and you can’t get to down about one release because, yeah, for sure, it is inevitable. And sometimes you’re lucky and there’s like two or three months where the model that you launched or the product that you launched is still state of the art across all the things you really care about.

Sometimes you’d last a week. And you can’t over rotate on either of those. You can’t rest on your laurels. You can’t do that. I think the thing that’s really useful too is, and it’s a chart I show to almost every every sales call, which is just mapping from Anthropic’s founding to where we are today and the milestones. And at any given point, you could say, wow, like Claude two, that’s like pretty far behind. Oh, Claude three is state of the art. And then no, it’s not.

And you got to look at the trajectory and trust that you are going to continue to make improvements is number one. And then number two, remind yourself that if everybody switched every single day purely due to eval being changed, one, that would be like an insane thing to do to your user base as a provider of software. But two, that would make for an even crazier industry. Over time, you start learning that people don’t just deploy models. They’re doing fine tunes or they’re deploying models. Plus they’ve done a lot of really bespoke work to make that model be great for that use case.

It’s not a thing that’s going to switch overnight. Or you’re one of three or four options within a model selector, right? Which like, for example, in coding environment. So you’re still in the mix and you still have a chance. But I’m not sure if it’s like finding the meditative zoom out angle of it or just like getting used to the bumpy ride or some combination of the two. But it is for sure a thing that like every time there’s a model launch, I assume every one of those labs is like either watching the launch stream, looking at the videos, being either, whew, or all right, now we got work to do.

I don’t. I think that is right. And I think especially on the consumer front, I was just reading Ben Thompson, he does Nat Foodman and Daniela Gross on there pretty often. And they’re talking about some people being Claude people and some people with ChatGPT people. I think that definitely happens where you like the personality, you like the interface design, you like the vibe. Again, it actually reminds me a lot. We had this interesting back and forth with Snapchat over the years with Instagram. And then even before that, people would launch a new product that’s like Instagram, but just for super high end photographers or with this like additional twist or just one photo a day, you know, let’s be real.

And I had this like fake formula. I’m not the mathematician, clearly Anthropic, but it was social networks are made of format or formats that you have in your product, audience, and vibes. And format, for Instagram, we had Stories, we had Feed, and then eventually we had video. Audience initially was sort of hipstery photographers. Eventually grew to be anybody that’s really interested in sort of visual storytelling or visual media. But the vibes of Instagram, even when we had more product similarities to a Snapchat, even to a Facebook, the vibes are very different.

And I don’t know what that fake formula is for AI products yet, but I think it’s some version of that where there’s like model personality is probably one of them. There’s likely something around the scaffolding prescriptiveness of the product that you’re working around it. And then there’s vibes. Like, I get hard to measure, but absolutely there.

Well, even like let’s take within the labs. I assume every single one of the labs is using, even within themselves, it is very valuable to be able to take the knowledge of your highest end model and then be able to make it lower latency, more affordable, etcetera. So there’s that loop, etcetera, overall. I think the places where this gets interesting are, one, do we want any nation to be able to be able to distill models from any other ones? Personal answer is no. I think that there’s value in even as AI gains and capabilities, being really thoughtful about that from a national security perspective.

And then the other piece is to have the advancements happen at the rate that they happen and be sustainable long term. You do need the labs to be able to commercialize all of that training and innovation, etcetera. And I think finding the right models for that long term is important. I think the open source models, take LLM, for example, they’ve been able to do that from their own research and perspective and data ingestion and training. And so I guess I would say distillation does not feel essential in order to unlock those things and poses other issues, even just from a terms of service perspective.

I think it’s a good interesting question is like whether LLM is the quality of LLM due to the fact that they can I don’t if they’ve said that they do, but they clearly can train on on Instagram and Facebook and etcetera data? Or is Gemini better for being able to train on on YouTube? It’s actually clearer to me that Gemini benefits from that. Like, whenever they have like a good video understanding demo, for example, I’m like, well, has like probably the largest repository of video in the world and can likely train on a lot of those pieces.

Less clear on the Facebook front. I’ve never heard from people, gosh, you know what LLM does extremely well is generate good content that would work well on social media. It just seems like a good general purpose model. So I’d actually go back to the value is all in how good is your team? Do you have the underlying data that you need to go do it? But then also, how useful is your model in actual use cases? And that is the highest order bit.

I almost wish I’d started with that because evals aside, evals are really useful for hill climbing and for internal research, but they don’t tell the story of like, is the model going to be excellent at what it needs to be excellent or deployed for? Or even if it is excellent at that thing, is it only excellent at that thing in very narrow situations? Or is it something that as an entrepreneur, outside the labs, you can rely on the model to be your representative, I guess, in that product?

So yeah, think the values for the labs, the values in the team, it’s in the model’s ability to actually perform the right actions in the real world without so much nondeterminism that it becomes sort of unreliable.

Yeah, I think the DeepSeek piece that people seem surprised that there were sort of cutting edge research teams there. And if you were paying attention, that part should not have been the surprising piece. We saw Instagram was blocked in China fairly early. And then we saw the sort of emergence of a parallel world of startups. When if you take up Facebook and Instagram, what happens and what emerges. And those products were often very high quality. They demonstrate a lot of creative thinking and were built at scale too.

They were solving problems. People love talking about the super app and WeChat. And there were some technical challenges solved by those at scale that were of the same scale of challenges that Facebook was doing. So it would absolutely be a mistake to have underestimated or continue to underestimate China’s ability to both train at the frontier, especially if they get access to compute, and then continue to innovate there too. So I think it’s a pretty Western centric view that I’ve definitely seen happen in more of traditional software.

I’m like, well, maybe it’s caught in this ’90s, early 2000s year of like, oh, all they’re doing is replicating what’s already being working elsewhere and doing that. There’s been products that I think take a differentiated view and grow, you know, internal to the Chinese market. And then sometimes make that a bit extra. I mean, TikTok being an interesting example of that on the other side.

There’s some architectural pieces that I won’t speak for the research team because they’re, you know, they’re they’re definitely the deep experts, but they’re like, oh, interesting. Like, that That’s worth us considering. Or some ideas that had been considered and maybe were worth reevaluating. So I think there’s that piece there as well. It’s interesting. Our plan was already to show the chain of thought when we launched our reasoning model. So that was not a reconsideration, but maybe it was interesting to see somebody else do that. And there’s some user interface kind of details in there.

And I think Grock does as well now and there’s. So I’d be curious to see how that evolves. To your distillation question, that might be a reason why more labs either choose to not show or otherwise obscure the chain of thought down the line. The other piece that like from a product perspective, there were two. I think that’s like the under talked about piece of DeepSeek. It’s, think I they were able to go from nobody knowing about them to them being like, frankly, in many circles, known than Claude.

Right? Like I like great aunt was calling me about DeepSeek. I’m even joking. Like, it was like cliche was actually happening. Like, I had to think like, what do you think about DeepSeek? I’m like, great. Like it’s broken through. And that to me What do you think they did to break through that maybe Claude hadn’t? I think there is a lot of interest, of course, in world politics right now. Have the narrative be this was much cheaper. And whether that was exactly true or like what, oh, they were able to figure something out.

It’s the story. Frankly, and I’ve had this conversation with our marketing team as well. I don’t think we tell the Claude story well enough externally yet around what is different or what is notable about the fact that at Claude three, we were training a model at the frontier that was state of the art with a team that was much, much, much smaller than any other lab. I think we’ve been always very efficient with our compute as we train. So I don’t know. I think there was whether that was a story that they told or was just told for them by the media because it legitimately was a really compelling story, the sort of uniqueness of the moment was a big piece there.

And I think especially like it’s January, new presidency, China relations, like it fed into the moment really, really, really well. So I think that worked well. And the second part on the product, like they went from not having a product to having like an iOS app that actually had a lot of good details. And for me, it was a good, I was gonna say nudge, it was stronger than that. Like a shove around, we need to be getting some of the ideas out to market quicker without, to your earlier question, focusing as much on exactly the polish that it needs to happen in every situation and instead be willing to put it out there and learn.

Because sometimes the novelty of experience is itself valuable, right? It was the first time most people experienced the live chain of thought. That’s interesting. And like, I wish we had done that sooner because it would have been novel for people to experience that.

I think that they already have this sort of like, they’re known at a level where that has some ability to generate that ongoing staying powder, etcetera. On the retention front, I think if all we’re doing in these AI first sort of lab generated products, even six months from now or year from now, it’s like asking questions, maybe sometimes having like slight proactivity. I don’t think that’s differentiated or interesting in the long run. It should be, wow, I can now do something uniquely because I am using Claude or I’m using DeepSeek or any one of these products.

And it unlocked hours of work for me. And it made me smarter. And it made me a better partner to whoever are the important people in my lives. It has to transcend beyond surface level utility. Some people find the deeper level, don’t get me wrong. And those are the people that are your DAUs right now. But for a lot of people, they’ll try it. They generate a poem with it. They write a letter to their son. There’s all this stuff that they can do that provide some value in the moment.

But I still think we are in like day one around is AI an indispensable part of most people’s work? And I think the answer is no for most of them. And so I think DeepSeek and all of our honest product staying power will come from who can get there and do that sustainably over time and have the right product design, the right integrations and the right deployment of that to actually succeed.

I think the two main criteria that I look at is, because our team, for all of Anthropic being big, I think we crossed a thousand people, our product team is maybe a tenth of that. It’s by Instagram year two standards, very large, but by large SaaS company, very small, we’re somewhere in between all of those. And we’re supporting like, now have Claude Code now, we have the API, we have Claude AI, we have Claude for Work. So it is across a lot of different surfaces. So I think generalizability is really important.

Even if we pick a persona or a vertical to go after, we are going to be building things that are general purpose as a rule with maybe some specialization at the user level, but not at the I don’t anticipate us building a lot of verticalized experiences that are fairly bespoke to a given workflow or use case. So I think that’s one I think

I think it does, except for the fact that I think that there’s a lot of valuable workflow knowledge that means that you can retain a differentiated product over time.

Yes. Yeah. I think the role of the great, we can help you translate this and from like a individual user, we’ll get you to pay $10 a monthly subscription. That feels iffy because I think that the models are quite good at that already. Right? And maybe you’re right. There’s not the like, if you play with eleven’s console and Workbench, a lot of the features that they’ve built are very clearly for people that are translating hours or voicing hours of content with a reliable voice across a whole work stream.

Descript is some of the best product design in AI. And they’ve clearly put so much time into the workflow. I had to use it once for a personal podcast. Was like, oh, this has clearly been built by people who are day in, day out sitting in this workflow and understanding it. So yeah, I think that maybe we’ve come to some sentences of our views, which is there’s value in the more professional use cases and the workflows that are unlocked by that. And I think on the consumer and maybe even prosumer side, it gets good enough from a basic AI product perspective.

You know, again, with the product focused lens, think I we have to pick our bets carefully. Even building, we built Claude code, which we just released as a sort of command line agentic coding tool internally first because we just wanted to accelerate our own team. And after seeing it play out for a couple months, we’re like, this is good. It’s not a solution to all the coding problems and doesn’t obviate the IDE, but it’s useful enough to us in enough cases that we want to see people use it out in the real world.

And shipping is never free, right? There’s like, you got to name it something externally. We got to find the right packaging around it. There’s a go to market piece. We do it carefully. I think my view of where the models are today is you still need hands on keyboard and you still need that exchange of, Hey, did this. Is this right? Well, let’s pursue this direction down. Yes, this is great. Let’s put up a blow request. No, we went down kind of like a false trail.

Let’s unwind the stack metaphorically and maybe an actual usage and then keep going. That’s why I think that there is a role for this sort of in between IDE and the full on Cognition, Devon, full on delegation of tasks, where they can be used for a certain category of tasks. Our product engineers love Claude code because a lot of product engineering is, we’ve to update the backend. We’ve got to create the frontend. We’ve got to submit these things for translation. We’re going to like, oh, this still doesn’t work.

Let me do this. And it’s that sort of build the product end to end workflow that does well with a thing that can work objectively across a lot of different I did two pull requests last week. I hadn’t coded since joining Anthropic, which made me sad. And so I got to finally use Claude Code. I have not opened our code base before, so I don’t really know how it’s even structured. But Claude Code is very good at finding the file that has the right piece and then going on and making edits.

Obviously not everybody’s in the same situation I’m in, but it is really valuable for those use cases. So when I think about the coding space and where we can play and add value, it really is on the agentic side. It’s not on the IDE side. There are other companies that are spending, like they wake up and go to bed every night thinking about how do we make a great IDE? And that involves things like low latency autocomplete that involves the right integrations, figuring out how you play with the Versus Code plug in ecosystem and all of that complexity.

There’s a bunch of work there that is valuable and different than what we’re doing. I think we can really play in, let’s be talking to these models and be doing real work with them in that agentic loop, but recognize that they’re not yet at the place where for many use cases, can let them run free for hours. You need that more human in the loop piece.

Yeah. I mean, I think it already looks starts to look different already. I was a huge early proponent of GitHub Copilot. I think my quote was on the homepage for a while. I don’t know if it still is because I saw the potential. Then even GPT-four came out before it, they had multimodal and I was trying to do Swift with it. I would draw ASCII art of the screens I was trying to build for Artifact and then go make coffee because it was, at that time, quite slow.

And come back and it had like an 80% version. Obviously, now it would be a 95% to 99% version of it, something like 3.7 Sonnet. I think the skills have become important. One, I think it becomes multi what am I looking for? Like multidisciplinary, where it’s knowing what to build as much as it is knowing what, like, the exact implementation that you want. I love that about our engineers. Like many, maybe even most of our good product ideas come from our engineers and comes from them prototyping.

And I think that’s like what the role ends up looking like for a lot of them. The second piece is code review really changes when all of a sudden you’re mostly evaluating AI generated code. I even experienced this. I put up a poll request and some of the comments that came back were, yeah, Claude code does this sometimes. We don’t actually use default arguments in this case. And I was like, oh, well, it. So it was sheepish. If I was coding it, I would have probably noticed those patterns a little bit better.

And so there’s kind of two sides that need to happen. One, models and just the infrastructure on models need to learn from code bases and code reviews better so that they can produce code that feels idiomatic to that company. But then also, how do we evolve from being mostly code writers to mostly delegators to the models and code reviewers? That’s what I think the work looks like three years from now. It’s coming up with the right ideas, doing the right user interaction design, figuring out how to delegate work correctly, and then figuring out how to review things at scale.

And that’s probably some combination of maybe a comeback of some static analysis or maybe AI driven analysis tools of what was actually produced. Like, is there security vulnerability? Is there some other flaw? Is there a bug? Computer use plays a part. So you can tell I’m very excited about this space. Automated testing of UI. So that what would be great is you delegate the task a year from now. Three years is crazy. Let’s even take a year from now. You delegate a task to it. When you come back to it, it says, I evaluated these three approaches.

I’ve tested them all out. I had a different agent actually try them out in a browser. This one is the one that worked best. I’ve run it through this additional agent that did a vulnerability test. It all looks good. All we need to do is help you resolve this one question. Let’s review this particular critical section of code to make sure it’s what you really wanted. That feels like you’re suddenly empowered to be more of a manager and delegator to these things rather than just a partner in the loop.

I think a lot about. I started the year by looking at our product development process and looking at where we are Claude ified, like where are we using Claude and where we’re not. And you look at it and say, Claude can be useful in sort of taking initial idea and creating a PRD out of it. And Claude can be useful, obviously, in the coding side. Claude can be useful in synthesizing a lot of conversations that people are having about a product and kind of finding the kind of thorny issues of disagreement.

Driving alignment and actually figuring out what to build is still the hardest part. That is actually the only thing that is still best resolved by just getting together in a room and talking through the pros and cons or going off and exploring it in Figma and coming back. And so like any dynamic system, if you optimize one piece, all of a sudden something else becomes the blocker or critical path. And alignment, deciding what to build, solving real user problems, and figuring out a cohesive product strategy, still very hard.

And probably the models are more than a year away from solving that. That is the constraint. It’s why I’m really bullish on at least startups being able to explore the space. Because I remember this from both Instagram and Artifact days. When it’s just a couple of you, alignment is a coffee conversation in an afternoon rather than steering the ship of a large company that has commitments to customers and all of those things. That’s still a very human problem that I think we’re at least three years away from the models being solving at that level of abstraction.

What we get out of each, and I think about that trade off. So I think we learn a lot more quickly with first party products. So as a specific example, with Claude Code, within a week of it being deployed internally, we had found a way in which one of the tools that it has access to, the model wasn’t using as well as it could have. And that made its way directly into 3.7 Sonnet. That’s a way in which internal dogfooding, first party tool, directly led to a model improvement in the next generation.

There’s like a few other places where we’ve hit that, even building first party products, much harder with a third party product. They’ll tell you if something’s wrong, but it’s a bit more of like that even though we work really closely, including with some of those coding startups that you mentioned, it’s still not the same. So there’s a lot of value in what we learned there. Then there’s the stickiness and we talked about brand and loyalty. I think it’s easier from a consumer if you can build a brand around a product than just an API.

The fact that we power a lot of this coding product is visible to people. It’s often the default in the dropdown selector. If you’re in the know, you know. But not everybody does. And it’s still not the thing that they downloaded, not the thing that they installed that they’re going to tell their folks about. But yeah, it’s also a place where we’ve gotten tremendous distribution. And we’re not going to invent every company. We’re not and this way, can kind of play this sort of it reminds me of my investing days where you get to see a lot more.

And there’s more than one shot on goal, it’s not all of those things. And so it’s been actually a fairly, from like a resource allocation perspective, fairly even split. I think we’ve, if anything, underinvested a bit in two things. One is just having a faster iteration speed on first party products. It’s like my current obsession. And then on the second part, the API side, how do we build abstractions beyond tokens in, tokens out? And every time we do that, we get great feedback from people.

So whether that’s helping the model plan and work agentically, whether it’s having the model build more knowledge graphs and repositories of how companies operate internally if you’re using the API to build more of an internal knowledge product, whether it’s perfecting tool use, whether it’s understanding very large bits of context and having memory that transcends conversations, those are problems that I think are worth us solving on the API because there are things that we can take what we learned on the training side and directly map it to the API and build good products around it.

So that’s how I think about those two. But it’s a new problem. On Instagram, it was easy. It was like 95% product, 5% API. And it was, you know, that’s all we really needed to do.

I think there’s two things. One recognizing that we were running, I think, a larger company playbook for what is actually like, we’re still on like start, our products are, even if the company has good traction and like the API business is doing really well, people are using Claude AI and upgrading Claude AI Pro, it’s still early days and it’s still like do or die or like make it or break it. So we need to operate in that way. And so getting the right people together sooner, faster, and ignoring organizational boundaries, we got too calcified, I think, in like, oh, well, this is on this team’s plate versus this team’s plate.

And oh, you can’t get this done this quarter because it’s not on this team’s list. I mean, I get why organizations evolve and some of that is natural, but we can’t afford that right now. So it’s been a lot more, we’re the right people, let’s get them together, let’s clear all the other distractions and then let’s clear out my calendar so that I spend more of my time in product review and design review than I do in administration.

The way I would put it is the adoption that we’ve gotten of our products is ahead of their actual, like, true product market fit because they are still the best ways of getting the models. And I don’t think that’s durable over time. So I think that’s, like, not a thing that to rest on. And two, I just think we’re under serving people because I don’t think we’ve gotten the right products yet. So it’s what I wake up stressed out about every morning or inspired by depending on the day.

It’s like, I think we’ve got we’ve got so much work to do on that side.

They’ve moved faster at shipping v ones even ahead of where the model is sometimes. What have they done worse than Yuan? Probably personality and having the features they build be cohesive. Which alternate

OpenAI. I think that they’ve balanced first party product development and and an API that, like, people people use at scale as well. But we had an Instagram principle that was do the simple thing first, and I think they often do the simple thing first.

I love this question.

It’s a

isn’t

It’s a really, really good one. I think the things that we built that were actually very valuable last year are now feeling like they’re having, this is a long answer rather than a quick fire, I’m sorry, have some cost to the information architecture, which sounds like a very nerdy way of describing it. But basically people should not have to think about projects versus artifacts versus chats and how they all relate. And I think tearing it all down and being like, what actually matters is do you have the right context into the right conversations?

Do you feel like you can always know where to go next in the product? And is Anthropic and Claude itself being a helpful sort of guide to what work is most important to do next is a different paradigm than like, I know to create a project. And then like, if you get good at that, it’s an amazing product, but there’s a lot of steps along the way. So that was that on the product side. I think that’s the fundamental thing.

On the stack, I mean, Claude AI and probably chatgpt.com were very much initially just built to be sort of showcases of the models and not really built in a lot of ways to be the foundational for a much more complex multi product sort of thing. And I think we have an active effort right now around tearing down some of that and rebuilding the core UX to just feel good. It doesn’t feel great right now. It feels a little bit like it’s been an evolution of a product that served a purpose at the time, but now is being asked to do way more things such that the incremental thing is now both harder to add and getting slow.

What have you changed your mind on in the last twelve months? How much first party stuff is important? Think I saw the growth in the API and I was like, is what we should just invest a lot more of our time in. I think that there’s you’ll you’ll miss out and not have enough of a durable moat if you’re not if you’re not equally investing, maybe even investing even more on the first party side of things. How much did it hurt you being late to that?

I think significantly, if you take a DeepSeek moment. Right? Like, ideally, the, like, the the the story of, oh, there’s more than one front or leading edge AI product to be used is some narrative that we should have captured. I think it hurt us there.

The models as they get more I’ll give you the headline, which is basically like discernment and privacy. So as the models get more capable, they’ll also become more knowledgeable. You’ll be in conversations with them about everything from something that might be quite intimate or something that’s quite sensitive from a company perspective, or they’ll have access to all of your particular companies’ things. And then everybody loves to talk about agent to agent interaction. The intersection of those two, not enough people think or talk about, I think, which is, do you trust your Mike agent or your Harry agent to be out in the world and then not be jailbreakable or reveal something that it knows that is quite personal or sensitive.

I think my metaphor is my five year old. It’s great watching her with somebody that she’s just met because she doesn’t quite differentiate between stuff that’s secret and private to our family and stuff that is like things that are okay to talk about with a new friend or somebody at the checkout aisle. So that discernment is something you acquire over time for people. And I think models, this is very underappreciated and probably under researched as well from a model capabilities perspective because models fundamentally want to be helpful, and that is not always what you want them to be.

And there’s a safety case for that, but then I think there’s also a privacy and data security case for it too.

I’ve had so many conversations with Alex Wang about this because he has this whole thing about how in the future most friends will be AI friends. And I don’t think he’s wrong. And I think that there’s ways in which that’s already starting to be the case with people having lots of online game experiences. Some of those are NPCs and you might just have more of a comfortable sort of existence in there as well, even if you’re not breaking through. So I worry. She is so gregarious that I’m not actually worried in her particular case, but let’s abstract to the broader sense.

There is a lot you can learn from what it feels like. Here’s the bull case. I was a fairly awkward teenager and I probably could have benefited from some practice mode, like AI interactions around some of these things to build it up. And at the same time, that’s not the real it doesn’t feel like it’s totally closing the loop around the consequences of real interaction. It’s the difference between reading about what it’s like to have your first really hard argument with your high school girlfriend and then actually having it.

And when you’re in that moment, this is now the classic, is it the Chinese room experiment where it’s like you’re it’s not the Chinese room experiment. It’s a different thought experiment where somebody is in a black and white room only reading about red and they see red. Is there something qualitative differently about that? Absolutely. And is there something different between talking to a model and engaging model, even in emotional role play, then having that same interaction with a real human? Absolutely. And so it is probably a helpful piece of future human interaction and absolutely insufficient as, like, the whole.

Europe? I want them to do well because I love a lot of Europe and I lived in Portugal growing up as well. I saw a funny, maybe somewhat defeatist argument where if real world experiences and human interaction become more valued, Europe becomes more valuable itself as the perhaps world capital of sensory and experiences. That feels weird if that’s all you’re resting on. That feels a little limited in there as well.

What I think will be really interesting from a Europe perspective or European perspective is what are the things, like, I think I really respect about Europe is there’s often been the case that there are things about the lifestyle or the society that they hold very, very strongly that then they, not always elegantly, but at least attempt to enshrine in either best practices or even laws. And so even as we think about doing our product design and data privacy and selling to German users or German companies, there’s a different set of questions that get asked that are often very helpful questions.

And so maybe the bull case there is that those are actually questions that are relevant to everybody, and they will just be at the leading edge of asking some of those questions. I think from a labs perspective, it’s a lot harder question to answer. I think there’s maybe some combination of access to compute. Maybe they move further up the value chain. And if it is the case that building applications on top of these models becomes it is a lot easier, and you can go from zero to one, and you can be more nimble than even these labs that are going to all have tens or hundreds of millions of users, you have to move slowly at that pace, Can innovation happen there?

Probably. But it probably involves a different both regulatory and startup ecosystem environment to really make that actually the case.

Yeah, I think the potential is huge. I think there’s everything from today where AI is helping, which is in closing the loop on drug discovery and closing the loop on clinical trials. Novo Nordisk used to take, I think it was something like fifteen weeks to do their clinical trial reports. Now they use Claude and get it done in twenty minutes. And that’s a step change. Now, there is years of research that preceded that. So I’m not saying that we’ve cut years to weeks or years to minutes, but that’s a point of the process that we can make faster.

That’s like with the models today. And then you see ARC, which is this science and research institute that Patrick Collison and some others have started and funded. They’re working on foundational models for cells, where you have all of a sudden a real cell model that you can run experiments on. And that kind of thing should also accelerate drug discovery and experimentation there tremendously because all of a sudden you’re cutting the loop there. So I’m very optimistic. There’s a lot of places where AI is, I think, underutilized relative to its potential.

And I think some of the smartest people in the field and the smartest minds of my generation were working on serving more targeted ads. Maybe that was true at one point. I think a lot of them today are working on how do you make models that are tremendously useful and valuable and intelligent across a lot of domains.

My pleasure. Really fun to do this.