Skip to content
20VCDec 20, 2023

Roundtable: Spotify, Adobe & Linkedin CPOs on How AI Changes The Future of Product

Why AI is Now the Product, How TikTok Changed Product, Why Cost is the Biggest Barrier to LLM Usage & Why Incumbents Can Adopt AI Faster Than Any Prior Innovation Cyc

With Gustav Söderström · Tomer Cohen · Scott Belsky · Harry Stebbings

Full transcript · 50 min · 10,942 words · 4 speakers

Cold open

Now AI is the product, and the UI is there to to help the AI.

Gustav Söderström0:00

When you think in an AI first principled kind of way, you’re really unleashing the idea of control, in my opinion. You don’t control the experience anymore.

Tomer Cohen0:03

Designers, they need to understand basically GPT four as well as they understand the user. Even though the models are there to generate great voice, the problem is actually doing it cost efficiently.

Gustav Söderström0:11

I can’t underestimate how much intelligence, even art, goes into building

Tomer Cohen0:20

the the right prompts. The big changes come when that technology enables

Gustav Söderström0:24

a new business model somehow. The functions of so many businesses are collapsing now. And so the idea of selling seats function by function, I mean, is such an old antiquated way of building a business to some degree.

Scott Belsky0:29

This is 20 VC

Harry Stebbings0:39

Intro

Harry Stebbings

with me, Harry Stebbings. And the roundtable shows that we’ve done so far have been the most popular that we do. And so today, we have three of the world’s best chief product officers discussing how AI changes the future of product and design. So in the lineup today, we have Gustav Soudersom, CPO, CTO, and Co President at Spotify. Tomer Cohen, CPO at LinkedIn. And then Scott Belsky, CPO at Adobe. We could not have a better lineup. This was so much fun to do. It’s a total masterclass.

Let me know what you think of this format of show on Twitter at Harry Stebbings. But before we dive into the show’s

· Sponsor read0 min · 147 words
Harry Stebbings1:14

date, HMC or Harvard Management Company, they’re constantly seeking out the next generation of truly great investors and entrepreneurs. HMC has managed Harvard University’s endowment for nearly fifty years and was one of the first institutional investors in venture capital. Their experience and long term investment horizon make them ideal partners to get world changing ideas on a path to viability and success. They work as a true partner, providing insightful perspectives to help managers succeed. I personally have had the pleasure of working with the HMC team and can say that they’re truly exceptional partners and just great, great people to work with.

Whether you’re launching your first fund or your fifth, HMC welcomes the opportunity to partner with both developing and established managers. Have an idea you want to share with the team? Just send it to venture@hmc.harvard.edu. You have now arrived at your destination.

Conversation

Harry Stebbings2:06

Alrighty. I am so excited for this. I wanted to do this one for a while. So we’re gonna do a couple of intros first so everyone can get familiar with each other’s voices. And so we’re gonna start with you Scott, then Gustav and then Tomer. Who are you and what are you most well known for?

Scott Belsky

I I was the founder of a company called Behance back in the day. We were acquired by Adobe. For the last six years or so, I I served as Chief Product Officer for about five years overseeing the Creative Cloud business and then have recently taken over strategy corporate development design and emerging products for the company in this kind of chief strategy emerging products officer type role over the last year.

Gustav Söderström

It’s

Harry Stebbings

quite an all encompassing role. Gustav, your turn.

Gustav Söderström

So I’m Gustav Saldostrator. And similar to Scott, I started and sold a few companies. And then I’ve worked at Spotify for almost fifteen years now as CPO and then CTO, and now recently as co president together with Alex Norstrom. So we run the company together. Tomer, your turn.

Tomer Cohen3:00

Everyone, chief product officer for LinkedIn, responsible for basically what we build from the company’s strategy to overseeing the teams building it. Been at LinkedIn for twelve years now. Joined right at the cuff of shifting from desktop to mobile as a company, so that was fun. And then I started my career as an engineer. So I was doing anything from semiconductors, chip on design algorithms to embedded systems, all the way to the Internet right now and AI.

Harry Stebbings

Okay. So now everyone knows individual voices. We’re actually also very lucky because they’re three very distinct voices. I didn’t realize quite how how apt the selection of this round table was in that perspective. I wanna start with probably kind of the most broad, but also kind of important, which is fundamentally we’re all told that AI changes everything that we do. When we apply that to product, what do we think are most significant ways that AI will change the product development process, specifically on the product development side?

What will change specifically with AI?

Scott Belsky

Well, I mean, I’m sure we’re all experimenting with GitHub Copilot and our engineering teams, and there will certainly be an improvement in both the productivity of developing code and products and also testing, you know, and and identifying and reconciling bugs and that sort of thing. But the only other thing I would throw in there is that when you’re developing the interface of a product, you are oftentimes trying a few different approaches to find the one that works best. And that just takes a lot of time to have three or four different options and, you know, and explore, but then, you know, stop going down that path and go down another path.

And then fast forward now into, you know, today into the future, we’re in a world where AI will suggest alternative scenarios. You can try variations and just look and, you know, with the mistake of the eye, find a better solution or at least something you wanna a b test. And so I think that the product development process, the products will become much better frankly. With fewer cycles, we’ll find better solutions. I have some thoughts

Gustav Söderström4:49

there. I mean, one way to maybe think about it holistically is that like, when I started and and Tomer just referenced this when when it was a transition to to mobile and even before then, at least I kind of thought of the UI as the product. And then machine learning came along and the machine learning was there to sort of help the UI. And that shifted completely. Now AI is the product, and the UI is there to to help the AI Mhmm. To capture better signal.

And I think, yeah, I think most people agree that the real transition there was TikTok, which is, like, almost no UI. It’s just the video, and the UI is just trying to make the explore exploit algorithm be more efficient than anything before it. And and, you know, most people in tech agree that it wasn’t really the algorithm of TikTok. It was the UI that maximized what was pretty traditional explore exploit algorithm, like, since the day so hot or not. So I think that’s the big change.

Like, the AI is the product, and the UI is there to help the the AI these days. And and maybe it always was. I mean, users never came to Spotify to click buttons. It was always about, you know, music or something. It’s just more clear now.

Harry Stebbings5:47

I’ve had so many guests on the show before say about the stupidity of UI and how AI will make UIs redundant. Is that wrong? Scott, you

Scott Belsky

go. I mean, UI is here to help us understand the object model of a like, you know, find what we want, you know, and and and with as little friction as possible engage. And so, you know, UI, in some cases, should disappear, but there’s new UI these days in the form of the tone of a product and the way that something is is the inflections that are used, the persona. I’m, like, thinking about UI design in the future as in some ways, like, persona design. When I’m interacting with with Spotify’s DJ and the DJ is like cracking jokes and like being casual with me, that’s a UI decision to some degree of a product that’s driven by AI as an example.

Tomer Cohen6:34

One of the biggest implications for me when building product is the understanding that AI is the product. But for many product builders, AI is something they delegate. I think that’s, like, really play delegated to the the AI team or the engineering team. I think AI strategy starts from the CEO and makes its way down. And with many companies right now, AI, that’s something that the team is doing. Reminds me of, like, early on with the mobile team, there was a mobile team building it, and it was a whole different the rest of the company was building something else.

Like, the analogy I give usually is, like, imagine, like, a re rafting a river rafting boat, the guide on the back with the two massive pedals, and that’s AI today in product. Then everybody else on the side, they add accuracy, they add some power, but they’re not as significant as the guy on the back with the pedals. And one of the biggest changes I’ve seen with AI is we moved from desktop to mobile, not everybody made it for the transition because it required you to unlearn how to build a little bit.

It wasn’t about hedging. I’m gonna put all these things there and what’s the kind of heat map. It was about making a decision about how do you build a relevance experience. When you think in an AI first principled kind of way, you’re really unleashing the idea of control, in my opinion. So what happens basically in AI is you don’t control the experience anymore. It’s not deterministic anymore. For many product leaders I know, it’s really hard to let go. It’s like you’re a chef at the restaurant and all you dictate is the ingredients, maybe the knobs a little bit, but you don’t control the output.

I can’t tell you how many people just flop on that. They just cannot comprehend the fact that they don’t control the experience. I like that. Massive formation of AI first mentality.

Gustav Söderström8:06

Probabilistic, yeah, experiences rather than deterministic.

Tomer Cohen

Right? 100%. It’s nondeterministic.

Harry Stebbings

Do you guys not find that inherently concerning? Every company on this call is a public company when you don’t have the control of the outcome and the outcome could be a hallucination. Is that not concerning

Scott Belsky

to you when you don’t control the outcomes? Well, it’s funny. I mean, for some practices, hallucination is a bug, but in some areas, it’s also a feature. If I’m trying to discover some cool new music, I mean, I can imagine that might be a feature as opposed to a bug. You know, when you’re trying to do generative fill in Photoshop and imagine what’s behind an object or, you know, extending the frame of a photo, hallucination is actually a feature. And if you get the wrong answer or something you don’t like, you can just run it again.

So I think it is a bigger problem though in applications that really, you know, are mission critical around, you know, writing NDAs, you know, for legal purposes or where a hallucination actually could get you in trouble. I wanna build on

Gustav Söderström9:04

something Scott said there because I think it’s the definition of what is UI changes. So you you took the example of the AI DJ we did, and you’re exactly right. It was the design team that user tested. What is this? Is this a utility like Google? Is it an AI person like Alexa? Or is the UI that we digitize the real person with a real personality that exists? And we chose the latter. That is design, and it is brand. So I think designers need to think holistically about the experience in that sense, and that that is still design.

It is the user experience. And I also think something that designers need to get good at in this world is to understand the the capabilities they have. They need to understand basically GPT four as well as they understand a user. What what what, what it needs and how it works. So an example that I think is a good example is mid journey. So mid journey, the designers, they clearly understood the performance of the model when they built the mid journey experience. Right? It took like two minutes to generate an image.

It was wrong one out of four times. They could have built a horrible experience. We waited for two two minutes and got disappointed 75% of the time. Instead, because the designers there and product people understood the capabilities, they built a fault tolerant UI. They gave you because they also understood how diffusion works, so you can do it in stages, they gave you four low res shots at the same time in twenty seconds. Is that is any of these good enough? Right? So that’s what I think design needs to do.

Designers and product people need to understand the models and the performance very, very deeply so that they can make sure the experience matches the current level of performance. It’s the same for us. If we have a one in 10 shot in recommending a good song, we probably need to recommend 10 on the screen at the same time to get like a good chance of of a hit. Right?

Harry Stebbings10:41

Gustav, you said there about kind of understanding the models very deeply, but again, I am basically an absorber of knowledge from smart people, and then I try and kind of amalgamate it together. Everyone tells me that actually every company or the best companies will use multiple models at the same time and transition between them. How will product people be able to have eight different models in use at the same time, know all of them really well to utilize them effectively? How does that look?

Tomer Cohen11:06

I don’t think it’s a product people. I think this is where you build like a platform that basically enables you to understand the task you’re trying to accomplish. Cost, we should talk about cost at one point because this is a very costly software and costly technology. And then that’s actually what you’re being mass people with. Deciding what kind of model you use for what purpose really starts at the application layer, but the decision is not made at the application layer.

That’s actually if I think of, like, why I would expect to see some massive innovation and startups to show up, it’s in this tier to really allow people to leverage multiple models at multiple cost centers and resources and efficiencies, and then completely mask it from the developer, from the designers, the product folks.

Scott Belsky

I think it’s actually common belief that there are gonna be a few mega models in the future that are gonna do everything for every company in the cloud. And I think what we’re saying is actually, it will probably be the opposite. Right? There will be many, many, many long tail models, some learning locally on people’s machines or in applications that they install, some in the cloud, some open source, some not open source. And there needs to be, like, logic around the routing, you know, not only to the model that’s the best prepared or, you know, the most specialized model for a given query, but also the most cost efficient.

And a lot of companies like ours, you know, in market with AI products today are actually, you know, counting on the reduced cost over time, you know, from some of those technologies. So I think that’s a great point and I actually you know, there are some startups now that are kinda defining themselves as like router, you know, startups. There

Tomer Cohen12:37

there’s like two levels I think which are kinda fun to think about. One is at the application layer and one is at the kind of more of the kind of mid tier layer, which is like the dispatcher analogy. And there is a dispatcher at the kind of application layer from the application. Like imagine the idea of agents, right? You come in, we all know by now building one agent to rule them all is not the right design. You wanna have multiple types of agents. Like, I take a a complication like LinkedIn, there’s like a seller agent, a knowledge agent, a job seeker.

There’s so many agents you wanna build, but you wanna build as a dispatcher around that that know almost like a a team coach that knows, like, how to get that team to work together. Same at the mid tier level, when you wanna start building some kind of dispatcher or router to Scott’s point around which technology to best use. That’s where I I could imagine tremendous innovation happening.

Harry Stebbings13:23

When we think about the multiple models that we’ll use, you mentioned there that we’ll have actually kind of open, close, very specialized. When we think about like cost, the more models we have, the more costly it is also, and margins are impacted. How do we feel about cost and implementation when thinking about model adoption?

Gustav Söderström

From Spotify’s point of view, most of the stuff we built in house, to Tomer’s point, is actually mostly about cost. For example, as we talked about generating a lot of voice, we wanna generate like two minutes of voice per day for half a billion people. It’s a billion minutes that can ruin you. Like, cost is actually already, like, the biggest factor in if you can deliver products. Even though the models are there to generate great voice, the problem is actually doing it cost efficiently. So I think that’s completely right.

A lot of the innovation and and technology has to go towards routing often actually for cost purposes. I do think I have a little bit of a different view on there is some chance that per company, I think you will want to embed the entire user history in one model. Alright? So today, most companies, including Spotify, many separate systems that are optimized for different purposes, and they sort of collaborate in semi predictable ways. There’s not one system.

But if you look at some of the papers coming out of Amazon and so forth, they are starting to look at you literally tokenize all of it, basically, the log, the how you scroll, how you click, what you listen to, what was in the content that you listen to. You just do a token prediction based on that. Based on all of this, what is going to happen next? So we may still see, like, one large model. I think the world is going to go towards you embed your entire user history, everything they did into one space.

I still think to to your point, Scott and Tomer, you will have specialized models for, for example, for voice, for video, for for different purposes. So it depends on if you’re talking about the user data or about generic capabilities like rendering and voice and software.

Scott Belsky15:07

Yeah. And in some cases, I guess you could take the user data at at the router level and kind of, you know, go to different models for different purposes. Exactly. You can embed that. Right. You know, as a user history. I think Gustav’s point about the cost efficiency. The good news is that the desire to have more performative models and the desire to have more cost efficient models are often like parallel efforts, you know, towards the same outcome. And so, you know, the margins will get better as these models get better also.

Harry Stebbings

Do you think this follows standard Moore’s Law theory in terms of development of technology and the cost reduction that we see there?

Gustav Söderström

I think it will. You know, some people say Moore’s Law is is starting to come to an end in terms of just more transistors, but we barely started on neural hardware. So I’m sure we’re going to see the same effect even if it’s not more transistors per square inch necessarily for a good while. I also think we’ll start we will continue to see bigger and bigger models, but there is the opposite pressure as well. Every month, there’s a much smaller model that did what the bigger model did a month ago better.

So both things are happening at the same time. So I do think we’ll see that progress for some time.

Tomer Cohen16:09

Now coming from embedded systems, you can already see verticalization of software with chips dedicated where, you know, Microsoft announced they’re gonna start building this. Can actually see, like, the idea of how do I get that efficiency and gains in resources and power and cost. You know, when I was building semiconductors a long time ago, it was like every chip was about how do I get, you know, in half the power, double the capacity, and, like, lower the cost. So that was always the kind of mindset.

And I think you can start start seeing that verticalization. This goes to open versus closed. There’s so many implications of that. But if you’re cloning to that closed, highly resourced, optimized system, you’ll start seeing that from companies and Apple and Microsoft and so on.

Harry Stebbings

Each of you have incredible amounts of data. You know, LinkedIn, Adobe, Spotify, insane amounts of user data. Everyone puts the question forward of what comes first, the size of the model or the quality and size of the data. How do you think about that and the importance of data versus model?

Tomer Cohen17:03

I I think the size of the model matters, but it really depends on what you’re trying to do. If you wanna have this, like, amazing kind of personal assistant like in the movie Her, then you wanna build a massive model. By the way, the model size is really the number of parameters you have. That’s really pretty much it. But on the flip side, and this is where this is where, like, it depends on what you’re trying to solve, and the volume of your training data also matters.

If you have a large model that is undertrained, it will underperform a small model which is really well trained. So you’re just wasting resources, and you’re gonna get, like, less efficient, results. And then there’s already examples right now that, like, when you build specialized model for media knowledge, it’s like an athlete that you transform into a weight lifter or, like, a long distance runner. They perform much better. I can’t talk on behalf of Scott and Gustav, but if we’re trying to build if I’m trying to build a job seeker coach guide, I’d be better build that agent specifically for that role versus some kind of like lifelong coach that will be very hard for me to build.

Those are those are before much, much better.

Harry Stebbings18:02

What’s the hardest thing about model implementation? When you look at current product and current tech stack, when thinking through model selection and then model implementation, what’s the hardest thing?

Scott Belsky

The little secret is that there’s a lot of final mile tuning and work that probably we’re all doing under the hood. That is work that only we could do because we know our customer really well. We know our product and experience really well. And, you know, it’s those little finesse moments. I mean, it’s the same playbook all along. Right? What makes a customer love using a product? Because they the product is empathetic to their problems and if the interface, you know, meets them where they are.

And, you know, I think we get lost sometimes in the technology and forget that it’s not the technology that makes us successful. It’s the user’s experience of the technology that makes us successful. And that’s why I think the role of designers is more important, not less important in this modern world. And, you know, I I’ve been thinking a lot about the pace of change that we’re all dealing with these days, you know, in our respective companies. I’m sure, like me, you’re waking up and every morning be like, oh my gosh.

What new breakthrough do I have to figure out today, you know, in terms of how it how it impacts our business? You know, and in this, like, newsletter exercise I force myself to do every month called implications, like, I’m calling it surfing waves in the Cambrian explosion. This notion of like, you pick a wave and then you’re like, oh, wait, I’m on the wrong wave. Like, how do you transition to the next that wave over there, but then wave this wave over there. And it’s almost like this frenetic emotion that we’re all in.

And the only, like, solution I can come to is just, like, doubling down on empathy with the customer. There’s so many new technologies and possibilities being thrown at us, but if we just kinda, like, take the pulse of where the customer is and where they’re likely going even more so, like, maybe we can help make the right decisions. Otherwise, it’s, like, wild.

Gustav Söderström19:47

Yeah. I I agree with with Scott. And I I think on your question on what was the hardest thing, that depends, as Tomer said, sort of like, what are you asking about and and when? Initially, it was very hard to find talent on the technology side. That’s now getting easier. So And the problem keeps moving up the stack. And as we talked about, I actually think the hardest thing for us now has been to sort of retool the company to think about what the model needs to serve the user, to rethink design, to rethink product.

For a while, there was a lot of work around making sure data was useful. So it keeps changing what the problem is.

Harry Stebbings20:19

What does retooling the company mean, Gustav? Is it like, again, this is kind of my.

Gustav Söderström

It’s it’s what I spoke about. We we have a bets board where we kind of stack rank what we’re doing. We literally had a bet called AI is the product for, like, two years to really emphasize the shift. And so getting people in that mindset, educating them to understand how this works, to start to use these things and get empathy, not just for users, but for the models. Because people first underestimate the technology, and then they’re excited, they overestimate what it can do. So they they create products that are not mature yet.

And so there’s a lot of work in getting people realistic about where these things are right now. That was probably the hardest thing to get everyone sort of on the same level of knowledge and understanding.

Tomer Cohen21:01

Yeah. One wait. Wait. One fun one, and I’m curious if Scott and Gustav are doing this as well. But I remember when we shifted from desktop to mobile, we had literally had to force people to bring mobile designs. Because, again, they were like, we had to force them. Literally said, like, there’s not gonna be a jam session unless you bring a mobile design. Now I now I wanna see your prompt. Yeah. Because ultimately, that that’s the interface that people are using to generate the results. I I wanna want to learn how you’re building your prompt, and I wanna critique your prompt.

I wanna understand your prompt really, really well. I I can’t underestimate how much intelligence and even art goes into building the the right prompts. For sure. I never imagined doing product gems and asking to see people’s prompts going into the into the session, and now we’re doing this on a regular basis.

Gustav Söderström

I think this is, like, a very democratic thing because it used to be if you were, like, a product person or a designer and you had an idea, you kinda needed to convince engineering to build a prototype. In this space, what’s happening more and more is like the designer or the product person come with the prompt and the idea ready. It just generates something. And then you can you know, that’s very empowering,

Scott Belsky22:00

I think. It’s so true. And, you know, we’ve all been talking about design driven, you know, innovation and product and whatever for a decade or more. But I I think this point about how you have these core teams that are building APIs that are ultimately representing the capabilities of the model, and you can actually directly empower designers to start to explore interfaces and and, you know, and to the point about the intonations and the persona and all those other decisions that these designers are now making.

They thought they went to school for graphic design or product design, but now they’re actually, you know, thinking about the conversation inflections and all these other nuances of design. But that’s what we did too. You know, we had our design team actually build a team within that did all of the early development of Firefly. And it was a very, like, design driven exercise to figure out the interfaces and how they integrated into the products.

Gustav Söderström

I wanted to just quickly go back to one question you asked there about sort of size of models and and data. Because I think it’s interesting to sort of think about. My bet is that size of models will keep being very important for some time. Actually, now, you know, the chinchilla paper, it’s very predictable how it’s going to scale. It’s unclear why it wouldn’t scale even if it’s slightly diminishing for a while more. We’re not even at the level of the brain yet in terms of connections.

So that seems likely. My hunch would be though that in the longer term, you’re going to be able to do most of what you want with reasonable sized models. And like having a much bigger model doesn’t have that much. So I would still bet that having lots of lots of user data and lots of high fidelity user data, basically great user understanding, is going to be important in the long term. To to your question of like, does does data really matter? One argument is it doesn’t matter.

It’s all going to be in the model. The other argument is like, no, the models will be very powerful. To be able to ask them good questions, you need a lot of data about that user. And that’s kind of the bet that I would make. There was this paper called MedPrompt that came out recently where GPT-four, like a general model, if you prompt it the right way, it actually beats a fine tuned model that was fine tuned on medical data through this very, very clever prompting.

But it sort of still proves that in order to do this prompt, you needed to embed a lot of user data. So I still think long term, data is going to matter to have proprietary or a great user understanding of of the specific user.

Harry Stebbings24:14

Why doesn’t Spotify create its own models, Gustav?

Gustav Söderström

So we have our own models as well. We do both. We have lots of our own models, but we don’t have a GPT four that we haven’t told the world about. I’m sorry to reveal that. But it’s also what I said before, like our goals are different. OpenAI’s stated goal is AGI. That will drive different incentives. It will not optimize for delivering two minutes of audio to half a billion users per day, right? It’s not necessarily gonna optimize for for cost efficiency and building pragmatic products.

That’s why, you know, there’s no point in trying to compete with them for us because we don’t even have the same goal. So we’re trying to build exactly these things that we don’t think they will build, and we would buy the things that they will build or someone else will build. Obviously, we’re working with with Google and GCP, so that’s where we do most of our work.

Harry Stebbings

Guys, how do you think about the decision to build your own models? I’m just intrigued on, mate. You know, I asked Scott at Atlassian that question, and he was like, no. It’s not our core. We don’t need to.

Scott Belsky25:05

Actually, I think that’s that’s sort of the right answer is you should build models if you’re the best company in the world to build them. And so, you know, for us, it was like, who’s gonna build a better imaging model? We’ve got the advantage in terms of understanding the customer’s use cases. We have the data. No one should build a better imaging model than Adobe. And so we’re like, okay. We need to set out and make sure we hold ourselves to that level. But when it comes to, like, building a massive LLM, to Gustav’s point, like, let’s partner with the LLMs of the world that do this, you know, as a business.

And and I think it’s as simple as that. So when we’re looking through all the different vectors, you know, we wanna build the models that we believe our competitive advantage is to is to be able to build and be best at, and then those others we should partner with.

Tomer Cohen

There’s a junction there, at least for us, given just the amounts of data and nodes and the graph, is that what we’re building is an integration model with the models to be really powerful. So today, many companies will just push all the information to the prompt. More of, like, I would say, like, a interim solution until you have your own integration with the model on the first serve. So imagine the the prompt already knows to reach the dispatcher or the router, and that will basically grab the both of the there will be conflation at the tier level that the prompt can also be masked from.

And I think that’s actually really powerful. I don’t know if I wanna compete with the, you know, the next dev developer conference from OpenAI despite us being some, you know, siblings to an extent from a Microsoft standpoint. That’s that’s what they’ll do best. But what we do best is understanding how it works for LinkedIn. But Gustav does amazingly well is understanding the idea of of audio and and media. What Scott will do really well is the idea of design and and visuals. Building that asset for me is a massive differentiation.

Competing for which I think would be also super hard to compete, and you’re competing for technology. I don’t think you’re competing for really solving problems. I don’t think it’s the right solution. I I I don’t see anybody winning there, really.

Harry Stebbings26:56

Does it change how you do anything with regards to data collection, data treatment, data cleansing? I

Tomer Cohen27:02

I I love that.

Harry Stebbings

Okay. This is

Tomer Cohen

I have to take this one. Okay? I’m gonna start. Oh. He was keen for that one. Because, like, this is this is a big one for me. It’s been a pet peeve of mine for a long time. So when I was I’ve been preaching AI first mentality for a long time and really trying to push both design and product folks and even engineers to think about data collection and quality data for a long time. And, again, this has been like I thought I’ve been like it’s almost like somebody else will do it.

Data science will figure out how to just collect it from the signals. And, like, you don’t understand. This is this is the literally, this is oxygen. This is what feeds. If AI is at the center, what basically brings it to life is data. I remember, like, early on at LinkedIn, I used to literally, like, spend so much time filtering by myself data so I can basically say, this is good food. This is good diet. This is what you’d eat for, this is what the algorithm should should eat.

But somehow people gravitate towards the UI or they gravitate towards the specific individual elements of it, not understanding that what actually feeds the algorithm is the data. My opinion as a product leader, whether you’re a designer or an engineer or a product manager, data is literally at according to your second most important job in your role, it’s understanding how do you how do you basically start to infuse all of data collection and make sure it’s high quality. The first more important job is understanding the objective of the algorithm, which most people also.

People outsource this stuff. It drives me insane. They outsource what is the objective of the algorithm to do. They kind of create some kind of spec. Now nowhere in the spec is it ever written. What is the algorithm supposed to do in a nuance way? And they outsource this stuff. But data is a massive, massive pet peeve of mine because most people just disregard it, or they just assume it will be there. Somebody’s gonna create it and collect it.

Scott Belsky28:40

I mean, we we just always have to be very careful because we’re so focused on for the purposes of generating content, the commercially viable and safe nature of the model matters a lot. We’re always trying to learn how customers use our tools, but we don’t ever train our models off of the customer’s creations. So, like, what the customer makes unless they submit it to stock and wanted to train our models. You know, it’s interesting, like, when we first launched our models, like, there were there were some people who went in and, you know, typed in Spider Man doing something and said, oh my god, this sucks.

Like, you know, Firefly doesn’t even know who Spider Man is and we’re we had to explain, like, that’s actually that’s a feature. You know, it shows that we didn’t train on copyrighted content, and you can trust the way that these models are trained as opposed to some other models. So the the data policies matter matter a lot. But one thing I should say, you know, Adobe has always been two businesses, really. It’s like the digital media business and the digital experience business, which is a set of tools that big companies use for their marketing.

And whether you go to Nike or any of these other brands, like, lot of them will use our tools to manage their customer experiences at scale. And I’ve always felt like in some ways, Adobe was two different companies sometimes. And in the last year, suddenly, makes so much sense because the digital experiences you’re delivering, like the marketing, the content, you know, your experience on the Nike website or whatever can be informed by the company’s relationship with you and the sizes of shoes that you’ve purchased before and, you know, and where you live and things like that.

So for marketing purposes, you know, companies are gonna leverage their own data to dramatically personalize digital experiences in ways we can’t even fathom right now. That’s really cool, you know, for the future of digital. From our point

Gustav Söderström30:18

of view, we have the same journey that Tomer had about data. We did realize the value of data early on with all the playlist we had, which was the basis for our first sort of simple embeddings. For us, obviously, the usage data, how you playlist, how you listen is very important. We had that journey as well. I think the best example is actually Scott and Firefly, where you’re solving an IP problem. You’re also solving a technical problem of I want painting and so forth, But you’re also solving a business problem by the licenses and the rights you have to the data.

That’s a very important angle that you may forget. You can solve many problems for a consumer or a user. In general, we’ve seen the first wave of AI now where the technology is very impressive. It’s starting to do useful things. But we still usually, the big changes come when that technology enables a new business model somehow, and that hasn’t really happened yet. I’m absolutely certain it’s going to happen, but so far, it’s like mostly sustaining innovation. What do you think that business model looks like, Gustav?

I don’t know yet. I think I think one example is is Firefly who is trying to solve a business problem as well. Most of these things are not yet trying to think about if business models can change completely, and and they probably could. But I think before that happens, this will be sustaining innovation that largely accrues to bigger players. Usually, when the business model is challenged and someone counter positions like a big business model, that’s when things really change. And that hasn’t really happened yet.

Harry Stebbings31:40

You said innovation and the value accrue into bigger players there. I think that that’s kind of what everyone’s asking, which is like, who wins? Is it incumbents or startups? And all startups say, well, you know, the incumbents, they can’t move very fast. And I mean, respectfully, all three of you move really fast. My question is, like, is it kind of old BS that incumbents can’t move fast? How have you guys been able to move so fast to embrace it? Scott, you talked about kind of the Cambrian explosions or Cambrian waves and jumping to, How do you move so fast at scale, and is it BS that incumbents can’t?

Scott Belsky32:12

Well, here’s the thing. I mean, every platform shift is not the same. They’re all different. And if you’re going from on premise software to delivering in the cloud, That is a major, major transition that I can imagine many incumbents would struggle to do in a way as fast as a startup could do. And maybe that’s why startups totally led the pack in terms of cloud alternatives to on premise software in the past. And then you mobile was probably the same sort of thing. Different stack of developers, totally different go to market in some cases, different relationships with Apple app stores, and like, you know, there were so many differences.

But with AI, I’m not actually sure that as a platform shift, which it certainly is, there are as many differences in how a company facilitates, you know, and and and leverages that platform shift. Because, you know, when someone comes into Photoshop and they have an image that they wanna change, right, if I can just give them a default bar that just shows up where they can just articulate what they want instead of using all the knobs and bells and whistles of Photoshop that have accumulated over the last forty years, right, that is like a very incredible way of disrupting Photoshop to some degree.

I don’t have to build a net new company and a net new product from zero to one in order to reimagine the capabilities of Photoshop using AI in this instance. Right? Now people might not wanna even download Photoshop, in which case that’s why we brought these capabilities to Firefly as a standalone app as well. On behalf of Adobe, I’d love to take the credit of, like, moving super fast, you know, and I do think we executed well, but I also think that there are some nuances of this platform shift in particular that do favor those companies that know their customer well and already reached them.

It’s always easier

Gustav Söderström33:54

to analyze, other people’s companies. I think, like, the risk for Adobe was before they moved the subscription model. Like, the subscription model works with this as well, So that’s why it’s not disruptive. I think what has to happen somehow, someone has to figure out a business model that is disruptive to someone. Some people think that it will be disruptive to to Google’s business model, you know, that the ads model won’t hold up in a conversational world. Not so sure that’s actually what’s gonna happen, but something like that would need to happen, I think.

For Spotify, for example, when we started, it was the business model of streaming that was the disruptive thing. The technology was a bit of peer to peer and stuff, but it was a business model that took Apple eight years of collateral damage to follow. That’s kind of what needs to happen, I think, and we haven’t really seen that yet. It could also be that this is just more productivity for the existing players.

Tomer Cohen34:38

I I appreciate the push on the business model because I I don’t think we often talk about it. But you can play the the regular, like, SaaS model based on seats. And if your technology really makes the organization much more successful, then your seats volume model does not work anymore. Yeah. That’s a good point. Is it really just price that you increase? Becomes like a really big conversation to have around what’s the business model when your product makes the organization more efficient. You’re cannibalizing your own seats model.

And

Harry Stebbings35:04

there’s a there’s a brilliant piece by Sarah Campbell at Benchmark, actually, which just talks about selling the work and not the seats.

Tomer Cohen

Exactly. You actually still need to sell because the idea of, like, you’re hiring we talk about hiring, and we immediately imagine hiring people, but you’re really hiring task completion.

Scott Belsky

I I love this this question about what business models will be disrupted by AI is something we probably don’t talk about enough. I mean, think about how many people in the world get paid on an hourly basis for what can be achieved in an hour. I mean, even lawyers charge by the hour still. Designers charge by the hour. And now you see these new technologies that are readily available. It’s like, how does it even work? You know, what you’re really paying for, of course, right, is like the the depth of experience and judgment that one applies to their work.

You know, your lawyer’s ability to know based on their, like, you know, thirty years in practice, like, what’s likely to happen to advise you in the right decision? If that’s made in three minutes, like, how do you how do you compensate them? And I think the same goes for the seats part as well. The functions of so many businesses are collapsing now. And so the idea of selling seats function by function, like how many people in procurement? How many people in financial planning? I mean I mean, it’s such an old antiquated way of building a business to some degree in the age of AI.

So I’m excited about this new value based innovation vector for business model.

Harry Stebbings36:19

Is that challenging for you buying software then? When you think about buying new tools today, does it change how you think about buying tools? We’re all huge buyers of tools internally as well. Like, you say antiquated, Scott. I mean it nicely, but I’m sure, like, we all buy a huge amount of seat based tools.

Scott Belsky

Well, we do. And at at some point, probably sooner than later. Again, we’re early in the days of AI, but at some point, when there are more tools in the organization that do many different things, you know, I think that the idea of buying seats function by function will just evolve. You know, I I but you’re it’s a good question, Harry. Like, when is that really gonna be material and how is it gonna manifest? Yeah.

Tomer Cohen

Harry, I think I think I think your next podcast is with a bunch of CFOs and how they’re making the because this is actually this is where everybody’s

Harry Stebbings37:04

Come on. Give me a break. Again, like Yeah.

Tomer Cohen

This is gonna be the next productivity layer. And I think, like, for many companies I I don’t think coming and saying, I wanna deploy this pro you know, this program over 20,000 employees is gonna work anymore. People are gonna try and understand, like, wait. Wait, what’s more productive? How much output are we gonna get? And what are those are phenomenal questions we never used to ask a year and a half ago. That’s like an innovation catalyst.

Harry Stebbings

Again, speak to many people. Everyone says that AI development progression technology, stellar. 10 out of 10. We’re doing a great job. Enterprise adoption, about a negative four out of 10. No good. Like, you know, I think as a stat, 32% of European corporates do not know what Slack is. That’s concerning. How do you think about enterprise adoption keeping pace in any way with AI development?

Scott Belsky

I think it’s probably good news that we’re so early in the adoption curve because that means that there is a lot of potential ahead for these companies. And I also it doesn’t happen linearly. I think that the adoption happens in moments where there’s like a step function, you know, breakthrough or people just suddenly realize that I mean, a lot of enterprise selling also happens because of who else is buying it. When I’m trying to work with customers who are thinking about are we real are we ready to change the way we do this?

It’s much easier when they hear about that company that has already changed the way they do this and how, you know, they’re saving money and and they’re and they’re producing better output. It’s like, oh, okay. But, you know, I think that the part of part of this is always about getting small teams in big companies to start to play with something. You know? And that’s where I think enterprise sales can fall flat sometimes, and you have to have design partnerships with early customers as opposed to traditional sales, you know, to go into early customers and say, hey.

I’m not even trying to sell you on this. I just want a small team that can start to play with some of our technology. We have this tune your own model capabilities now for Firefly where brands can use some of their own IP and make a version of the model for them. And we’ve been going to some specific companies and saying, hey, be our partner and let’s play with it. And then, of course, like, that play becomes utility and then that utility becomes a reference point and eventually, you know, when we’re ready to sell it more broadly.

So I agree with Scott both

Gustav Söderström39:11

that it is early. Yeah. Maybe it’s our job to start using these things ourselves internally first and get them to work. But I also agree with Scott that I think it’s gonna go much faster than moving to cloud. I know we actually were on prem companies that old. It wasn’t possible to just also be on the cloud and try a little bit. It’s incredibly hard. I don’t think this will be that hard. I think you can try these things in parallel. It’s gonna be an s curve, but I think it’s gonna be a sharper s curve once it happens.

Harry Stebbings

We’ve spoken about kind of the seismic change needed in product and designer minds. If you were to sit down with young designers, young product people, and we have like 700,000 that listen to 20 products, What would you say to them in terms of what they can, should do to equip themselves for the changes that we’re seeing so they’re best placed in their careers?

Tomer Cohen

Again, I’ll start with a quick just a stat from this. We’re currently tracking very closely at LinkedIn. They’re just looking at talent overall for the world. Like, the job is changing on you whether you like it or not. So, like, I think this one mindset for us, I think everybody here on the call and for every generation to come, like, your job is gonna change much faster than you think, but it might even change titles. The tasks will change. So it’s really about how you morph for that.

We looked at, like, the last five years. Obviously, AI has been been playing a big role, but the skill set necessary to do a job changed by 25% in the last five to six years. By 2030, they’re gonna change by at least 65%. Whether you want it or not, your job is changing. So the necessity to learn and to really start shaping your understanding of the task you’re trying to do, how the task is morphing is critical. For me, I would put it under the umbrella of growth mindset, just the ability to learn.

Then there becomes like more of a question around how should I think about beyond being AI literate and beyond being focused on an AI first mindset, what should I do? I think there’s two aspects that come to mind to me. One is like a more of a T shaped perspective where I would assume you should have more broad skills than you had before. The ability to activate or to that angle, I would still build expertise around a specific domain or specialty. We talked about disruption before between incumbents and startups.

Usually, talk about tech. There’s so many industries yet to be disrupted that nobody talks about, but those could be remarkable areas to focus on. And lastly, I think it’s innately what makes us human. Things like soft skills, interpersonal dynamics, and imagination, like just focusing on those, those are not going anywhere.

Harry Stebbings41:34

Guys, hit me. Go granular. What do I actually do? My job’s changing.

Scott Belsky

Treating yourself as a business is something also everyone should always do. It’s, you know, how you organize your your ideas and information, you know, how you track your own spreadsheet for your own budget. Like, we’re all we all have our own business called ourselves that we we also manage part of being a human being. So embrace these tools. A lot of folks used, you know, Google apps for themselves before they introduced them to their companies. You know, a lot of us used Evernote or Notion or other things, and then we said, wait, I wanna use this with my team.

Have some embracing. Novelty precedes utility. Playing with something gives you ideas of how you could maybe pilot it with something you do in your day job. And then if you have a pilot, that’s how you learn as to whether or not it’s, you know, relevant for actual practice. So that’s how we stay on the edge. And it’s interesting, like, we all know people who are those early adopters who love playing with new tools. And and then we all know people who are like the total pragmatists who are just like, I don’t use it until I’m told I have to change my tool and you can rip this old tool out of my hands, you know, at the last minute that it’s supported within the company.

If you’re listening and you’re early in your career, you can be the person on your team that introduces new practices. That’s your advantage in a company, especially working with a lot of people that have been around a lot longer, is you can be the one who who pioneers new practices. And I think in

Gustav Söderström42:59

general, like, the one way to think about big shifts is that’s actually when the opportunity arises. To Scott’s point, a lot of people are slow to change. So the it usually is the case that one of the advantages of coming straight out of school is what the disadvantage is you have new experience, but the advantage is usually you have the latest and greatest knowledge. Or someone who worked in a company for ten years, they’re using the old tools. So I think that’s still true. You can have that advantage.

But even if you’re in a company, the only advice is to educate yourself. And as we said before, in this instance, it’s actually simpler than ever. You don’t have to ask for permission to learn how to prompt and how to use these things. You know, education is on the Internet. It’s basically free now. It’s very cheap. I I think it’s easier than ever, more democratic than ever. To Tomer’s point as well, it’s just gonna keep changing. There there’s actually the risk is if you’re in a company that you don’t develop fast enough.

I think if you’re if you’re fresh, you’re gonna have those latest skills.

Harry Stebbings43:50

Listen, I wanna dive into a quick fire. So I say a short statement, you give me your immediate thoughts. Scott, I like your newsletter. What’s your biggest lesson from running applied to product?

Scott Belsky

Oh, from running. There are a lot of lessons that I have extracted from running. I’ll tell you one of them, which is when I’m running, I often have ideas. And I’m obsessed with capturing ideas and I’m always worried about forgetting ideas. But when you’re running, I just like force myself to keep running because I don’t wanna give myself an excuse to stop running to capture an idea. And that period that I am forcing myself to keep thinking about this idea as opposed to writing it down, it always gets better.

I think about that now in my, like, everyday work. I’m so impulsive. I’m always, like, trying to be decisive. I feel like that’s one of the best practices of being an executive is decisiveness. But if you can, like, sit with something a little longer than comfortable, oftentimes, you end up with a better approach to how to say it, you know, to how to give the feedback to the person, you know, how to, like, solve that problem with the product. I have this, like, mantra of wait for it that I’ve learned from running.

Gustav Söderström44:51

The cadence of running actually, I think, is almost like a process. You’re forced to iterate the idea. It’s it’s almost like in a, you know, in a trance. I I totally agree. Having to for us to be thinking of something, you have this clock going tick tick tick for an hour. It

Harry Stebbings45:03

certainly helps the idea. Tomer, if you could change one thing about the LinkedIn product today, what would you change?

Tomer Cohen

What’s interesting right now, we talked about the the idea of UI and complexity. There’s this great law that I’m I I really like. It was really helpful early in my career to think about how to build products, which is the conservation of complexity, the idea that every product has an inherent amount of complexity. Idea is can you solve it in the product development side, or do you actually put it on the user to solve by themselves?

And I think we’re in this phase right now where there’s so many use cases and audiences that people come to the LinkedIn main app for, and that just had inherent complexity because there was so many you know, you come in the morning, you’re trying to see what’s happening in the world. Later in the afternoon, you interview somebody, you wanna check out their profile, then, you know, your boss, upset at you later in the day, and you wanna see what’s out there for you. This is all three use cases in the same, like, twenty four hours.

Now with AI, and I would claim probably this would be more generalized the more complex your product, the more impact AI could have in terms of simplifying it for your user base. So one thing that were already underway for us is truly take away the complexity and solve it with the idea of bringing in more of this new large language models, and in general, this new brain that can like sit on top of it. So instead of adding more features, it’s the same experience. It just morphs towards your need.

So more TBD on that one.

Harry Stebbings46:20

I think you should call it a name like Einstein. Don’t you love Einstein as a name for Salesforce?

Gustav Söderström

We call it fighting the entropy in Spotify.

Harry Stebbings

Tell me, Scott, what have you changed your mind on in the last twelve months?

Scott Belsky

Well, I think within a big company, you know, I’ve I’ve changed my mind on the the need for centralization where things should and shouldn’t be centralized. Like, I always go back and forth, honestly, on this. You know, there have been periods of time where I felt like design needed to be in different organizations because people had to prioritize and properly resource design for their business. And then recently, I decentralized design in one organization because our strategy, you know, required it. And I guess one of the things I’m learning is that sometimes, you know, you make a completely opposite decision at different times in the same business because the playbook is different.

And we typically don’t think that way. We like when we make a huge decision and it’s like this is the way it’s gonna be, we just stick to it because we almost think like it’s first principles. But you just have to be willing to discard the playbook consistently.

Harry Stebbings47:20

Okay. Let’s go for you, Tomer. Can product leaders not be trained in AI today? Can you be a CPO and not have really spent hard yards in AI?

Tomer Cohen

I would not hire you to my org if you’re if you don’t have the willingness and the aptitude to go deep. Again, you don’t have I don’t need you to be the engineer working on the model, but I need you to understand, one, the objective you’re trying to build and how to build it. Otherwise, what are you here for? And two, all the underlying principles that come with it. The idea of it’s not being deterministic, so how do you build the knobs so elegantly that the experience is great in any shape or form.

Two, how do you think of data collection in a way that’s responsible, but really fuels what you’re trying to build. And three, it’s that velocity we just talked about. If you don’t have this velocity of learning, I don’t think you’ll last.

Harry Stebbings48:07

Gustav, final one for you. What are you most excited about when you look at AI in the coming years and what it can do for everyone, but also for Spotify? Just what excites you most?

Gustav Söderström

For me, we experienced the shift to mobile. It was very scary. Our business model had to change, could have disrupted the company. It was also by far the most exciting time I had. And to be honest, it was a little bit too stable for a few years there, and I feel like we’re back to that kind of change. I’m I’m secretly hoping for that business model challenge and and those things happening, that everything changes again. I don’t know if you agree, Scott and Tomer, but, like, if you’re if you’re a designer product person, those are the exciting times.

We’re right there now. I don’t think we’re at the end of it. I think we’re at the beginning of it. So I’m kind of back to like more excited than maybe in the last seven, eight years. It’s unclear now what’s gonna happen. We were on a little bit of an iteration track for a while there. It’s not iteration anymore at all.

Harry Stebbings

So that’s what I’m excited about. Thank you so much for this. This has been fantastic and I so appreciate you putting up with my questions.

Scott Belsky49:06

Thanks for having us and Omar and Gustav, good to see you guys. You too. Thanks weekend.

Harry Stebbings

I cannot state enough how much I love doing that show. If you wanna let me know on Twitter at Harry Stebbings what you thought and feedback on the format itself of the show, I’d love to hear your thoughts there. Likewise, you can check it out on YouTube by searching for 20 VC. Love to hear your thoughts on that format also. But before we leave you today,

· Sponsor read0 min · 167 words
Harry Stebbings

HMC or Harvard Management Company. They’re constantly seeking out the next generation of truly great investors and entrepreneurs. HMC has managed Harvard University’s endowment for nearly fifty years and was one of the first institutional investors in venture capital. Their experience and long term investment horizon make them ideal partners to get world changing ideas on a path to viability and success. They work as a true partner, providing insightful perspectives to help managers succeed. I personally have had the pleasure of working with the HMC team and can say that they’re truly exceptional partners and just great, great people to work with.

Whether you’re launching your first fund or your fifth, HMC welcomes the opportunity to partner with both developing and established managers. Have an idea you want to share with the team? Just send it to venture at h m c dot Harvard dot e d u. As always, I so appreciate all your support, and stay tuned for a fantastic episode this coming Friday.

↑ Top