Should teams write explicit success criteria before a period begins and review against them afterward?

Glen Coates · Mar 8, 2023

Teams should sketch out their outcomes, assumptions, and principles at the start of a project

Those three are the places where projects go off the rails, and catching them early prevents teams heading in the wrong direction and wasting work that ends up in the trash

Scope: motivated by the scale problem — with ~1,000 projects running he can't dive into all of them

9:03 20Product: Shopify's VP Product on Why the Founder is Always the Head of Product, What Makes Truly Special Product Managers, Why The Majority of Product Managers Need to Change, Why Top-Down Decision-Making in Product is Good & How Shopify Will Be Bigger

Glen Coates · Mar 8, 2023

Teams should be able to answer up front how they will know a project was worth the investment, and surprisingly often they cannot

It is an easy question — if a project takes 10 engineers six months, you should know at the outset how you'll judge that it paid off

10:35 20Product: Shopify's VP Product on Why the Founder is Always the Head of Product, What Makes Truly Special Product Managers, Why The Majority of Product Managers Need to Change, Why Top-Down Decision-Making in Product is Good & How Shopify Will Be Bigger

Matteo Franceschetti · Nov 20, 2023

Running a written 'pre-parade' at the start of each quarter and a postmortem against it at the end is a valuable practice

Writing down in advance what would make the quarter successful gives you something concrete to review afterwards

63:29 Pre parade success criteria plus postmortem review improves quarterly outcomes

Mike Duboe · Feb 15, 2023

Experiment one-pagers should define quantified success criteria up front, and all experiments should live in a single company-visible prioritized triage list

Pre-committing to what success looks like prevents revisionist history where teams claim a test worked despite the data, and a shared sheet lets anyone see what's running and what already ran

31:24 Experiment one pagers should define quantified success criteria upfront and live in a shared prioritized list

Glen Coates · Mar 8, 2023

Project outcomes do not all have to be quantitative metrics; subjective or aesthetic success criteria are legitimate, provided the team agrees upfront on what form the answer will take.

Some work is inherently aesthetic, so what matters is that everyone knows in advance how the question 'did it go well?' will be answered, and what they are working towards.

10:58 Qualitative agreed upon success criteria are legitimate if defined upfront

Glen Coates · Mar 8, 2023 · hedged

Outcomes is the element of the framework that gets done least well, because work low in the stack cannot be credibly tied to financial results.

For core platform and developer-platform features the web of dependencies is impossible to unpack, unlike products such as Shopify Capital where dollars are directly attributable.

14:49 Platform work outcomes cannot be tied to financial results so goes undone