Designers can learn a lot from experienced product leaders.
In fact, I regularly encourage designers to stop asking the question, "Should I learn to code?" and to start considering whether they should learn more product management.
Why?
Because knowing how to map your design activities their business impact is the most important thing you can do to bring value to your team.
But measuring business impact starts with understanding how to measure product success.
Today, we're going to review 3 quick lessons about measuring product success from world-class PM Shreyas Doshi (ex-Stripe, Twitter, Google, Yahoo).
Lesson 1: The 6 types of metrics
Measuring a product (and the design decisions we make) falls into 6 primary categories. Sometimes, a given metric can fall into multiple categories. It's also not uncommon to skip measuring a category entirely.
1. Health
- Key question: Is the product available & performing in the manner that users would reasonably expect?
- Examples: latency, initial load time, uptime, data loss rate, and http errors
2. Usage
- Key question: Is the product available & performing in the manner that users would reasonably expect?
- Examples: time-of-day/day-of-week trends, top N actions, funnel metrics, help docs usage, retention, and password reset rate
3. Adoption
- Key question: Is the product (along with its key features) being used as much as we’d hope and in the ways that we’d like?
- Examples: active users, DAU:MAU, N of M day usage, strategic feature adoption trends, and free-to-paid conversions
4. Satisfaction
- Key question: What is our customers’ overall sentiment towards the product or its main features?
- Examples: overall CSAT, new feature CSAT, and support CSAT
5. Ecosystem
- Key question: What is the macro state of the product within the domain in which it operates?
- Examples: share of wallet, 3rd party integrations, industry rank, marketshare within target segments, and % of TAM
6. Outcome
- Key question: What overall results are we seeing from this product?
- Examples: revenue, margin, revenue per user, active users, marketshare, transactions, and % of Fortune 1000 covered
Lesson 2: How to choose minimum-viable metrics
We improve what we measure, so it's important to focus on 3-5 key metrics when you're just getting started.
But which metrics should you choose?
Use this criteria for choosing your minimum-viable metrics:
- Is it responsive to product changes?
- Is it an aggregate measure of the product's value to users?
- Can it be easily tied to business value?
- Is it expected to be long-lasting (e.g. 2-3 years)?
Lesson 3: Don't become dogmatic about metrics
Shreyas reminds us to be metric-informed rather than metric-driven, and that overindulgence on metrics may lead to focusing on the wrong ideas.
During his interview on 20 Product, he cautions against focusing on "output metrics" before "input metrics".
For example:
- Adoption is an output category. It doesn’t happen on its own. Pre-conditions must be satisfied in order to get to adoption. It is an end result of your team's activities.
- Usage is an input category. It is about how users are using the product. Usage metrics lead to adoption.
Lastly, he points out that for early stage products, more meaningful signal can come from qualitative inputs (e.g. customer interviews) than from trying to find signal in quantitative metrics.
Consider these lessons on measuring product success the next time you're trying to help your team consider how activities map to business goals.
