4 Tips for Picking KPIs to Measure Your Product’s Success

new_kia_car_financing

Of all the items on a new car sticker, which ones should matter most to the car’s product manager?

Product managers, how do you know if you’re doing a good job?  Your manager tells you so? A customer leaves a glowing product review? Your coworker likes the new feature you launched?  Nope. You’ll know if you’re doing a good job if your product is successful.

But what does success means? It depends on your product, but no matter what, if you don’t have a way to measure success, you’ll never know, and neither will your stakeholders – internal or external.  Here are some tips on choosing Key Performance Indicators to measure the success of your product.

Tip 1: Start With the Money

I’ve written about this before: the ultimate measure of success should be a business outcome, so you should have at least one KPI that has a dollar sign in front of it.  Profitability is ideal, but revenue or cost savings for your company are also good.  If you need a shorter feedback loop on a KPI like revenue (because your sales cycle is really long), use a proxy KPI (for example, maybe you know that your sales team closes 80% of deals after 2 meetings with the decision maker – great, use that instead of revenue).

 

Tip 2: The Customer KPI

At least one KPI (not revenue) should be one that you share with customers and that measures the outcomes they’re trying to achieve through your product.  It might be time or cost savings, or weight loss if you’re building a health app.  Don’t be shy to get creative with this one – maybe you derive a new formula that combines cost savings, time savings and weight loss if all 3 are key outcomes your customers are seeking.  Having this customer KPI focuses the team on delivering value regularly, and helps you avoid awkward “I don’t know what I’m paying for” conversations when you ask customers to extend their term with you.

Tip 3: Don’t Pick Too Many KPIs

At most, I’d suggest 2 or 3.  Why? Because if you’re prioritizing changes (or tests) to improve your KPIs, it’s hard to juggle too many.  Don’t forget about the “K” in KPI.  In an ideal world,  you might even assign a KPI to each of your product owners and development squads, so that they know whether their work is resulting in meaningful business value.

Tip 4: Measure Often

KPIs don’t matter if you can’t measure them easily, especially if you’re releasing frequently as a part of an Agile process.  Make sure you can measure your KPIs within a few minutes, and that the data needed to measure them is updated often – at least daily.  If you need to prioritize time to instrument your product to make measurement easier, do it.  Otherwise you’re either flying blind, or there’s too much of a delay in your feedback loop.  Also, don’t forget to publish your KPIs regularly for internal stakeholders, so that they can also see how your product is doing.

 

Want to discuss your KPIs?

5 Tips For Building Product Roadmaps

roadmap

Managing a product without a roadmap is like driving off road.

April 2017 update: my thoughts on roadmaps have changed – if you’re working in Agile, be sure to read Why Agile and roadmaps don’t mix.

Not every product needs a roadmap. If the product is really young and you’re still trying to find product/market fit, don’t waste your time planning a roadmap that’s probably wrong. Instead, spend your time building what you think will sell and iterating based on feedback. (yes, I’m assuming you’re building your product using Agile because it’s 2016)

But if you have a lot of stakeholders (clients, sales, marketing, support, operations, etc.) who need to have an idea of what’s coming down the line, you’ll need a roadmap. Here are 5 tips:

1. Group changes into epics.

Ideally you’re grouping together changes based on the problem you’re trying to solve for a user. For example, with HelloWallet, we recently spent a few sprints focused on helping users decide how to juggle their savings and debt paydown goals simultaneously, since we knew from our user research that it was one of the toughest choices our users were facing.

The reason to group changes together is for efficiency sake. If every sprint is a hodge podge of user stories that cover a bunch of parts of your product, there’s a lot of context switching for everyone on the squad — design has to understand the existing UX and suggest changes, engineering has to touch different parts of the code base (some of which might have cobwebs on it), and QA won’t be able to prioritize regression testing scenarios since you’re changing things all over the place.

2. Prioritize epics based on KPIs.

A roadmap is nothing more than a listing of the things that you think have the highest likelihood of moving your KPIs (in particular, the $$$ KPI I discussed yesterday).

Prioritization is a science and an art. You can get scientific by creating a score card to evaluate epics on their likelihood to move different KPIs, or based on other factors. You can edit the weights of the different factors easily so that you’re scoring epics consistently.

example score card

The art comes in with a gut check of whether you’d actually prioritize epic 3 at the top of your list. Maybe you socialize the score card with your squad and get some qualitative feedback (“ugh, that?” from your favorite engineer or “oh, shit, that’d be killer” from your favorite sales lady). And you adjust accordingly.

3. T-Shirt size epics.

As a Product Manager, you should never make up timelines. Mainly because you’re not the one doing the work. You’ll need buy in on timelines. No one on your squad wants to feel like they are handed an unrealistic timeline and have to execute against it. Hence getting them involved in the process.

At HelloWallet and Morningstar, we use t-shirt sizes to get us in the right ballpark for an epic (it’ll never be very accurate). We start by writing the epic as a user story and getting a group of people together to estimate it — tech, design, product is usually a good starting point.

During estimation, a million questions come up. That’s why we write down assumptions to make sure we’re all on the same page in terms of scope. (“oh, what about this scenario?” — make an assumption about how to handle that scenario).

T-shirt sizes are as follows:

  • XS = < 1 sprint
  • S = 1–2 sprints
  • M = 3–5 sprints
  • L = 6–8 sprints
  • XL = 9–13 sprints

Once you have an estimate, get conservative and use the high end of the range. Assume you forgot about some part of the scope or that something you all thought was going to be easy will be hard. No one will ever complain if you execute on your roadmap faster than you said you would (well, I suppose the would if you ALWAYS executed faster). But they’ll really give you the stink eye if you’re late in delivering something they planned for.

4. Build a 12-month roadmap.

Don’t try to plan the next 3 years — it’s gonna be wrong. Just plan the next 4 quarters.

At Opower, we ran some stats and found that our roadmaps were really accurate for the next quarter, pretty accurate for the next two quarters, and almost always wrong after 6 months. That’s not bad — a lot changes in 6 months (especially at startups).

5. Get feedback.

Socialize your roadmap, especially with sales. If they shake their head, find out why. Is there something else they think would be more meaningful? Talk to other people in your organization, even ones who don’t work on your product or in your business unit. And in true Agile form, iterate the roadmap (once or twice). But don’t update your roadmap more often than quarterly. As you can see from above, there’s a lot that goes into the process. If you’re updating it monthly, you’ll never have time to execute on the roadmap.

The Only Product KPI That Matters

Managing a product without KPIs is like driving a car without navigation — you’ll never know if you’re closer to where you want to be, and neither will your passengers. (“Are we there yet?”, for your road-tripping parents out there)

KPIs

Key Performance Indicators are simply metrics you use to measure the effectiveness of your product. They may sound simple and obvious but getting stakeholders to agree on what actions are most important to track isn’t easy. But doing so ensures alignment, which is really important, as I’ll explain in another post about using KPIs to generate roadmaps.

The Only KPI That Matters

The $$$ KPI: profitability, which can come in the form of “scalable revenue” or cost savings (or maybe both). If you’re not prioritizing your roadmap according to this KPI, don’t expect to be managing your product for long (even at a startup). But wait, what about helping users solve problems or reaching your company’s mission? Yes, those are definitely important but if you don’t have any users through sales, does it matter how great the user experience is or how much you want to change the world?

Other KPIs

I’ve found it best to have only 2–3 KPIs, including the $$$ KPI, to keep things simple (and to avoid prioritization conflicts when using the KPIs to generate a roadmap). The other KPIs should be related to the $$$ KPI, though — most likely leading indicators of whether you’ll generate more profit. These KPIs may change over time as you learn more about your business, customers and/or users. Maybe you think logins or email opens are the most important metric, until you realize that there’s a particular feature that’s really moving the $$$ KPI.

As an example, with HelloWallet, we use the Financial Wellness Score as a KPI — if we can measure whether we’re improving our users’ personal finances, we’re more likely to keep existing customers and win new ones.

Tracking KPIs

Monthly (maybe quarterly) is perhaps the cadence to track KPIs. Ideally you have a dashboard that’s updated automatically every night. But you probably don’t because it’s hard and there are a million other things that are more important (or are they?). Even if you have to manually update a dashboard, publish it somewhere so all your stakeholders can see it, and discuss the status of the KPIs regularly (including what actions can be taken by different parts of the organization to move the KPI). Otherwise you’ll never know if you reached your destination…