Evaluate Your Product by Intercom
Let’s start at the beginning. You’re a product manager? A startup founder? CEO? CTO? Whatever your job title let’s assume you’re making the decisions about your product. What features to develop, when to develop them, how to get users to start using them – that kind of thing. As product people we all have great hopes and dreams for our products and how they are going to fare once they hit the big bad world. You’re probably filling notebooks full of ideas that will make you the next Uber, Slack or Facebook. But before you start creating a product roadmap that’s going to lead to greatness it’s time to take stock and see what’s currently going on in your product.
What are people actually doing in your product?
Are all users using all the features in your product? Of course they’re not. Let’s start by talking about that. When planning your roadmap, and where your team spend their time, it’s useful to ask “how many people are actually using each of our product’s features?”. This is product management 101 and will probably take you a few minutes of SQL. A simple way to visualize feature usage is to plot out all your features on two axes: how many people use a feature, and how often. You’ll most likely see trends like this one.
The core value of your product is in the top right area, up where the star is, because that’s what people are actually using your product for. Sidenote: Exclude administrative features like account creation, password reset, etc. from this exercise. They’re not relevant here. Also, exclude features that only certain users (e.g. enterprise customers) can access. They should be evaluated separately.
If you have features in the top left it’s a sign of features with poor adoption. In other words, there are a small amount of customers depending on this feature, but most rarely touch it.
An even simpler way to think about it is this: What percentage of your customers or users have adopted each feature? You can do this with simple bar charts. Shown here is a dream product. The one we all think we’re creating when we have Photoshop open. All my users are going to use and love all these features, right?
But as every product manager discovers, the messy reality of a product looks a lot more like this.
How did you get here? You built a product that had solid messaging, files and document editing features. These key features were highlighted on your marketing site, documented clearly with great screenshots, included in your product tour, and every new sign-up loved them.
But success can be a lousy teacher. You believed you could do no wrong and that you could build lots more.
So you added a chat room, but it didn’t go so well, you kinda botched the launch of it. Then you added a calendar and that went worse. No one created more than one event and it’s still not even mentioned on your marketing site. Now you’ve built this time tracking features that’s kind of popular, but only with a certain type of user.
This is your product, you need to fix this.
What do you do with your feature audit?
For any given feature with limited adoption, you have four choices:
• Kill it: admit defeat, and start to remove it from your product.
• Increase the adoption rate: get more people to use it.
• Increase the frequency: get people to use it more often.
• Deliberately improve it: make it quantifiably better for those who use it.
Roughly, you can visualize it like this:
How to improve your features
Kaizen is the philosophy of continuous improvement. Web businesses searching to find product/market fit all follow some variation of Kaizen whether they know it or not.
Shipping code doesn’t mean that you’re improving anything. Similarly, you can make undeniable improvements to parts of your product and get no response or appreciation for it. It all comes to down to the type of improvements you’re making.
The two most popular ways to improve a product are to add new features, or to improve existing ones. First we are going to look at ways to improve existing features before moving on to new features in the next chapter.
Improving existing features
You can improve an existing feature in three different ways:
• You can make it better (deliberate improvement).
• You can change it so customers use it more often (frequency improvement).
• Or you can change it so more people can use it (adoption improvement).
1. Deliberate Improvements
This is when you know why customers use an existing feature and what they appreciate about it. A deliberate improvement seeks only to make it better in ways that will be appreciated by the current users. For example making it faster or easier to use, or improving the design.
Use deliberate improvements when: there is a feature that all your customers
use and like, and you see opportunity to add significant value to it.
It’s worth noting that deliberately improving a well-adopted frequently used feature is high risk high reward. As an example think about improving the editor in a blogging platform. Get it right, and every single users gets the bene fit every time they use it. Get it wrong and you’ve broken the workflow of your entire userbase. High risk, high reward.
2. Frequency Improvements
These are improvements that hope to get a customer to use the feature more often. Adding more items to an activity feed, or more options to a search tool means that people read it more often, or use it for more tasks each day. This type of improvement can turn a once-a-week feature into an every day feature.
LinkedIn endorsements do this quite well. They added in these one-click multi-directional endorsements where you can easily (dare I say, accidentally) endorse four of your friends for skills they don’t have, and accidentally connect to four more people as a result, which in turn creates more endorsements, more logins, and more single click broadcasts. A vicious circle.
What they’re following is a pattern explained by Nir Eyal, author of Hooked, who says habits are formed from a repeated pattern with four key elements…
Trigger: the reason the user goes to the product (e.g. you received an email to
say a contact had endorsed you for a skill).
Action: they take in anticipation of the reward (e.g. scroll, search, browse, etc).
Reward: the user gets from taking their action (e.g. seeing a beautiful Pinterest board).
Investment: the user makes which will plant the seed for more triggers (e.g. subscribe, pin, like, connect, etc).
As an example, Nir points to Pinterest’s hook as the following:
Use frequency improvements when: there is a feature that the majority of your customers use infrequently, and you believe that using it more would be of benefit to them. It’s worth considering how your business will profit from this increased frequency. For example if Basecamp improve the frequency at which users create projects, they’ll naturally profit, as that’s how their pricing works. But there are lots of easy ways to gamify/hack feature usage that have no net benefit, or in some cases actually damage the core product offering. For example, LinkedIn’s metrics for endorsements might look much better as a result, but it’s worth asking have they paid the price in credibility?
3. ADOPTION IMPROVEMENTS
Adoption improvements target those who don’t use a feature. To get more people using it, rank and resolve the issues that are stopping them from using
it. This is where the five whys technique is genuinely useful. You might have a situation around users not using your reports feature. Why? They don’t see the value. Why? They can’t show it to their boss. Why? They can’t get it into a suitable good format. Why? Because our export tools aren’t good enough? Why? Because our API doesn’t produce good data. If you ask why enough times, eventually you’ll work it out and get to the root cause.
Don’t just talk to one customer because invariably things are more complicated than that. You’ll find these blocking patterns over and over again. You can then resolve the key issues, the fundamental things that are actually blocking people from using all of your product.
Use adoption improvements when: there is an important feature that a good chunk of your users have yet to adopt, and you see some obvious integrations or changes that will make it easier for them to get on board.
When planning adoption improvements, always consider improvements outside of the software too. Sometimes it’s not about how the feature is designed or built, it’s about how it’s explained. Often users just need to know why or how to use a reporting feature. In those cases better product marketing and customer communication is how you solve it, not product tweaks.
Continuous improvements
In their early stages startups have advantages over the incumbents. They move quicker and adapt faster, without much technical debt, legacy features, compatibility issues, or high value customers restricting their movement. Sometimes this speed and agility can cause startups to pivot like headless chickens, rather than focusing on improving their product in meaningful ways. The product manager’s challenge is two-fold; firstly, finding improvements that will benefit the business and its customers, and secondly, ensuring that these improvements don’t get lost on a whiteboard somewhere, and actually make it out the door. Because if there’s one thing that’s true for startup web products, it’s this: if you’re not shipping, you’re dead.