Why Silicon Valley’s glorified “MAU” KPI doesn’t actually matter (and what does).

Your business has done its research and has identified a potential market opportunity to expand your revenue through a digital channel. So your next step is to hire a team of software programmers and get it launched, right?

Wrong.

Your actual next step should be to define a product that fits market needs. Launching a full-featured, multi-platform, ERP-backed digital application without ensuring it’s designed properly would be like proposing on a first date… a bit premature. Consider Airbnb, a startup that originally advertised air mattresses (“Air”-bnb) in empty commercial offices before completely pivoting away from that market segment.

So how do you correctly assess market needs? It takes a highly iterative development approach that puts the user experience at the center of every aspect of the product… rapid prototyping.

What is “rapid prototyping”?

Rapid prototyping involves developing, testing, and adapting a product repeatedly, with each iteration incorporating lessons learned from previous versions.

So let’s say your app idea is to allow users to subscribe to a custom sock service that allows them to customize socks and have them delivered to specific people on a specific date. Your rapid prototyping approach could look something like this:

As you can see, the actual “building” part of the app is broken down into phases, with each phase being designed to gather feedback on different aspects of the product to improve and build upon an increasingly solid foundation. By the time the app gets to an MVP (Minimum Viable Product) for widespread launch, it has gone through multiple iterations and validation steps with actual users.

When should we consider it?

Rapid prototyping is particularly useful in the following cases:

1. Entering an entirely new market: Understand the new market, the user behavior, cultural differences, market demand etc.
2. Implementing a new business model: Confirm critical product economics, market demand, user perception and expectations etc.
3. Need solid proof of concept to raise funds: Validate key metrics in a more cost effective approach in order to raise funds for final product development based on proof of concept.

How do we do it?

1. Define your goals

If you test but have no means of gathering feedback (your testers are just yelling into a void), or it’s not being analyzed, or no changes are made based on said feedback, then it will all have been for nothing.

So before you start testing the product through various phases, you need to write down the goal for each phase, and create KPIs to make sure you stay on track and are measuring accordingly.

For example:

KPIs are critical to the process because they will help you determine if you have successfully tested and validated your goal for the phase and can move on to the next one or not.

So in this example, if you realize the users are not using the sharing feature at all, you could redesign and re-test it, or decide that it is not a core feature, remove it and focus on the core features your users are actually interacting with.

2. What KPIs should I use?

Everyone has heard of DAUs/MAUs (daily/monthly active users), conversion rates and average minutes per session. But when you are rapidly testing a product and your number of users is likely to be small and not statistically significant, it is difficult to use metrics that depend on large user volume. We find these more interesting:

1. “Purchase intent” – the number of times a potential customer shows purchasing behavior. This might be the number of times a product is added to the cart or added to “saved for later”. This metric is key because it does two things: (1) it tells you which products are popular and (2) it provides you with an opportunity to build reengagement campaigns around events like “abandon cart” and “similar products.”
2. “K-factor” – the viral potential of a product. You could look at how often are users sharing content, the app itself, or any metric relevant to your product that can be used as an indicator for potential growth.
3. “Retention” – how often the user comes back. This is a particularly interesting metric because retention isn’t always a good thing. For a la carte purchasing, the goal is to maximize retention and keep purchases up, so high retention is good. For subscription products on the other hand, having users come back can be good or bad depending on your model.

There are of course many more that will be relevant to your business and you need to take the time to understand what they are.

3. How do I analyze them?

The way you analyze your data will depend on what data you collect and what your goals are. But long story short, you should analyze your users actual behavior, not what you want to see, or think you will see. You should also take their feedback with a grain of salt:

Your customers are terrible product designers, but their feedback is critical.

The Simpsons explained it best:

‍

What we mean by that is you have to gather user feedback, but rather than implementing it blindly, you need to analyze the “why” behind it, aka understand it. Once you understand the why, you will be able to gather meaningful data and make appropriate changes.

The 5 Whys and Ishikawa Fish Bone Diagram can be helpful tools to understand the root cause of user feedback for example.

4. Other considerations

You will also want to think through who your target users are, how you will acquire them, which channels you will use etc. But that is part of larger strategic thinking and a story for another article!

Conclusion

There is no one size fits all when it comes to digital products. What works for one product or industry may be completely inapplicable for another. Nonetheless, rapid prototyping is a very useful tool to tackle the development part of your product in a strategic way. And though it in no way guarantees your digital product will be successful, it is another tool on your belt, alongside marketing, research, and many more that can help you get much closer to understating and validating key aspects of your business and digital experience before launch.