App Metrics 101: What You Should be Measuring

Whether you’re serving your colleagues with an internal company app or you’re dreaming of topping the charts, some would say the hard work is only just about to start. No matter how much testing and research you’ve done, nothing compares to real world usage. 

You’re about to unleash your new app onto the world. Months, if not years, of work are coming together. 

To understand how your app is performing in production, you need to measure the right things. And while it’s tempting to track everything, too much data can be overwhelming. So, what should you be measuring and how?

Three types of app metric

There are three broad types of number that you should be tracking:

• Engagement: how much people use your app
• Business: how well you convert that usage into revenue
• Performance: how well your app code and infrastructure stand up to the task.

Let’s take a moment to look at that word “revenue”. If your app’s primary goal isn’t to generate income, then replace it with “your app’s reason for existing”. Maybe that’s getting people to exercise more or reducing the time people in your company spend filing expenses. Ultimately, it’s a question of ensuring that engagement results in a return for your effort.

Measuring engagement

If engagement is getting fit, then acquisition is buying a gym membership. It’s easy for someone to download your app but getting them to stick with it is another matter. After all, a quarter of people abandon apps after just one use. 

That’s why download numbers are useful to track but don’t tell you much by themselves. So, what should you be tracking? Four numbers in particular are must-haves:

• Daily active users (DAU)
• Monthly active users (MAU)
• Average session length
• Retention.

The first two are pretty straightforward: how many unique users engage with your app on a daily/monthly basis. Which one you care about most will depend on your app. Newspaper app? You’ll care more about DAU. Credit card app? Maybe MAU makes more sense.

Someone opening your app every day, getting confused by your UI, and then closing it after ten seconds would still count towards your DAU. So, if that’s all you measured then you’d be getting a false impression of your app’s success.

That’s where average session length comes in. Now, the raw number tells you relatively little. For example, an ebook app with an average session length of 30 seconds would surely be doing something wrong, whereas for a weather app that could be just fine. 

Similarly, does an increasing average session length mean you’re doing something right? Or does it mean you’re doing something wrong because it’s taking people longer to get things done? Context is essential to interpreting average session length.

That’s the basics covered. You know how many people use your app day to day, month to month, or session by session. But they tell you nothing about who is using your app. 

Measuring retention, on the other hand, tells you who how many users keep coming back month on month. Say, for example, you find that 80% of your users are brand new each month but your MAU is growing by just 20%. That suggests that you have a good acquisition campaign but, perhaps, people just don’t find your app that useful.

To calculate retention over a month, you need to:

• Subtract the number of new users acquired that month from the total number of users at the end of month
• Divide the result by the number of users at the start of the month
• Multiply that result by 100.

Let’s say you start 950 users and end it with 1,000 users. During the month you acquired 200 new users. To calculate the retention rate, your sum would look like this:

[ (1000-200) / 950 ] x 100 = 84.21

That would mean you held onto 84.21% of your users within that period.

Business metrics

Business metrics get very specific to your individual situation very fast. However, there are three numbers that apply to every financially motivated app:

• ARR: average recurring revenue
• ARPU: average revenue per user
• CAC: how much it costs to acquire a new customer.

Let’s start with ARR. Outside of inward investment and one-off charges, this is the money that your app generates reliably again and again. It’s the baseline heartbeat of your app’s viability today.

ARPU, on the other hand, helps to tell a story about the future. Let’s say your app has 100 users and the ARPU is $100. If all other things are equal, you can predict that adding another 100 users would increase your ARR by $10,000.

What if those 100 users and their $10,000 of ARR cost $20,000 to acquire, though? ARPU must be balanced by cost of acquisition. The goal is to find the best ratio between cost of acquisition and ARPU. Over time, you might replace ARPU with customer lifetime value (CLV).

This is why people speak of building an acquisition engine. The idea is that you have a repeatable process that acquires new users. Put more into the engine –– money and people, mostly –– and the result is more users. If you get it right, you know that for every dollar invested you’ll see a predictable number of dollars in return.

How Well Does your App Perform?

Three types of performance issues can kill a user’s interest in an app: slow loading, network errors, and crashes. From a user’s perspective, it’s reasonable to expect apps to function smoothly. As a developer, though, you’re intently aware of how many moving parts are required just to get that first screen to load.

While tracking metrics won’t fix your app’s performance, it can help you both to spot problems before they become serious and also to piece together the story following an incident.

At the minimum, you should be tracking:

• Crashes: capture not only how often crashes occur but whatever anonymized data dumps you can acquire with the user’s permissions.
• API latency: what is the round trip time from your app making an API call to receiving a response?
• Peak API load: how many requests per second your API handles at its busiest.

The good news is that you don’t need to write your own instrumentation to track these metrics. Instead, third-party tools such as Librato and New Relic give you a ready made toolbox for application performance monitoring.

Depending on your deployment platform, you might be able to get lower level numbers for free in your provider’s dashboard. Heroku’s dashboard, for example, tracks response times, load, memory usage, and more out of the box.

While engagement and business numbers can require additional work, monitoring your app’s technical performance can be surprisingly easy.

Apps are a numbers game

After the initial flurry of creativity is over and you have your first GA release, running your app is as much about tracking numbers as anything else. Part of your success in running an app comes down to how well you can interpret those numbers and act on them.

Luckily, tooling to track the most fundamental metrics –– essentially, whether your app can handle user load –– makes getting started easy. And if you’re running on a low-ops platform such as Heroku, matching your resourcing to user load can even be largely automated.

As you progress along your app journey, though, you’ll find the specific numbers that matter most for your business. In the meantime, the metrics outlined above will give you a good foundation on which to build.