Are you experienced? Measuring success in the new mobile economy

Nolan Wright is the CTO of Appcelerator, the only platform built for a Mobile First world. Nolan graduated from Vanderbilt University with a degree in Electrical Engineering.

Fifteen years ago in the Harvard Business Review, B. Joseph Pine and James Gilmore argued that businesses, facing the commodification of almost every kind of good or service, would soon be driven to offering superior “experiences” as the only means of differentiation.[1]

Predicting the rise of an “Experience Economy,” the authors couldn’t know how accurately they’d nail the mobile world to come – one where apps vie for attention through the elegance of their design, and where users decide corporate fortunes with the swipe of a finger.

Call it the User Experience Economy.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Strangely, in this User Experience Economy, most organizations are trusting their fate to user feedback. There are a few of problems with this approach. First, it skews to polarities: usually only the most thrilled or disenchanted users bother to rank or comment on an app.

Second, user commentary is a lagging indicator. By the time a company has uncovered and diagnosed poor user sentiment, users have abandoned the app in droves and moved on.

Moreover, the crudeness of the star system leaves most enterprises blind when it comes to understanding their mobile apps. Ratings alone won’t deliver true understanding of user preferences (or frustrations), or how the app is being used, its performance, or what is the best business decision for the next version.

Experience-driven analytics for the experience economy

Mobile is driving the need for app and portfolio measures unlike any we saw in the days of web. Good mobile analytics must provide insight into both the behavior of the app and the behavior of the user.

Understanding the behaviors of the app is a good start. Crashes, for example, are the kind of app behavior that should trigger an alert, as well as the source of the trouble. Otherwise there’s not much hope of fixing the problem before users begin to mutiny.

But seeing into app behavior alone isn’t enough. For a complete picture of engagement, we need to understand user behaviors as well.

What is the frequency and duration of user engagement? When and where are users most often interacting with the app? On what types of devices & platforms are users engaged with the app? Which features are most popular?

Just as apps are seldom built without some form of requirements, neither should they be launched without measurable usage goals. The analytics above stand as a quantitative, metrics-based strategy for app improvement. This is a chief difference from pre-mobile applications, few of which provided the specificity of usage and context data that mobile apps do.

The five mobile measures no company should be without

Companies positioned to win the User Experience Economy track five key metrics for their mobile apps:

1. Adoption: As measured by the number of installs of the app – that is, the number of downloads from either a public or enterprise app store. But an install by itself does not equal an engaged user. This requires…
2. Engagement: User engagement is measured by average session length. Session length refers to the amount of time a user spends in the app each time it’s opened.
3. Retention: The number of active users divided by the total number of installs.[2]  Retention carries an important time dimension, usually comparing changes week over week or month over month to establish trends.
4. Conversion: This measures how many users who begin business processes enabled by the app actually get through to completion. Take a basic order-to-cash example: 1) user logs in 2) selects merchandise 3) chooses payment option 4) confirms and submits order. Measuring conversion tells you how many people who opened the app actually made it through all four steps. Moreover, conversion measures when and where users abandon the process – crucial for understanding where app bottlenecks or poor design may be causing users to give up. (The technical term for this is “funnel analysis.”)
5. Exceptions: The ratio of app crashes to app sessions. In the User Experience Economy, any ratio over 1:20 spells trouble – users will lose patience and delete the app. World class mobile organizations achieve exception ratios of 1:100 or better.

Barring the last (Exceptions), the target number for each of these measures will vary based on the aim and function of the app or its addressable audience. For example, a transactional app for movie tickets or restaurant reservations should expect briefer engagement than a magazine or other content-heavy app.

These metrics are most valuable when viewed in the context time, location, device and operating system. More advanced mobile organizations will slice the data by cohorts, for example, comparing these measures across different versions of an app to understand how the new release performs relative to older ones. But the main thing is to start… the sooner the better.