A Process for Setting UX Metrics


Dear Reader,

This is a question I hear a lot. Maybe it's rumbling around in your brain.

Q: How do I set UX metrics?

Organisations run on metrics. It's pretty hard to make the case to improve usability if you can't translate that initiative into UX metrics.

I believe that UX can and should be measured, but it isn't easy. Is there some grand, universal UX metric we can all follow?

The reality is that there is no perfect UX or product metric that all designers can use. In reality, there are many ways to measure UX. So how do we find the right metrics?

Last night, Maria Panagiotidi gave a talk for our monthly meetup called Choosing the Right UX Metrics. In it, she shared some practical ways to find UX metrics. I love what how she defined UX metrics:

"UX Metrics are user-centered algorithms that measure performance."︎ -Maria Panagiotidi

That's so good. I want to zoom in on one thing she covered in her talk called the Goals-Signals-Metrics Process developed by Google. If you've never set a UX metric, I think it's a great place to start!

Goals

The goals are the big-picture targets for your product team. Goals should always be based on outcomes, not outputs. An output example would be completing 10 cards in a sprint. An output example would be increasing your efficiency at work.

I'd recommend that you start with one goal. These goals are usually well-defined quarterly goals for mature product teams, like increasing adoption by 15%. Startups might have goals like improving sales to break even. Don't make these goals up. Instead, ask your PM what the goals are or facilitate your team in that process.

UX Example: We want to improve the usability of our app

Signals

Signals tell you whether you're on the right path toward your goal. Based on customer behaviours, a signal is an early sign of success or failure. Signals can tell you when you're reaching the goal: after using the new feature, a user submits a positive app store review. These signals could tell you that you've failed: a user submits a bug report while using the feature.

These won't be outcomes-based like the goals, so it's ok to have outputs like the user finishes 3 tasks a day. Try to pick 2-3 signals at first.

UX Example:

  • Users complete tasks faster after usability updates
  • We get fewer bug reports submitted after we improve the usability
  • Users log into the app more often as usability is improved

As you can see, the signals don't have to contain numbers until the next stage, but it's ok if they already sound a bit like metrics. Finding signals can be a fun activity so bring your team in on the action. PMs will be the decision-maker here, so defer to their judgment as they know your product metrics best.

Metrics

It's finally time to choose the right metrics by translating your signals into measurable criteria. The metrics should reflect a change over time. Avoid selecting "vanity metrics" or counts that always go up, like increasing follower count. Instead, use ratios or percentages over time, like growing followers by 30% every month.

UX Example:

By Q4 2022...

  • We will reduce the time it takes for users to complete their [main task] by 50%
  • We will get fewer bug reports in Q4 2022 than in Q4 2021
  • Our weekly active users goes up by 10% every month of 2022

If you want detailed directions to run this process with your team, check out these detailed playbook directions.

We'll post Maria's full talk on our YouTube channel next week. In it, she talks about pairing this G.S.M. with the H.E.A.R.T. framework. It's very inspiring! Until then, Maria was kind enough to share her slides:

UX Metrics_deck.pptx.zip

If you find them useful, show some love to Maria on Linkedin!

Until next week, look for signals in customer behaviours!

Jeff Humble

Designer & Co-Founder
The Fountain Institute

P.S. We just added a lot of free tools and articles (and even a few memes) to our new Resources Page if you like advanced UX and Product Design stuff!

The Fountain Institute

The Fountain Institute is an independent online school that teaches advanced UX & product skills.

Read more from The Fountain Institute

Assumption Olympics: Why I Always Win Gold in Overreacting By Hannah Baker Dear Reader, I once pitched a new workshop format to a team of collaborators. One person nodded slowly and said, “Hmm… okay.” That was it. I smiled. Externally, totally composed. Internally? I sprinted up a mental staircase of conclusions: They don’t like it. They’re being polite. They think I’m not strategic. This was a bad idea. I’ve blown this opportunity. Maybe I’m not cut out for this work. I didn’t realize I was...

monopoly dotcom edition yahoo

Monetizing at the Cost of the User Experience by Jeff Humble Dear Reader, Companies don't like to consider how customers can be annoyed by monetization. Companies are always looking for ways to monetize their assets. Even attention is monetized these days. As we'll see in two examples, monetization can make or break the user experience. Monetizing Fail: Yahoo’s Decline (1990s-2010s) Yahoo used to be the tech darling of the internet, much like Google or OpenAI is today. The company had humble...

How to Build Comfort with Ambiguity By Hannah Baker Dear Reader, You're expected to lead even when you have no idea what the hell is going on. (I know, you’ve heard the next part a million times. Bear with me.) The world is moving faster than ever. Technologies shift overnight. Markets pivot on a dime. And somehow, you’re still just trying to get everyone to show up to the team meeting. We all know this, we’re surrounded by it. The speed, the volatility, the endless flood of decisions....