profile

The Fountain Institute

Understanding UX Metrics, Part 1

Published almost 2 years agoย โ€ขย 3 min read

Dear Reader,

The next two weeks of Beyond Aesthetics will be a three-part series on UX metrics. Jump to Part 2 or Part 3โ€‹

Q: What are UX metrics?

UX metrics are the numbers we watch to measure the user experience.

Designers have the opportunity to bring a human-centered viewpoint to metrics if they understand them.

We can and should measure the experience of our users using our products, but quantifying UX can be a balancing act of business and user metrics.

Q: How are UX Metrics different than business metrics?

UX metrics differ slightly from business metrics because of their use.

  • Common UX Metric = task success rate
  • Common Business metric = conversion rate

Sometimes the difference is more about the framing and point-of-view than the metric itself.

UX metrics are usually more specific than business metrics, and many UX Metrics are also Business Metrics.

The goal with UX metrics is to center our metrics on the user and their point of view.

I like to teach UX metrics through these five dualities:

  • Qualitative vs. Quantitative
  • Attitudinal vs. Behavioral
  • Vanity vs. Actionable
  • Correlated vs. Causal
  • Leading vs. Lagging

Today, I want to talk about the qualitative vs. quantitative and attitudinal vs. behavioral side of UX metrics

Let's go! ๐Ÿ

Qualitative vs. Quantitative UX Metrics

Qualitative data is messy, unstructured, and anecdotal. Itโ€™s difficult and time-intensive to measure qualitative data, but itโ€™s worth it for the richness of understanding you will uncover.

Quantitative data is precise and easy to measure, but itโ€™s tough to extract insights.

Quantitative can tell you the โ€œhow manyโ€ and โ€œhow much,โ€ while qualitative data can get at the โ€œwhyโ€ and "how."

Design is traditionally a subjective, messy practice that values insights and isnโ€™t easily quantifiable. Thatโ€™s why itโ€™s no surprise that many designers are stronger with qualitative data than quantitative data.

Here's a graphic that I made that helps me remember the difference:

At the Fountain Institute, I teach a course on qualitative data as well as a course on quantitative data. There is power in both, and if you can work with both, you will always have a job in product discovery.

Attitudinal vs. Behavioral UX Metrics

Attitudinal metrics are measurements of how users feel about your product. These metrics are dependent on the userโ€™s ability to answer the questions honestly since they are self-reported.

We mainly gather these metrics from surveys, interviews, or any place where we ask the user about their experience. These data collection methods are time-intensive and manual.

Attitudinal metrics are better suited to pre-experiment research or post-experiment evaluations. They can give you the โ€œwhyโ€ behind the behavioral data.

Behavioral metrics cover what users โ€œdo,โ€ and attitudinal metrics cover what users โ€œsay.โ€

Behavioral metrics are measured real-world interactions with your product, such as clicking, opening, or downloading.

We usually track behavioral metrics without the userโ€™s knowledge using analytics tools such as Google Analytics or bit.ly.

Problems arise when users say one thing and do another.

For Example:
โ€‹
Many people will state an intention to live sustainably, but their behavior may not reflect their intent. Understanding the gap between the intention and the action can provide insights into a solution.

Intention and action can be miles apart, a concept known as the "Say-Do Gap." Learn more the Say-Do Gap in my free 7-day mini course on UX Research.โ€‹

Itโ€™s important to closely watch any Say-Do Gaps because potential ethical issues arise when our products force behavior that doesnโ€™t match user attitude.

For Example: โ€‹
โ€‹Designers might notice that users prefer scantily-clad models in advertisements. This emergent behavior might not reflect true intention. The intuition of the experimenter can avoid issues and discrepancies in behaviors vs. attitudes.

Behavioral data is generally more reliable and easier to count...but it takes more setup than a survey. To generate data, you need to simulate an experience with a concept test before you can gather any behavioral data.

Ok, now that you get the two dualities, let's put it all together and look at the domain of UX metrics:

Designers usually measure a user experience based on a mix of attitudinal and behavioral quantitative metrics.

UX metrics most often live on the right side of the graph because they're usually built on a combination of attitudinal and behavioral quant. metrics.

Of course, this will change depending on the company.

BONUS: Knowing the four poles allows you to better choose which UX method to use based on the resulting data:

Read more about this framework by Christian Roher here.

That's it for today! ๐Ÿ†

You just learned Part 1 of UX metrics. ๐Ÿ‘๐Ÿ‘๐Ÿ‘๐Ÿ‘๐Ÿ‘๐Ÿ‘๐Ÿ‘

Check out Part 2 where we'll get more advanced and talk about 3 more powerful dualities in UX metrics.

Talk to you next Thursday!

Jeff Humble
โ€‹
Designer & Co-Founder
โ€‹
The Fountain Institute

โ€‹

P.S. We just announced an event on September 10th called How to Lead with UX Metrics. RSVP for free here.โ€‹

Here's the poster for the event:

P.P.S. Last week, we hosted a workshop called How to Interview Users. Grab the recording + Miro assets here for free.โ€‹

The Fountain Institute

The Fountain Institute is an independent online school that teaches advanced UX & product skills.

Read more from The Fountain Institute

What is product discovery? by Jeff Humble Dear Reader, So, there's this secret term that product leadership uses to describe your work behind your back. And today, I want you to be crystal clear on what it is. It's a word they use to describe it when you do extra research to ensure you're designing the best possible idea. It's a phase of work that you might call UX, but they call it something else. They call it "product discovery." What is Product Discovery? The term comes from consultant and...

12 days agoย โ€ขย 2 min read

Why Introverts Struggle with Leadership [2024] By Hannah Baker Dear Reader, I want to revisit the "Why Introverts Struggle with Leadership" newsletter I wrote in 2022. It's been two years, and the insights shared back then are more relevant than ever in 2024. I'll also provide new reflections from our current experiences. Why is this important now? In 2024, designers face new challenges, including the rise of AI and the threat of layoffs during economic downturns. Facilitative leadership is...

26 days agoย โ€ขย 5 min read
3 levels of strategy

Why You Need to "Get" the Business Strategy by Jeff Humble Dear Reader, I ran a strategy workshop for 20 design leaders and managers here in Berlin yesterday. We discussed the 3 levels of strategy. Here's the slide: Ok, here's a quick breakdown of the 3 levels: Corporate Strategy - Most companies don't need a Corporate Strategy. An example would be Alphabet's Corporate Strategy for Google and YouTube. You can probably ignore that one. Business Strategy - The main strategy at most companies....

about 1 month agoย โ€ขย 2 min read
Share this post