Product Design Experiments


The best and worst thing about being a junior is that you have to prove yourself.

I remember when nobody listened to my ideas. It sucked, but it taught me how to communicate my thoughts. Here are a few methods I used to defend my ideas when I was a junior:

  • Sketching a solution and showing it to a stakeholder
  • Collaborative white-boarding in the kickoff meeting
  • Making a prototype to demo the idea

Classic design stuff, right? Except visualizing ideas doesn't tell you whether it's a good idea or a bad idea. Without involving the customer, it's pretty hard to predict if something will be a good idea or not.

HMW generate evidence that something is a good idea?

Test Everything

Most leaders don't generate evidence for their ideas. They have people like you build them first and ask questions later. They don't submit their ideas to a test because they wouldn't be leaders if they didn't have good instincts, right?

In this instincts-driven environment, an internal discussion is considered good enough to test ideas. Ideas only require a consensus, and the project is green-lighted. Yay! It's so comforting to have a strong leader and a sense of agreement. High fives all around!

Except your customers don't give a Flying Figma about your instincts. They could care less about your little customer roleplaying meetings where you all agree on their future behavior.

Customers care about their problems, and they don't care about your solutions unless they're considerably better than what's available.

We call the idea of customer-determined value Desirability, a.k.a. Will people want this? When we make assumptions about Desirability, we play with fire.

Designers may be trusted to test Usability, a.k.a. Will this be understandable? Often, executives are the only ones allowed to check Desirability, and the teams aren't allowed to test anything except Usability.

Google's Big Experiment

Google Glass is an excellent example of a Usability Test that should have been a Desirability Test. In their trial program, Google already had a fully-functioning product. They launched it as a Beta release to learn: "Will people use a camera and a screen strapped to their face?" They thought they needed a few usability tweaks before the public launch.

The problem is that people didn't find it desirable to be filmed by some tech bro wearing a weird device strapped to their head. There were colossal privacy concerns, and despite being usable, the product never passed the desirability test.

At least they tested it, and a failed test can teach you more than a "validated test."

But I can't help but think that they could have saved a lot of time and money with a cardboard prototype. It wouldn't take much observation to figure out that an always-on camera on a stranger's face creeped people out.

The Google Glass team made a huge assumption about the human side of their tech-focused idea. It's easy to make fun of their blind spots, but I think they were acting a lot like designers.

They were obsessed with the implementation of their idea. I can almost hear the team saying, "Let's figure out if this button is in the right place, then it will be perfect!" All the while, they were missing the big picture and making huge assumptions.

Testing Ideas = Testing Assumptions

The flaw lies in making giant leaps or assumptions about how something will go. It turns out humans are terrible at predicting the future. Most product ideas are flawed because they're full of huge assumptions and predictions about messy humans in a chaotic world.

Testing the assumptions hidden beneath ideas is challenging. It requires a data-driven mindset and an experimentation system. We want to give you both in just two days.

On November 22nd, we're running a two-day workshop to teach you how to turn ideas into learning experiments:

Testing Product Ideas: Learn a data-driven approach to product experiments
Learn more about the workshop here

Until next week, think about ways that you can test your ideas!

Jeff Humble
Designer & Co-Founder of the Fountain Institute

The Fountain Institute

The Fountain Institute is an independent online school that teaches advanced UX & product skills.

Read more from The Fountain Institute
A collage-style image showing two identical women in long dresses near a revolving door, one entering and one stepping away, with abstract starburst shapes behind them.

Why Decisions Feel So Hard Right Nows By Hannah Baker Dear Reader, Over the last few months, I’ve been talking with design and product leaders across very different organizations, large companies, smaller teams, fast-moving environments, and slower ones. And I keep hearing the same thing. Their teams are being asked to make decisions faster than ever, and yet, deciding feels heavier than it used to. Not slower, exactly. Just harder. At first, people often explain this in familiar ways: too...

IBM Watson AI Failure

When Speed Stops Being the Bottleneck by Jeff Humble Dear Reader, Quick question: What happens when the thing that used to take 12 weeks now takes 4 days? I've been watching this play out across the industry, and it's wild. Lots of companies aren't sharing their new speeds, but a few are: Code and Theory (an agency that works with Microsoft and Amazon) is building dashboards in 40 minutes that used to take a week. They report cutting time-to-prototype by 75%. Coinbase reports a 2-5x increase...

Missed a few newsletters last year? Start here. By Hannah Baker Dear Reader, Before we jump into new ideas for the year ahead, we wanted to pause for a moment. If your inbox was anything like ours last year, there’s a good chance you missed a few newsletters. So instead of sending another new idea right away, we put together a curated catch-up, a handful of pieces from 2025 that capture the questions we kept returning to. If you only read a few things from us last year, these are a good place...