Using Lean UX to Solve the Customer Puzzle
By Rachid Harrassi, UX Design Manager
--
There is an undeniable correlation between experimentation and business growth. The more we experiment, the better we learn about and understand our opportunities. The more we understand our opportunities, the faster we grow. And the faster we grow, the more we come together as a team to make a positive impact on our business.
But experimentation without insight can be pretty random if we don’t understand our users, their desires, needs and challenges. And guess what: those tend to change over time based on demographics, location, where they are in the customer journey and much, much more.
Enter Lean UX. We use it as a tool to understand a problem or test a possible solution idea.
Here’s how it works: the product team makes an observation based on data, trends, user research, or simply seeing someone in the wild. Then, based on these observations, the team creates a series of assumptions about a range of business-related subjects: customer value, business outcomes and hypothetical solution ideas. Each assumption becomes a single hypothesis.
We first test out the riskiest assumption. We do the riskiest assumption first because if it’s proven wrong the whole idea fails. Therefore, we want to test it out as soon as possible so the team can start to experiment with another idea for a solution.
It’s common for teams to not get to the real breakthroughs — mostly because they have not comprehensively validated their assumptions. Experimentation is essentially an educational effort to validate these assumptions and therefore find the opportunities for real breakthroughs. Lean UX allows us to experiment with high velocity (and therefore frequency) from soup to nuts.
At StubHub, my team relies on experimentation for two reasons: to build new features and to promote business growth. And from my experience, high-velocity experimentation requires outcome-driven product teams that think problem-first, make their assumptions explicit from the start, and then focus on outcomes rather than outputs.
So, let’s take a closer look at one hypothetical problem-experiment-solution cycle…
Lean UX through 4 Phases of a Growth Experiment
Let’s say there is a feature on an event page that works beautifully to facilitate sales in some markets, but for unknown reasons is not having an impact in others. Given that problem, my team begins an important, four stage process:
1. Observation
“Observations,” to me, are clues or insights stemming from different sources: data trends or anomalies, market or user research; and simply watching customers in their natural environment. The Lean UX framework is most successful when experiments stem from observations and not exclusively from intuition. This requires teams to be data-driven from the very beginning of the process.
2. Define the problem, the goal and the measurable metric to improve
To elaborate, a problem statement should contain a customer issue and a measurable outcome that can be monitored throughout the experiment. Crafting this problem statement helps center the team from the very beginning, keeping them on track and aligned during the project. In the end, this will help validate that the effort at hand is headed towards delivering the outcome stated in the problem statement.
3. Create the experiments
The rule of thumb is to build the easiest, quickest and cheapest ways possible to validate this assumption. Depending on the assumptions, this may be tested qualitatively or quantitatively.
Then, using the hypothesis, we’ll come up with a hypothetical solution based on the simple “If, Then and Because” formula. For example, if we offer this solution to filter and interact with the available listings on the event page, then we will increase conversion rate because it’s quicker and easier to find a ticket based on the user’s mental and behavioral model.
4. Testing the final solution quantitatively and making it a fact
In general, something is tested quantitatively when we want to test the impact of a solution idea on our business metrics. It is tested qualitatively if we want to understand why something works — why it changes something in customer behavior, leading to an improvement in business metrics, etc.
This is where Lean UX comes in. Using a prototype example, our team interviews the customers, both in-person and via a digital tool we present our customers through a particular feature on our website. We then gauge what they understood and didn’t understand about it.
Generally, this prototype isn’t a product on our website and it isn’t necessarily meant to look great. Because of its minimal design, we don’t need to use engineering resources and can instead rely on a design tool. That means we can upload a prototype and be global in range in two-day’s time. If we have 10 people in Germany, 10 in Italy and 10 in Japan, we can just let them interact with the prototype. We record video of them using the tool and then we analyze the videos. Suddenly, we’re doubling the number of experiments we can do by prototyping and not prematurely building a finished product. Now, that’s high velocity.
Based on the qualitative findings, multiple A/B variants of possible solutions are tested against the current version of our product. This is the essential, most accurate step.
The advantages of quantitative research are that it gives us an understanding of how the feature produces against KPIs. It allows us to test a pre-defined solution, giving us data-driven confidence about a feature. Lastly, all the final features should be tested quantitatively to measure the changes in a feature’s performance.
All of this Lean UX, high velocity experimentation helps us deliver a better product by better understanding customer behavior. Each discovery is a piece of the puzzle. The more we complete the puzzle, the closer it brings us to understanding our customers.
Lean UX Testing for a Real World Solution
On our website, we have the tickets and map for an event on the same page and we discovered that people interacted with the map before they interacted with the tickets.
So we asked ourselves: How can we provide a better experience to customers when they’re browsing for tickets within an event, while also increasing Average Order Value (AOV)? What would happen if we offered a solution to select a certain zone before filtering for tickets? The assumption: if users are prompted to select a zone first, they are more likely to start by browsing tickets in the premium areas that would give them the best live event experience. Therefore, customers would be getting a better on-site and live-event experience, while increasing AOV.
People who started to use this option had a dramatically higher conversion rate and AOV. This is an example of a test which we first ran qualitatively, because we wanted to know if it would help us directionally. When we achieved a very high confidence score that the hypothesis was true, then we started building a map of the venue where the customer can see the different zones. By selecting one of these zones, a set of listings would be displayed for that specific zone.
The prototype that we built for testing took us four hours to make. In all, it took us two business days to go from idea to prototype to feedback from the users.
On average, only 20 percent of our ideas are the actual solution. But by testing multiple possible solutions at high velocity, we increase our chances of having a breakthrough.
We are just starting off — but we are already seeing positive results. And now, we are looking to increase the number of experiments we run from five per week to at least double that amount. It’s an exciting time. And thanks to Lean UX, we’re getting smarter, strengthening the team, and making a real business impact for the company.
