Design Talks: Research in Action with UX Researcher Caitria O'Neill

Welcome to our Design Talks series dedicated to sharing the insights of thought leaders and top people engaged in design from around the world. We interview experts who work with design in different contexts, with different purposes, and through different approaches. We hope to provide intellectual and creative inspiration to all of our readers.

This week’s guest is Caitria O’Neill, a UX researcher with a fascinating past who studies people’s relationships with technology and how they behave with digital products.

Caitria is a UX researcher at Airbnb. She is a former Facebook researcher and Fellow of Stanford’s d.school (Hasso Plattner Institute of Design). Prior to entering research, Caitria co-founded Recovers.org, a community disaster relief software. She’s a graduate of Harvard College and spends her spare time climbing mountains and reading/writing science fiction.

Hello Caitria, it’s a pleasure to have you on the Toptal Design Blog.

One could say that UX research is the soul of the product build process. It’s dedicated to going out into the market and truly understanding how people will interact with a product. What type of UX research do you perform day to day, and which do you find most helpful?

I think there are three main types. The bread and butter of every user researcher is going out and getting feedback about the product—how people are touching it; what they understand, what they don’t understand… how they are actually using it.

There are two other tracks of research that are quite important, even though they are used a little less frequently… and I think I might enjoy these more. The first is foundational: what is the core understanding about the person that indicates you even have a product to build?

For example, if the researcher’s going out with, say… a banking app, and time after time people aren’t able to use it, it might not be that the app is no good; it might just be that they are using a different system. They’re keeping their money in a pillowcase at home… or something like that.

Then second, there is the ongoing research about a product’s health that helps track performance over time—tracking surveys, user sentiment, user experience; are you doing well, or are you doing poorly in areas that matter the most?

Day to day, I’d say I do more actual UX product research—going out and talking to people about the product. But foundational research is definitely where my interest lies because it’s where you learn about what really impacts a person, and what they need most.

It’s getting down to brass tacks, starting with an everyday kind of problem statement, right? Would you say it’s more generative research, not evaluative?

Brass tacks is a great way to put it. It gives you enough information to solve the problems that you uncover, but it doesn’t necessarily focus on saying yes or no to a solution.

On Getting Buy-in for UX Research…

How do you get buy-in to do UX research when the resources or appreciation for it may not be there?

You have to make noise and build trust at the same time. Making noise is basically explaining to people… for example, “we’re finding that it’s hard for some people to use this feature.” But unless you actually explain what it means to the business, there’s no reason for the entire engineering team and the CEO to listen, slow down, and change everything.

What you need to do is build trust so people are aware of your expertise and your motivation around helping with the UX, and supporting the company goals.

In order to demonstrate the value of research, I use themes… a lot. I theme presentations, I use cartoons, I use slang, I swear freely. 🙂 Rather than saying, “you did it right” or “you did it wrong,” I try to be as human and open as possible with the information, and encourage people to work with me to solve the problem.

On Building Bridges Between Designers and UX Researchers…

I’ve seen UX researchers work where they’re like, “I’m just doing my job and I’ll deliver the report, you can take it or leave it; it’s up to you, but I’ve done my work.” How do designers and UX researchers build those bridges effectively when they’re having trouble connecting?

I think it’s knowing at what point in the process to reach out and connect—in Silicon Valley, a lot of the companies that use UX research embed the UX researcher. They’re a full-time team member. They’re in stand-ups, they understand the work, and they’re there when the project starts, armed with everything they know. They’re there to check in with users and figure it out when the first set of design decisions are being made. And they’re there when it’s implemented to test to see if it’s usable. They’re also there after the launch and can see if it actually impacted metrics positively as projected, or if something really terrible happened that wasn’t intended.

I think being embedded helps, but it’s also good if a designer reaches out to work with a UX researcher, or if a researcher reaches out to work with a designer to identify those points where information from the users is going to have the biggest impact.

Stay focused and stay connected throughout, and follow through until the very end.

Yes. In my experience, a lot of good research is wasted. I’ve joined several companies where I did a set of research and then found a similar set that had been performed months before.

Many times the reason that research is done over and over again and the same insights are found, is that the researcher or the team are not pushing them through. It’s often because the process isn’t a good one. UX researchers can beat that process, but it shouldn’t just be on them to make noise and try to convince everyone to do the work the findings have shown needs to be done.

On Misconceptions Between Product Teams and UX Research…

Where are the misconceptions and the biggest disconnects between product teams and UX research?

A big disconnect is that product teams are testing the finished product. They’re testing the solution. They’re setting themselves up for a yes or a no, not setting themselves up to learn anything.

They’re not “testing the problem,” they’re testing the solution.

I know it sounds funny but… is the problem really there? Are we really solving problems?

You need both types of research—generative and evaluative (user research and testing). Unless you’re looking for a yes or a no, when you only check in with research at the end of a product cycle, you’re setting yourself up to launch a subpar product because you’re already invested. At the end of the cycle, it’s already implemented—it’s done. You can’t just roll back and fix the issue unless you know exactly what it is. You’re also potentially setting up a conflict between teams because the UX researcher is then seen as the only obstacle preventing a good, already finished product from going out.

I see it happen a lot, even at big companies. Often, teams will spin for months and months around highly developed, high-resolution designs, only to realize they have to ask a question using cartoons, or simply watch someone use their laptop to understand the way their customers are currently doing something.

On Presenting Research Findings…

How do you present your research findings to other team members or stakeholders effectively? Could you talk about specific tools and formats? For example, slides, spreadsheets vs charts—that you have used and found most effective?

It depends on what I’m trying to do. For example, if I’m trying to create empathy, and there is a whole bunch of engineers and designers in Silicon Valley with the latest iPhones who want to build a heavy app—something that takes a ton of processing power and slows the phone down, I would show them a video of someone in India waiting half an hour for a video to load, sitting there patiently because this is how their everyday life goes. You can make people sweat just by showing them a video and by that, induce empathy. That’s one type of sharing. Video and quotes are great for that. And you can also bring people out into the field (immersion).

Other types of documentation are more evergreen. I’ve found documents are really good when there’s a lot of writing required. Like when there’s a story—maybe a case study. Though typically I prefer slides for presenting research findings. With both a document or a set of slides, even if you’re not there to present them, you can put into effect how someone will be able to access the information in the future and get value from it.

I generally use images, screenshots, and zoom-ins to specific parts of the product, including any findings and recommendations about it. To be effective, I combine all of these methods—slides, documents, share-outs, and so on—with a whole bunch of color, cartoons, themes, slang… code names, occasionally even pictures of team members. Whatever I can do to keep people reading.

At times, this kind of presentation can seem to leadership kind of “dumb,” or that it cheapens the research, but it keeps the team engaged; so I will continue to use Lord of the Rings images for example, and Dogs in Space… I have no shame.

Make it accessible and relevant to the team, right? However, as you suggest, you could risk being called unprofessional in some work environments, especially with the executive suite. They might think, “Why am I not seeing charts and spreadsheets?”

The defense I bring to it is that I’m the only person in the company whose work is never, ever seen externally because of confidentiality. I create the expectation that research is kind of this creative, wacky, helpful, information resource. As an aside, often UX researchers are brought in, not just for their research or to share the findings and work with the team, but also to coordinate and facilitate workshops and creative sprints.

They’re kind of like the “Switzerland of Design” 🙂 —third party, neutral, not interested in which design wins but rather in the outcome. So, if you can create that kind of vibe, then the jokes go over a little better.

On UX Artifacts Used During Research…

When you’re evaluating designs and doing research on existing products, what types of UX artifacts do you use? For example, do you use sketches? Or even paper prototypes, very early stage low fidelity prototypes? Or wireframes? Or high fidelity, finished products that have been quickly put together to prototype?

All of the above. It depends entirely on the question you need to answer at that point. As I said previously, if we’re validating the problem, I’m going to draw some cartoons. We’re not going to spend months building a product. When you start with something low resolution, you can start super broad and compare several concepts head-to-head.

I might bring cartoons out to test concepts. I might sketch with users, have them draw or create their own story illustrations out of existing images or site pieces; create their own information architecture, and then talk about why they did it that way. What’s useful? What existing mental models do they have of other websites that inform this?

After that, I’d move on to low-level prototypes—showing wireframes where we have split concept decisions. We’d ask questions like: Should the tabs be above or below the search bar?

Then you get into the finer stuff. Whenever you need to actually test the usability of something beyond just finding out what the likely mental models are, you need some kind of interaction in order to watch what someone would naturally do.

On Her Background That Led to Becoming a UX Researcher…

You’ve done a lot of different things—apart from UX research, you’ve worked in disaster recovery; you were a backpacking tour guide; you built things; you write sci-fi. How has such a varied background contributed to your user research work?

I wasn’t going to go into design or technology at all. It was the mixed background that led me into disaster relief first… and then someone had to build some spreadsheets, someone had to build a website. I kind of got into tech that way. I think the varied background has helped my UX research career because whether you’re at a startup or a big company, there’s always so much work and the jobs are so diverse, you’re constantly evolving… re-inventing yourself.

If you have only done one thing and you’re very good at it, you might not be able to cope with the amount of chaos and insanity that you often encounter in this kind of work. Even if I’m not using a skill every day, at some point my thought was, “I don’t know how to do this, but I think I can, so I’m gonna do it.” Now I can bring that confidence to not having previously worked with a particular method or type of cross-functional stakeholder.

Sounds like you’re quick to adapt to a situation… I also know that you write a lot about design and research. How has that impacted your career?

Apart from introducing myself and some other amazing people, it’s been a way to get more feedback on how I frame ideas. Not just how_ I_ do, but generally… we don’t often sum up the way we understand things, and then tell someone else and have them give feedback on that worldview.

It’s been very helpful for challenging the way I think about research because I have to put it out there, and then have conversations with the people who want to talk about it. It’s also been really wonderful for connecting with other people in the industry.

On How to Work with UX Researchers…

What is the one thing that you wish designers understood about UX research, and how to work with UX researchers?

There is always something a designer can ask a researcher that will help them with their work. No matter what stage of the process, no matter how shaky or sure a designer is about their design, they can always reach out and talk to the researcher. You can bet that the UX researcher has been pretty much stewing in all of the information—reading all of the CX (Customer Experience) tickets and app store reviews, as well as talking to users every day.

Even if there isn’t a study in process or a specific meeting… or your team doesn’t have a great relationship, you should just ask to build that bridge. The UX researcher won’t be hesitant or surprised. They’ll be delighted that you want to learn more about what they’ve been trying to teach people.

More about Caitria O’Neill:

Medium: https://medium.com/@caitriaoneill

LinkedIn: https://www.linkedin.com/in/caitria