10 Learnings on Maturing UX
Lessons we learned working with our first in-house UX Researcher in 2017
Written by Marlene Schufferth, Product Design Lead
One topic that has kept my mind occupied this year is how to mature UX inside GetYourGuide. It meant understanding and formalizing what design, copywriting & most importantly research add to the product conversation and how we can give all a seat at the table so they can be a catalyst for ideas and strategy instead of trailing behind tickets in the backlog.
In this article I want to share the top 10 learnings we collected bringing in our first in-house UX Researcher this year and how we turned user-centricity from buzzword into more of a reality.
Understanding the status quo
In one of her first presentations to the company our UX Researcher Tamara, breezed through a UX maturity matrix that she was part of developing to help you identify how user-centric your company truly is?
Although it was more of a good to know, it really stuck with me because it revealed a very painful fact: we were just at the beginning of the journey. At least we were past level 1 I thought, but there was a long way ahead of us.
In order to get there, we had to make maturing UX a company effort. It doesn’t help if one person is already level 4 another level 5, if the majority is still at 1 or 2. You need to advance the whole company through every stage to build up the understanding, methodologies and practice so that UX is distributed across the company and can have the biggest impact for your users and the business.
Fast forward 9 months and we find ourselves on the verge to level 3(spoken as the optimist that I am 😁).
Here are 10 things we learned along the way:
Learning #1: Don’t trust buzzwords
These days people working in UX call them themselves many things: UX Designer, UX Researcher, Service Designer, Design Thinker, UX Prototyper, Experience Designer … Sometimes it is really hard to navigate the ocean of UX titles. But it is essential to not be blinded by the shiny buzzwords but to get to the bottom of it!
Someone says “I created personas” — Great, why? On what basis? How did they look like? What did they impact? How did the team work with them?
Another one says “I mapped out the whole user journey” — Great, what for? What was the goal? How did it help? How did the teams work with it?
Identifying if they do things with purpose and great reason is what will navigate you to the talent you need and away from the buzzwords. The best way to do that is asking a lot of follow up questions. Use Why/How/What questions to peel away layer after layer and make sure to always bring it back to real life work examples. It is one thing if someone can talk a good talk (which is also very important) but you also want someone who walked the talk and can take you by the hand if you are not an expert yourself.
Learning #2: Your research is only as good as your question
It is important for the team to understand that any research they initiate is their research. It is not the researcher going off by herself and then coming back with action items. Research is a co-created, co-owned process that starts with your questions. This also brings the difficulty of reflecting on what you actually want to learn from research?
While asking questions comes easy for most team members, formulating questions for research for your every day process isn’t quite the same and quite as simple. This is why it is helpful to have a list of questions to guide the conversation and help shape their hypothesis:
- What is the problem?
- What is the solution?
- Who is the user?
- What questions do you have?
- How does success look like?
These questions are similar to questions used in the product development process and ideally the team has already answered them earlier on. The universal fact is: if you don’t understand the WHY you won’t be able to design a great solution or run a successful research. So make sure you are really confident in what you want to learn, if you are not, then research is maybe not the right next step.
Learning #3: Your insights are only as good as your participants
One thing I learned this year is that biases are very powerful and inevitable. Moreover no one is free of them, which is why it is incredibly important that you find research participants that are as unbiased as possible.
It is always a positive sign when a Designer mentions in an interview that she user tested prototypes to get feedback before implementation. When asked who they tested with the answer is more often then not “the people in my office” or “we went into a coffee shop”. While this is a good way to practice setting up tests, you need to be aware that people you work with are incredibly biased towards what you show them. Even if they fit the demographics of your user persona they …
- know the product (sometime inside out)
- live in a educated tech bubble
- don’t want to hurt your feelings or are extra critical.
All of these factors and many more, really skew the quality of your results. And even if you use a remote tester base like we do through usertesting.com screening for the right participants will highly improve the results of your tests. Before having a researcher we set barely any limitations for our screener yet a lot of our prototypes were assuming a certain travel knowledge or experience. Being mindful of which audience will use your feature will also help you screen the right participants for your user tests.
Learning #4: If you have no question, don’t research
Having an in-house researcher got everyone really really excited. Every Designer, PM, team had something they wanted user insights on immediately. Soon we had a waiting list for tests and sessions. Designers would create a prototype for every feature and launch a remote user test for it. Looking at the analysis together we soon realized that a lot of the questions included in the research were already answered by quantitative data or the confidence in the solution was so strong that research was more used to create an extra argument for something that was going into implementation anyway.
The truth is UX Research is expensive because it is very time consuming. So make sure to invest your and the researcher’s time in the right places.
There are certain times in your product process where research can create great value. The emphasis is on CAN. It doesn’t have to be part of it if your problem understanding is strong enough and your mind is already made up on the solution.
If you are expecting research to give you 100% certainty or absolution you are using it the wrong way. It can help you mitigate risks for launching big features, but if the risk is small, it might be better to just launch and learn.
I like to believe that 50% certainty are good enough to run with it (most of the time)!
Learning #5: Make research the job of the team
Like mentioned before, a key ingredient of successful research and maturing UX is creating an agreement that the research is for and owned by the team. People have various experience working with researchers internal and external, but a common pattern seems to be that the research team is regarded as a service provider that delivers actionable insights. We learned that successful research is coming from collaboration with the team very early on in their product process, strong involvement in the sessions and ownership of the analysis.
Why? Involving research early on in the feature discussion gives the researcher full understanding and context of problem, solution and goal which will reduce the time spent in research briefing a lot. Having every team member join sessions will create the necessary context for them so that later on when you move into the analysis part everyone can contribute with the same contextual understanding. Also every member will contribute different observations which will diversify and enhance your analysis. The more context you have, the more sense the analysis makes to you, the more likely you are to take it and run with it.
We created that shared understanding initially by simply making it mandatory for all team members to either sit in in sessions or watch the recordings and streams. Otherwise they would not be allowed to participate in the analysis part. Even though there was some resistance in the beginning, because again it is a significant time investment, witnessing users struggle created great motivation. Even the strongest critics eventually asked to be part of the next session.
Learning #6: Leverage your existing insights
Sometimes asking for research is just being too lazy to dig for some data to understand the problem. The answers to a lot of your questions are often hidden (not even too deep) in the data you track and collect every day, every minute, every second.
Especially if your question starts with a “how ” it is a good indicator that quantitative data could deliver you the insight. How many users click on this? How long in advance do they book? How long does it take them? How deep do they scroll? But also FAQs, customer service requests or reviews are a great source for quantitative customer insights. It takes some time to mine it but it already has heaps of valuable insights burried in it.
The intersection of quantitative AND qualitative insights is really where things start to get interesting. It allows you to see the how and what with your existing users plus understand the why behind it. Very powerful and probably the closest you’ll come to certainty.
Learning #7: One step at a time
Rome wasn’t build in a day and so won’t be your research skills and customer understanding.
We quickly realized that our team members had very different levels of research experience. Some had never done any, others did it DIY and then others had worked with researchers in various methods from interviews to usability to card sorting etc. In order to bring us all on the same page and have us all talk the same language, our researcher moved away from trying different evaluative methods at once but focused on one and how to get that right.
With only one researcher in-house you want to be very mindful of how their time is invested. Is it the most useful for them to run every usability test and do every analysis? You guessed the answer. The key is to educate and coach the teams in particular methods so that they can become independent in using them. In our case it was remote unmoderated user tests through usertesting.com.
As Benjamin Franklin once said:
“Tell me and I forget. Teach me and I might remember. Involve me and I learn.”
So tackle one method at a time, teach, try, teach, try, teach and try again.
Learning #8: Make research easily accessible
This almost goes without saying, but the easier your research is accessible to everyone in the company, the more it will be used and the more leverage it will have.
It is a good idea to set up a directory or channel for it as early as possible because as soon as you present the first insight to any group big or small people will ask you where they can access it. The time you invest initially in setting it up will be saved time and time again when you can just link to the assets.
Learning #9: Show instead of tell
You can talk the best talk on why having a researcher in-house is important but you will still look into doubtful faces until you show them what user insights can look like and how powerful they can be.
In our experience, this is hard to do through evaluative research but requires investment into generative researcher. If you are as clueless about the difference between evaluative and generative as I was, here is a good article for you.
While our researcher was doing a great job training our teams on evaluating prototypes, they wouldn’t create the game changing insights everyone was longing for.
It was when we launched research on how people plan and book attractions based on a set of interviews with UK travelers that really demonstrated the quality and power of research insights.
A researcher we spoke to recently formulated it very well:
“To realize the full potential of user research, teams and companies need to prioritize importan generative research over urgent evaluative research.”
This research not only started great conversations across multiple teams but also informed priorities and goal setting. So much so, that when we kicked off a new product mission in the end of this year, we made generative research the first step to kick-off the strategy.
Learning #10: Keep calm and carry on
Last but not least: there will be set backs, hurdles, critics and painful times of learning. Do not give up, push through and you’ll be rewarded.
If it hurts it simply means your learning at the right intensity, so enjoy the pain knowing that you’re building up your UX muscle. 💪
With that said, I wish you all a fantastic 2018 full of learning and growing and hope that some of our learnings can path your way next year.