Two weeks ago we visited User Research London: once again a lovely and inspiring experience, gathering many interesting researchers. We enjoyed the mix of topics that were addressed in the talks. In this blog we share some of our take-aways from the conference.
Two major themes were addressed in several of the presentations. The first one is that we as UX researchers (still) need to define our role more clearly. The second theme is the importance and complexity of measuring the UX of products and services after they’ve been released.
Why we need to define our role more clearly
The research field is growing, we are getting more mature. However, it is still important to keep our team members, customers and other stakeholders on board along the way. Sometimes we get so optimistic about steps already taken, that we forget not everyone around us knows what we can bring to the table. (To go deeper into this issue watch the 12 min interview with Dan Ariely about the ‘curse of knowledge’ when we know so much of a topic that we cannot understand anymore that someone else doesn’t.)
According to Sabine Seyffarth (Facebook), if you work in a team you should start by educating your team members on research. Don’t assume they know the difference between qualitative and quantitative research. Tell them about different research methods out there. She illustrated this with a quote from Michelle Obama:
If you don’t get out there and define yourself, you’ll be quickly and inaccurately defined by others.
Nabeeha Ahmed (Ministery of Justice UK), gave us a personal insight in her career path, with accompanying ups-and-downs. With an emotional story she made a good point that it’s so important for us researchers to clarify to the people we work with, who we are and what they can and can’t expect from us. Nabeeha quotes:
To give content to a concept one has to draw lines, marking off what it denotes and what it does not. – by Richard Rumelt
Moving towards definition and impact
According to Dave Hora our impact starts with intent: our job is to figure out what the team needs to learn to push the work forward. Also, we need to be aware of the larger company setting that we operate in, and how the team fits in this larger picture. Our language and mindset need to be focused on impact: how will learning [this kind of thing] help make decisions or change how we act?
Nabeeha Ahmed compared a UX team to a surgical team. Everyone in the team works together towards the same clear goal, but every team member has his or her own specific knowledge and skills. We know how to do research. It is good to communicate to the team what you are doing and why, but it should still be our decision how to run the research.
So, we need clear roles, boundaries and responsibilities. Make room for user researchers to do good research – in collaboration, not in committee. As researchers we should know and show the craft- do the best we can. Weak research is worse than no research and it may even be unethical according to Nabeeha.
Measuring the user experience
Keynote speaker Tomer Sharon (Goldman Sachs) shared his view on how to measure KEI’s: Key Experience Indicators. He uses the Google HEART Framework in his work, although he mentioned that it’s just a model he personally appreciates, but there are other models to consider.
He listed a number of useful principles to take into account when measuring KEI’s:
- Measuring specific features is better than measuring overall experience: the problem with measuring overall is that when your score is low, you still don’t know what the problem is.
- Measure in context: lack of context can give distorted outcomes.
- Core over perimeter: when you start measuring, focus on the most important part of the UX. Select a limited amount of topics to start measuring and built upon that. This was also described by Monica Todd (Financial Times): their team selected 9 different themes to provide feedback on. Scores on these themes are available to the rest of the organisation via dashboards.
- Use Ratio over Count: if the number of users changes, your counts on KEI’s change as well. But your Ratio’s might not. So by using ratio instead of count you can see patterns over time.
Measuring the UX was also discussed by Bill Albert (Bentley University); he focused on emotions. In spite of the complexities that come with measuring emotions – emotions are brief and fleeting, difficult to report on, highly contextual, a product experience is generally of low emotional intensity, and if we see any, it is hard to differentiate between content and form – Bill explained why he is in favor of it. He argued that it brings a more holistic approach to UX, we learn about trust and confidence, and we can think more deeply about the emotional experience that a product brings.
Facial recognition is the method that Bill recommended most, it offers accurate measurement of subtle fleeting emotions and changes. Other methods include self report, behavioral observation, EEG and eye tracking.
That’s it! We had a blast in beautiful sunny London and look forward to the next edition of User Research London (taking place on the 25-26th June 2020, tickets will go on sale on the 1st September).
Did you also visit this year’s edition? We’re curious to hear about your take-aways!