User research is a team sport

‘User research is a team sport’ is the most powerful thing I’ve learned in government. Forget user needs. Forget discovery, alpha, beta, live. Forget agile. Forget the service standard and service assessments. This is the one thing I would save in a fire.

I love ‘user research is a team sport’ for two reasons. First, it increases the impact of our user research on the services we make. Secondly, and less obviously, it rapidly improves our research practice as individuals. These are precious outcomes.   

Yet recently I've heard some worrying interpretations of what ‘user research is a team sport’ means. That it means anyone in a team can and should do user research. Not true. Or that it means user research is not a specialist skill. Also not true. I’ve even heard stories of ‘user research is a team sport’ being used to shame user researchers for not sharing every aspect of their work. Unacceptable.

These interpretations are starting to spook user researchers into disavowing ‘user research is a team sport’. Given the precious outcomes, this is a tragedy.

So I’m going to talk about my own experiences of ‘user research is a team sport’. I’ll show why it’s powerful. I’ll show that it goes hand in hand with being a specialist user researcher. But most of all I’ll give you a detailed guide of how to make it work in practice so you can do more of it yourself.

Big-reveal research doesn’t work

I used to be that researcher that swans in and thinks their job is to save the client with user research. Elaborate methods, psychological concepts, scientific analyses, fancy graphics, flashy presentations. This made me feel heroic, knowing and enlightened.

The problem was that it didn’t work. Little changed after I left the room. No matter how ‘right’ or compelling or evidence-based my findings felt to me.

I slowly realised that change in organisations comes from change within other people’s minds. Not my own. Most people only change their mind when they experience things themselves. People don’t respond well to being told what to do.

So I changed my research practice to be more collaborative. This worked well. And then, inspired by ‘user research is a team sport’, I decided to go in-house with GDS.

Involving the team in research does work

I spent the first six months at GDS working out how to apply ‘user research is a team sport’ in practice. On GOV.UK Notify, through gradual trial and error, I found ways to involve the team in five critical areas of the research process:

• Generating research questions

• Observing sessions

• Taking notes

• Attending playbacks

• Prioritising findings.

As I did this, I watched my research make changes to a real product for the first time in my career. The team responded to the research without being told what to do. It finally clicked. If you involve the team in the research, in a way they understand, they will make better decisions whenever they make decisions. Not just when you’re there.

Something else happened too. The team began to share their thoughts about ways to improve the research. This group of non-researchers helped me learn some of the deepest lessons I’ve learned about research. Stop researching when you’ve learned the big things. Pick the riskiest things from the roadmap first. Know that some things are OK to ignore completely because they’re easy to fix later if they blow up.

This is the power of ‘user research is a team sport’. It increases the impact of our research and it improves our research practice at the same time. Precious outcomes.

A field guide to ‘user research is a team sport’

I want to save you from the trial and error I went through in working out how to apply ‘user research is a team sport’. To do this I’m going to step through the four stages of the user research process and show how it worked for me:

1. Planning research

2. Doing research

3. Analysing research

4. Communicating research

At each stage I’ll talk through the most productive ways I found to involve my team in the research. I’ll also show that specialist researchers are central to ‘user research is a team sport’ by covering the things that I did on my own, away from the team.

1. Planning research as a team sport

Planning research is about coming up with research questions and turning them into a research plan made up of different activities. It’s also about arranging participants, a location, a discussion guide and your research materials.

These were the best ways I found to involve my team in planning research:

• Generating and prioritising research questions. Every three months I would drag my whole team into a room for an hour and ask them what they wished they knew about their product. They wrote questions on post-its, clarified what these meant, and then prioritised them. These sessions took a little practice to get right but ensured that my research was relevant to my team’s needs.

• Discussing the riskiest items on the roadmap. Once I’d won the trust of the team I found ways to meet regularly with the product manager, technical architect and designer to understand what was coming up. Research has to work a little ahead (all that planning!) and I wanted enough time to research the riskiest assumptions. Looking ahead helped me avoid blocking the team’s progress.

And these were the things that I did on my own:

• Finalising the research questions. My team came up with loads of research questions in a million different formats. It was my job, alone, to synthesise my team’s questions into a coherent research plan.

• Choosing research methods. My team were not experts in research methods. I knew what methods we could use to answer the research questions in our research plan. I tended to use simpler methods to bring my team along with me.

• Recruiting participants and arranging the space. Any time there was detailed, logistical, grunt work this was my job alone. That was OK with me. My team had better things to do than help with with research admin.

• Preparing the discussion guide and research materials. I’m the one that knew how to write a discussion guide that answered our research questions, that moved from broad to specific, and that left enough room for improvisation.

That’s what it means to me to plan research as a team sport. You invite your team to help you choose the research focus and then you - as a specialist user researcher who owns the process and is accountable for the results - do the detailed work to prepare for success on the day.

2. Doing research as a team sport

Doing research is about running sessions with participants. You might be in a lab or in someone’s house. You might be interviewing or testing or observing. There is always someone moderating the session and some other people observing and taking notes.

These were the best ways I found to involve my team in doing research:

• Observing research sessions. Everybody on the team observed some research. For lab sessions I invited the team into the observation room. When we went out in the field I invited a non-researcher to come with me. I even combined the two, putting the team in the lab’s observation room and streaming back sessions live from the field. It’s important to note that I never insisted the whole team spend the whole day observing. One or two sessions each because people are busy.

• Taking notes in research sessions. Observing was great but people got bored and started checking emails unless they had something to do. So I made them take notes. I set up a different person as the primary note taker for each session. I asked other observers to note down the five most important things they saw at the end of the session. This made sure everyone paid attention and had a voice in the notes. It also helped me jump-start analysis many times.

And these were the things that I did on my own:

• Moderating the research sessions. It has taken me years to be a passable moderator. Leading questions. Body language. Neutral confirmations. Knowing when to probe. Getting consent. Dealing with ethical situations. Navigating dodgy prototypes. Slipping into role playing. I am still learning all of this.

• Ensuring notes were high quality. My team wrote terrible notes at first because I hadn’t found a way to explain that I wanted observations (what individuals do) and quotes (what individuals say) but not findings (what clusters of people do) or solutions (what changes we should make). Over time I coached them into writing better notes. Until then I re-did the notes myself where it mattered.

• Storing research materials safely and securely. My team were not trained to worry about the ethical implications of the personal information stored in our recordings. Nor were they accountable. It was my job to store things safely.

That’s what it means to me to do research sessions as a team sport. You invite your team to observe the sessions but you - as a specialist user researcher who owns the process and is accountable for the results - do the moderation and ensure that the notes and recordings are dealt with appropriately.

3. Analysing research as a team sport

Analysing research is about taking the observations from your research sessions and turning them into compelling findings that prompt your team to make better decisions.

The analysis stage has the trickiest relationship to ‘user research is a team sport’. Bluntly, I tell less experienced user researchers to do analysis on their own until they’re 100% confident in their own approach. But even for experienced researchers there are two reasons that analysis is tricky as a team sport:

1. Analysis is time-consuming. It easily takes as long as the research sessions themselves but you can’t divide it up between different team members like with research sessions. Having the whole team spending a day is usually unjustifiable.

2. Analysis is a subtle, personal, intuitive process. The best analyses I have done were rambling journeys. I used my intuition to discount or emphasise things. I inferred things beyond the visible data. I was influenced by outputs that I imagined. I know this makes analysis sound like weird magical alchemy. I’m sorry. Analysis has always resisted my attempts at clear, rational explanation.

That said, these were the best ways I found to involve my team in analysing research:

• Quick and dirty usability analysis. In sprints I often needed findings the next day and wasn’t bothered about being comprehensive because I knew we’d be back in two weeks. So I’d invite multiple team members to each session and ask them to write down the five most important things they saw for each participant. The next day we would cluster these, name the clusters, rewrite as findings and prioritise as a team. This flushed out 80% of the big problems. The team ended up so bought-in that they often didn’t even need a playback.

• Transcript-based exploratory analysis. Occasionally with depth interviews I’d invite the team to a multi-hour analysis session. Each person went through a transcript, highlighted quotes, and passed the transcript on until each transcript had multiple eyes on it. Then each person wrote up the 10 (ish) most compelling passages onto post-its for the transcript they were holding. Again, we would cluster, name, write findings and prioritise. This was a great way to immerse the team in discovery research but it wasn’t something I felt able to do often.

• Remote tagging analysis. Sometimes I used the team to tag a large volume of research data. This took a bit of preparation on my side because I needed to make sure the tags worked and explain how to apply them. When it worked it exposed the team to research they would not have otherwise paid attention to.

And these were the things that I did on my own:

• Preparing materials for analysis. Whether it was sorting through the post-its generated by my team, cutting up my own notes onto index cards, or transcribing my recordings, there was a lot of solo work preparing for analysis. This work remained mine whether I was doing analysis on my own or with the team.

• Hunting down that last 20%. Finding only the biggest problems was OK some of the time but most of the time I needed to go deeper. In these situations I would do the hard, deep work of analysis myself. Or I’d take what the team had done and then go back over it to fill in the gaps.

• Writing clear research findings. Throughout analysis I am turning noun-based clusters of observations (e.g. “password problems”) into clear finding statements (e.g. “the password rules are so complex people can’t remember what they entered”). This last step - writing a bold, clear research finding - is the critical moment in my research. It’s something that only I will ever take responsibility for.

That’s what it means to me to do analysis as a team sport. Most of the time you do analysis on your own, but sometimes you involve your team. Either way you - as a specialist user researcher who owns the process and is accountable for the results - prepare the materials and take responsibility for writing clear research findings.

4. Communicating research as a team sport

Communicating research is about taking the research findings and inserting them into the minds of the people making decisions. This might be standing in front of a wall telling stories, it might be presenting slides, or it might even be a big old report.

These were the best ways I found to involve my team in communicating research:

• Whole-team dedicated research playback. Every two weeks I would gather the whole team for 30 minutes and share findings from the research. This always felt like it came around too often but it forced me to get better at sharing findings earlier. It also helped me see whether the research findings had impact. There’s nothing like watching someone’s eyes close as you are talking to them.

• Prioritising research findings together. When my team bought into the findings prioritisation they responded to the research. So I ended every playback with an invitation to correct my prioritisation. These conversations forced the team to reveal what mattered most to them. It also meant that I could walk away at this point and give the team creative freedom to solve the most important problems.

• Critiquing the user research process. I used to run a research retro every four or five playbacks. I asked each team member to post up what worked well, what didn’t work well, and what they would change about the research. This was exposing at first - it’s hard listening to non-researchers critique your practice - but it ended up as my favourite part of the whole research process.

And these were the things that I did on my own:

• Doing the first-pass prioritisation of findings. I would often have upwards of 50 findings from analysis. But I knew my team would rarely pay attention to more than 10. It was my job to work out which 10 findings to share. This is an art.

• Preparing a compelling playback. My team were often grumpy about being collared to listen to research for half an hour. So I spent a lot of time thinking about better ways to communicate things to them.

• Documenting findings for the future. I involved the team to make sure the research findings got into their heads. But I also had to document the most important findings so that they were available for other teams or future members.

That’s what it means to me to communicate research as a team sport. You stand up and share your findings but only after you - as a specialist user researcher who owns the process and is accountable for the results - have prepared a compelling playback.

‘User research is a team sport’ is a big ask

That’s my field guide to doing ‘user research is a team sport’. I have to be honest with you though. Doing research in this way is hard work. It means we need these things:

• Confidence in our own abilities. Once we open up our practice to our teams they ask difficult questions. We need to feel secure about our core practice.

• Clarity in our communication. Whether coaching our team to take better notes or preparing for our playbacks, we need to be able to express ourselves clearly.

• Ease with facilitating groups. For many researchers this is the hardest part. Our teams are not expecting to take part in research. We have to find ways to make it engaging, useful and fun. This means being able to facilitate reluctant groups.

• Humility to accept we’re often wrong. The smart people we work with will show us many ways where our approach isn’t right. We need to be able to hear this.

These things are hard for us. Lots of us are introverted, feel anxious, or have imposter syndrome. Opening ourselves up to critique, or dragging grumpy people away from their screens, is challenging. This is a tough journey for all of us.

It’s the right journey though. At a certain point, being a better user researcher is not about learning new research methods, it’s about finding ways to increase the impact of the work you are doing. That’s what ‘user research is a team sport’ is about.

There’s one more thing that is hard. It can feel like we are giving up control of the quality of the research. This is not even close to being true. The researcher finalises the research questions, creates the research plan, writes the discussion guide, moderates the sessions, has the final say on the notes, determines analysis approach, and is the one who writes research findings. Our control remains intact.

There are many ways to play our team sport

The stories I’ve shared here are my personal take and I’m often wrong. Yes, I’ve seen this work for me and the researchers I’ve supported over the years. But time and again I’ve learned from other researchers who have different ways of doing things and who disagree - often vehemently - with the things I say.

That’s OK. There is no One True Way. All models are wrong, but some are useful.

I wrote this partly to make it clear that ‘user research is a team sport’ is not at odds with being a specialist user researcher. These things go hand in hand.

But mostly I wrote this to pass on a framework for thinking about where to involve our teams and where to work alone. ‘User research is a team sport’ is the most powerful thing I’ve learned in government and I’d love to see more of us doing it...

Let me know what you think on @myddelton. And thank you to the researchers, developers, product managers, designers and other team members I’ve worked with. You taught me these things, one halting step at a time, and I’m forever grateful.

40 lessons

I turned 40 last month. I went through a lot of changes in my 30s. Last year someone told me I should write about lessons I’d learned. This seems like a good excuse.

These lessons are personal and specific to me. They are not an exhaustive list. In the future I’ll learn new lessons and disown some of these. Most importantly, these lessons are not a set of instructions for anyone else. Each life has its own lessons.

Truths about myself

#1. I owe more to luck than hard work. Part of me thinks I’m solely responsible for my successes. This isn’t true. Fooled by Randomness showed me that it’s mostly down to luck. I try to keep this in mind.

#2. I am highly privileged. I’m white, male, middle class, straight, tall, able-bodied, and young(ish). I’m WEIRD - western, educated, industrialized, rich, democratic. Realising how privileged I am made me deeply uncomfortable until Caroline Jarrett told me that my responsibility was to use my privilege to help others. I try to do this.

#3. I am unfashionably sincere. I’m enthused by all sorts of things. I gush admiration for people and ideas. I’m bad at postmodern cynicism. This sincerity didn’t work well at school and I quickly had it knocked out of me. A long time later I realised I’d been suppressing part of who I am. I don’t suppress that part any more.

#4. I’m not great at being tribal. My life is full of tribes. Spurs and Arsenal. Labour and the Tories. Leavers and remainers. Designers and researchers. House music and rap music. I’m amazed at how sure people are about their tribes. My mum once told me that many left-wingers have big ideas about public good but turn out to be terrible people in private. Ever since I can’t help but see the good and bad in every tribe. I particularly hate that in-group out-group bullshit.

#5. I am creative, but I’m not an artist. My dad is a jazz musician and values creativity above all. I spent a long time trying to be a musician in my 20s. Eventually I realised the artist life wasn’t for me. Freed, I fell in love with design because it unlocked my own creativity. I make new things. I play with ideas. I find ways to rethink problems. I love it.

#6. I am an explainer through and through. When I studied history I was scolded for writing grand narratives. These days I wrap up what I know about user-centred design in neat models. The Beginning of Infinity made me feel better because David Deutsch shows how better explanations transform the world. On the flip side, I know that I veer into mansplaining at times. Pull me up on it if you see it.

#7. I’m not really sure of what I know. Thinking Fast and Slow had a huge impact on me. All those cognitive biases blew my mind. I am haunted by the idea that I don’t quite know what I think about things for real. I’ve found it to be a liberating and exhilarating way to live life.

Truths about the world

#8. It's better to be kind than to be smart. At school I was that obnoxious kid that got good grades and made too much of it. It wasn’t a great way to make friends. I’ve spent my adult life undoing that side of myself. Anne Galloway was given this advice on entering academia. “We're all smart. Distinguish yourself by being kind”. I love that.

#9. Black and white thinking is rarely helpful. The truth is nearly always in the grey areas. The problem is it’s easier to communicate when you adopt extreme positions. False binaries are more compelling than ‘it depends’. One way around this is having strong ideas, weakly held. Another is that all models are wrong but some are useful.

#10. The dying tell us important things about life. My girlfriend’s dad died a few years ago and this made a big impact on me. Around then I came across The Top Five Regrets of the Dying which are: not living a life true to yourself, working too hard, failing to express your feelings, losing touch with your friends, and not letting yourself be happy. I’ve been using these things to think about my life ever since.

#11. People solve their own problems. For years I would listen to someone’s problem, tell them how to fix it, and then get annoyed when they didn’t. I still do this. Most of the time it doesn’t work. People don’t act until they’re ready to act. Maybe you’ll be there when they do and you can help. Probably not.

#12. Learning to say no is important. I’ve realised in the last few years that deep down I want to please people. Often I say yes to things that stress me out later on. My counsellor asked me to say no to ten unimportant requests in a week without saying sorry or giving an explanation. Nobody stopped loving me. That was a turning point.

#13. Bad behaviour often comes from hurt and trauma. A friend who works with vulnerable young people told me bad behaviour comes from hurt and trauma. I can’t unlearn this. I see it everywhere. It changed how I deal with humans behaving badly. 

Personal lessons

#14. It took me a long time to deal with my parents breaking up. It happened when I was 11. I wasn’t aware I hadn’t dealt with this until my girlfriend helped me face it in my 20s. I didn't properly make peace with it until my 30s. It’s the background to my life.

#15. My long-term relationship isn't what I expected. I’ve been with my girlfriend for 17 years. We’ve changed lots since 22. Being in love with someone for a long time is poorly represented in art and culture. Don't expect your love to follow those plotlines.

#16. Showing my vulnerability has been surprisingly OK. I’ve got used to talking about my mistakes and fears. It doesn’t emasculate me or leave me open to attack. It does help me build trusting relationships quicker. These are my favourite kinds of relationships.

#17. Talking things through helps me. Counselling, coaching, discussing things at work, going for a drink with a friend, or just talking to myself. All bring different perspectives to my problems. That perspective shifting is what helps me step past blockers in my life.

#18. Self care isn’t some new age bullshit. Ten years ago I laughed at self-help books. Then someone close to me revealed they had helped him. I read some. They helped me too. I meditate and write morning pages. I work four days a week to balance work with life. Life is a long road and I need all the help I can get.

#19. I cry a lot more these days. Films. Adverts. Books. Podcasts. Songs. Not because I'm sad but because I feel things more. And because I'm less bothered about playing the role of a man. It feels good. It’s addictive.

Work lessons

#20. Bringing new things into the world means living with failure. It’s taken me a long time to realise that I am drawn to designing new things more than improving existing things. The awkward thing is that most new things fail. My ten year design career is full of hard work that has gone nowhere. I’ve had to learn to live with this.

#21. My biggest impact is on the people directly around me. My life has been a journey from trying to impact millions through music, to trying to impact millions through design, to trying to impact the handful of people I work with. This is less ambitious but is meaningful and rewarding in ways I never expected. Maybe my work will impact millions one day. Maybe it won’t. There’s luck involved. I’ll always know I’ve been part of these people’s lives and that matters to me.

#22. Communication is everything. I was lucky to spend three years in a comms team at CABE. It taught me a huge amount about how to be clear. Working as a designer taught me more. But it never ends. Every time I stop paying attention to how I’m communicating I slip back into old bad habits. There are no shortcuts.

#23. There are hidden depths to listening. Some people don’t listen at all. Some listen for you to finish so they can speak. Some listen to your words as an excuse to tell a related story about themselves. A few people listen carefully and ask follow-up questions. Very few people do empathic listening. This is when you listen to someone and, instead of asking questions, rephrase the content and reflect the feeling of what the other person is saying. It avoids putting your own views into the conversation. It is incredible what it opens up. But it’s hard and goes against all our instincts.

#24. Criticising people in public is usually counterproductive. I’ve made this mistake too much at work. Many people I otherwise respect make this mistake too much on Twitter. Mostly it's better to praise in public and criticise in private. It’s not a universal rule though because at times you need to speak truth to power.

#25. Facilitating groups is a great way to get stuff done. Pete Gale showed me this. Gamestorming and Liberating Structures gave me techniques. I know it looks scary but we work in teams and this is a core skill that can be learned. It’s been so much easier to get things done in organisations now I’ve learned to bring people with me.

#26. Competition is harmful to working together. I grew up competitive. Not just grades but board games, quizzes, sports, ideas, conversations, anything. Through working with other people I’ve learned this isn’t helpful. I’m sad that our society encourages so much competition between individuals. Especially when it leaves us comparing ourselves to each other and coming up short.

Money lessons

#27. Money and power nearly trapped me in the wrong career. How To Do What You Love inspired me to walk away from big money with big pharma at a crucial time in my life. Big money doesn’t sound like a trap. But if you use it to get a big house then the big mortgage payments mean it’s difficult to step away from that career later on. You can’t change your life it you don’t leave your options open.

#28. Separating my spending from my salary was a turning point. Early on in my career I lived month to month. Once I got to £45,000 I had enough to live on and since then every payrise has mostly gone into savings. This is a double bonus - I save more towards retirement, and when I get there I’ll need less pension income to live the same life that I do today. Plus I’m not tied to the higher salary if I need to move.

#29. No one else will negotiate my salary for me. Someone told me “you don’t get paid what you’re worth, you get paid what you negotiate”. Sad but true. If you don’t negotiate your salary you’ll get less than other people who do. The good news is that once you’re setting salaries you can make it fair. Especially for women you hire.

#30. No one else will plan for my retirement. The generation above mine had better pensions and didn’t worry about retirement. I don’t have that luxury and had to learn about investments. My approach is easy - open a Vanguard ISA, start a monthly contribution (whatever is affordable), put it into a Vanguard LifeStrategy fund, increase the contribution with every payrise, and don’t touch until retirement.

Change lessons

#31. Change is possible. People say “people don’t change”. Not true. I’ve changed a huge amount in the last ten years. I’ve watched others change too. No, it’s not easy. And, yes, some things about people don’t change. But people do change.

#32. I have two selves and they fight over changes. One is a rational superbeing and wants to eat properly, go swimming, and write important things. The other is a human mess and wants to eat McDonalds on the sofa watching true crime. My first self is active at the start of each day/week. My second self dominates evenings and weekends. They are both me. Any change must deal with both selves.

#33. Commitment devices help with change. A commitment device lets the first self (rational superbeing) set up a commitment that the second self (human mess) can’t easily break. For example, when I want to stop checking Twitter I set up two factor authentication using my girlfriend’s phone so I can’t check without the shame of asking her. I have plenty of other commitment devices. It’s a powerful concept.

#34. Habits help with change. We have a finite amount of willpower. This is one reason change is hard. The most effective way to use our finite willpower is to create new habits. Ingrained habits stop needing willpower to maintain them. Then you can use that willpower on the next thing. This is a strong, workable theory of change.

#35. Focusing on what works helps with change. My natural tendency is to focus on what is broken. Switch showed me that it's usually more useful to look at what’s working well and imagine ways to expand that. This applies to work problems just as much as personal problems. Find the bright spots and amplify them.

#36. Things often get worse before they get better. The change curve shows this vividly. Although originally conceived for coping with death it applies to any situation where you experience a shock. It shows up when you form new teams. It shows up when you start managing people. It means we are most vulnerable just after we start to make a change. Knowing this helps me deal with its effect.

#37. Change is all about getting up and going again. Because things get worse before they get better I often give up quickly. Even when I persevere there is backsliding and falling off the wagon. I’ve learned not to beat myself up for this. Failure is part of change. Learning to get up and go again is a skill you can develop.

Final lessons

#38. Hedonism is problematic. Spinal Tap had a keyboard player called Viv Savage whose motto was “have a good time, all the time”. When I was younger I saw this as heroic. These days I see a sadness in it that wasn’t visible to me then. I’ve watched friends suffer with drink and drugs. It’s taken me a long time - perhaps too long - to accept the dark side of having a good time all the time.

#39. MeToo made me re-evaluate my life. I consider myself a feminist. I’ve even been a bit smug about this. Hearing stories around #MeToo made me question my own actions in a way I’d never done before. It made me think about what it means to be a man in a patriarchy that runs this deep. Deeply uncomfortable. Long overdue.

#40. There are only two types of music. I’m going to end with this because it’s my favourite of all the lessons. My friend Timber is a DJ with strong feelings about music. Very definite about what he likes. So it shocked me to the core when he said this.

There are only two types of music.
Good music.
And music you don’t understand yet.

I’ve had this in my head ever since. It’s arresting and beautiful in form. Even better, it opens this beautiful understanding that the things you know are only the things you know, that there are other things that other people know, and that it’s better to marvel at our differences and seek to understand them than it is to make fun of them. Let’s face it, we all need more good music in our lives…

Let me know what you think on @myddelton. And tell me if you think there are new lessons I need to learn. There always are, right?

What, So What, Now What?

The user research process is about going from observations to findings to actions. Step-by-step. In that order. This is obvious to long-time researchers. It’s surprisingly invisible to people new to user research.

For example, when we take notes in a research session we only note down observations. But if you ask someone new to research to take notes they’ll write a mixture of observations (participant struggled to enter a password), findings (hiding passwords stops users logging in) and actions (show the password!).

Mixing up these things causes us to jump to solutions too quickly. Or come up with general findings from too few observations. These things waste our time later.

I use What, So What, Now What to explain the user research process. It's about going through three stages, one at a time, pausing to reflect and prioritise after each stage:

• What?
  What happened? What did you notice? What facts or observations stood out?
• So What?
  Why is that important? What patterns, conclusions or hypotheses are emerging?
• Now What?
  What actions make sense?

What, So What, Now What is lifted directly from a facilitation technique used to debrief and wrap-up up workshops and meetings. It also happens to be an excellent way of thinking and talking about the user research process.

Cutting through our research jargon

One reason I love What, So What, Now What is that it helps people understand all the different, abstract words that we use in the user research process.

• What?
  Observations. Quotes. Videos. Photos. Facts. Notes. Clips. Transcripts.
• So What?
  Findings. Insights. Hypotheses. Theories. Reckons. Meanings. Explanations.
• Now What?
  Actions. Recommendations. Solutions. Next steps. User stories. Fixes. Designs.

Look at how many ways we have of talking about the same things! I'm sure you could add more. It's much clearer to talk about this stuff using What, So What, Now What.

Helping us prioritise effectively

Another reason I love What, So What, Now What is it shows that there are two different prioritisations that matter in the user research process.

The obvious one is prioritising the actions in the Now What stage. This prioritisation determines what work we do as a result of user research. It tends to be done by designers and product managers. We pay it lots of attention. It informs our backlogs.

Less obvious, but more important, is prioritising the findings in the So What stage. When teams jump ahead to solutions without doing this they come up with actions that fix unimportant or irrelevant problems.

People will always want to jump ahead to solutions. That's natural. It's our job to stop them doing this but, honestly, it's hard to explain why this is a bad idea. What, So What, Now What is a powerful explanation to arm yourself with for these discussions. 

Clarifying what researchers and designers do

The biggest reason I love What, So What, Now What is that it helps me explain the relationship of researchers and designers. When you split these roles it’s important to be clear about who does what in the user research process. How we work together. 

Here’s my take on how these relationships work in a perfect world. 

• What?
  Owned by researchers. We do research that generates good observations. 
• So What?
  Owned jointly by both of us. We create and prioritise findings together.
• Now What?
  Owned by designers. They decide on actions to address the research findings. 

Collaborating on the middle bit helps us get to actionable insights. The holy grail of user research. Researchers bring their understanding of human behaviour. Designers bring their knowledge of design problems. We do beautiful things together.

The big lesson for researchers new to product teams - especially those of us who used to work in agencies - is that we shouldn't usually be making recommendations. That's not our job, we're not best placed to do that, and it steps on others' toes.

Supporting less experienced researchers

In reality we don’t always work in a perfect world. There is often a mismatch in experience between researchers and designers on teams.

Let’s start with my tribe. Less experienced researchers struggle to create actionable insights. We report things that have little relevance or significance. Designers don’t know what actions to take as a result of the research findings.

In situations like these, I encourage experienced designers to take a leading role in shaping the findings during the So What stage. This can help researchers learn how to create actionable insights in future. It takes time to learn how to frame these things.

New researchers sometimes need help earlier too, in the What stage, because they’re not able to plan and do research that leads to good observations. Experienced designers often have good ideas about how to improve this research. I encourage them to share these ideas as suggestions.

Supporting less experienced designers

Less experienced designers struggle with actionable insights too. They often focus on findings they can imagine solving and deprioritise the rest. Big things can get brushed under the carpet because it’s not clear how to address them.

Here, I encourage experienced researchers to take a leading role in prioritising the findings in the So What stage. This can help designers learn to live with important findings that have no easy solution. This takes practice.

New designers sometimes need help later too, in the Now What stage, when the actions they take fail to address the research findings effectively. Many experienced researchers have good ideas about how to solve these problems. I encourage them to share these ideas as recommendations with the designer.

Adapting this to our everyday world

OK. These examples are a bit black and white. Let me cover some of the grey areas.

One is that it’s not just researchers and designers. Product managers, developers, and other team members are involved. What, So What, Now What flexes to include these people too because mostly (not always!) they act like designers here, in the sense that they're mostly interested in creating actions. Our teams love action.

Another is the idea of ownership. I said that researchers ’own’ one bit and designers ‘own’ another. Ownership is important for clarity of roles, yes, but user research is a team sport and so is design (at least in the sense of deciding which actions to take). I expect ‘owners’ to collaborate. In mature, experienced teams with high levels of trust I'd expect to see researchers and designers collaborating at every stage. 

The final is that these things are fluid, contextual and situational. No one way of working will work at all times. Even on the same team with the same people. It is up to researchers, designers and teams to make What, So What, Now What work for them. Let me end with an example that illustrates this from my own career...

Ending with an example from GOV.UK Notify

I worked as a user researcher on GOV.UK Notify with exceptional designers and product managers who made great decisions based on our research. 

There was one time when their actions were not addressing the research findings. We saw the same problems again and again. I agonised about intervening because I’d been clear that my role ended with creating and prioritising the findings together.

Eventually I chose to act. I explained that I was going to step, briefly and exceptionally, into Now What territory and suggest actions to solve the problems. I told them they could ignore these. But they listened and our team moved forward.

This is why What, So What, Now What is so useful. It not only explains the user research process and clarifies our roles, it also gives us a framework to talk about the inevitable exceptions. It’s OK to break the rules if you know the rules in the first place. 

Try using What, So What, Now What the next time you explain the user research process to someone. I think it'll help you have clearer conversations.  

Let me know what you think on @myddelton. If you ever run workshops you should try the What, So What, Now What exercise from Liberating Structures. I have a suspicion this pattern works in more places than user research and workshops too…

Three types of user research

I’m a lead user researcher at GDS. My job is to support our user researchers in helping their teams make better products. This means I think a lot about the relationship our product teams have with user research.

Over the last two years I’ve stumbled into a useful model for talking about this relationship with researchers and their teams. The model helps them understand what to expect from each other, recognise and support each others’ strengths, and work together to make better products.

The model? There are three types of user research product teams should care about:

1. Testing things the team have built
2. Working out what the team should build next
3. Understanding potential users and their lives

Not revolutionary. Not innovative. Just helpful in doing the trickier parts of my job. Like adding a new user researcher to a team, making the role of our user researchers clear, and helping our teams get value from research.

1. Testing things the team have built

For many people, doing usability testing on existing software is all there is to user research. It is the easiest type of user research to grasp. So this is where we start.

The benefits are clear. It helps find and fix problems before you release software to the world. It is easy to get started for researchers (easy to learn) and for teams (easy to prepare to do). It fits neatly with agile because you can slot it into every sprint.

The problem is harder to see. Building software costs lots of money. Even in agile. This sunk cost makes it hard to tackle the big problems hard-coded in architectural decisions. This limits the scope of designers to make necessary changes. At its worst, teams make tiny, insignificant changes and ignore the bigger, more important stuff.

User researchers and their teams must be alert to this. If big problems are showing up in testing, teams should be spending more of their research effort on working out what the team should build next...

2. Working out what the team should build next

Research can help you figure out what to build before you start building. This mostly involves interviewing users about specific tasks and watching them use prototypes.

The benefits are strong. You understand the big architectural choices before sinking money into building them. You empower designers by giving them space to explore multiple options and do rapid iterations. This leads to huge progress, quickly.

There are two problems that block teams from doing this well:

• At one extreme, teams avoid this work. Maybe because it doesn’t fit neatly with agile. Or because they won't invest their developers' time in prototyping. 
• At the other extreme, teams get stuck doing too much prototyping. They want all the answers before they build. Prototypes can never give you all the answers.

This is the hardest research to get right. You must balance doing too little (too risky) against doing too much (too slow). You have to work closely with designers, technologists and product managers. There is an art to this.

Although it’s the hardest research to do right, it’s not the most harmful to get wrong. That dubious honour falls to not understanding potential users and their lives...

3. Understanding potential users and their lives

Understanding people that might use your product (not just current users) is about listening to their stories and observing their daily lives. Away from your product.

The benefit is ensuring you have a workable proposition. This research can trigger innovative ideas, establish whether a market exists, and even help you pivot to new users when you need to. It’s worth saying that the findings from this last a long time too. People’s lives don’t change as fast as your product. This is worth doing well.

The problem is that many teams don’t value this work. Early on, they’re sure their assumptions are right. Later, they’d rather avoid knowing their assumptions were wrong. They say this work is a waste of time because it doesn't focus on the product or lead to immediate action. This work doesn't fit neatly into sprints, either, and things that are hard to manage often get ignored.

Let's be honest though. User researchers can get too fascinated by people and their lives. We sometimes get lost in this work for its own sake. Maybe some complaints about this work being a waste of time are not baseless. We should think on that.

Whatever, the most frustrating thing about my career has been watching teams of talented people working on things with flawed propositions. User research doesn’t have all the answers, no, but remaining unaware of hard truths about people and their lives is a pretty reliable way to undermine your product.

---

OK. That's the model. As Simon Wardley would say, all models are wrong, but some are useful. Here are three ways this model is useful to me.

Adding a new user researcher to a team

Talking about these three types of user research is useful when a new user researcher joins a team. This happens a lot in my job.

• If the team is starting a new thing, focus the user research on understanding potential users and their lives. No discovery should be building protoytypes, let alone working software, so there's no choice but to start here.
• If the team is underway but doesn't appreciate user-centred design, focus the user research on testing things the team has built. All teams grasp this and it’s a great way to build the trust you will need later.
• If the team is up and running smoothly with user-centred design, focus the user research on working out what to build next. In mature teams with good propositions, the biggest value that research can bring is to find the riskiest thing on the roadmap and work on that with the designer and product manager.

It’s helpful to have this clear focus when you join a team. Over time the researcher’s job is to expand to cover all three types of user research.

Making the role of our user researchers clear

I tell our user researchers that their job is to make sure that all three types of research are being done by their teams (except in discovery of course). Framing the role like this gives our researchers plenty of freedom to come up with their own approaches to their work. This has worked out well for us.

Beyond that I don’t much care what methods they use or how often they do research. I want to hear how they are balancing the three types of research on their team and whether their teams are blocking any of the three types. This is a good conversation.

Researchers tend to prefer one of the three types. Some are most comfortable testing, some love getting in there with designers and prototypes, and others just want to explore the wide open spaces of people and their lives. It's fine to have a preference, but I expect researchers in product teams to do all three well. So the three types are a useful tool for conversations about professional development too.

Helping our teams get value from research

Finally, talking about these three types of user research lets me have better conversations with product teams about how user research can solve their problems.

• Some teams get obsessed with building things above all else. This can be costly as they’re not using design to de-risk development properly. I’ll ask them to slow down and go back to working out what to build next.
• Other teams get paralysed working out what to build. This can be costly as they're wasting too much time on certainty. We are gamblers, not scientists. I’ll push them to take a punt so they can start testing things the team has built.
• We’ve all been on teams with a flawed proposition. This is not only costly but also emotionally difficult. I think of James Baldwin, who said “not everything that is faced can be changed, but nothing can be changed until it is faced”. I think of Rebecca Solnit, who said “we prefer ugly scenarios to the pure unknown”. Then I’ll try and help teams face the unknown by understanding users and their lives.

These are difficult conversations to have. Honestly, I've not always been successful in having them. But it helps me, in these conversations, to have a model of the value that user research brings to our product teams.

---

Sharing this model with you

I have thought long and hard about sharing this model. Leisa Reichelt, who I admire more than any other user researcher in the world, flatly told me that we didn’t need yet another way to talk about user research when I shared it with her six months ago. So I went away and wrote about discoveries instead. Thank you Leisa.

But I haven’t been able to let this go. On my good days I think this is because it’s a model with universal applicability. On my bad days I worry that it’s because - like all designers - I’ve fallen in love with my own idea.

All I know now is that it’s been useful to me. Maybe it will be useful to you too.

Let me know what you think on @myddelton. One final thing. You may be wondering what is wrong with discovery, alpha, beta, live as a model. The problem is people end up equating each stage with a type of research. Given that teams spend most of their time in beta and live, it's harmful to think they should only be doing usability testing...

I've made mistakes

For my end-year review one of our user researchers asked me to talk more openly about the mistakes I’d made throughout my career. She said it was important for senior people to show that they didn’t have all the answers. 

I loved this feedback. It struck a chord with me for another reason too. 

I’m about to leave GDS to go and work for the Home Office. I’ve been wondering how to mark this change. So over the last week I’ve been giving this talk to say goodbye.

My GDS best bits

Even at my most open and vulnerable I can’t just post up my mistakes! I’m human after all. So here are the five things I’m most proud of from my time at GDS.

• Working as a user researcher on GOV.UK Notify
  When I joined GDS I spent four months working on GOV.UK Notify as a proper user researcher. It was the first time I’d ever worked on an agile product team. I loved it. It taught me a huge amount in a short time. 
• Helping GDS think about service teams as users
  I stood up in front of hundreds at an all-staff and talked about the idea that civil servants were our users. It was terrifying. I said difficult things. But people listened. That’s when I fell in love with GDS.
• Understanding the user needs of service teams
  Given civil servants were our users we needed to understand their needs better. We interviewed 150 teams from huge online services to tiny offline transactions. This work changed how we think about our users.
• Exploring the biggest unmet need of service teams
  Three things are common to all services - publishing to all (GOV.UK), talking to individuals (GOV.UK Notify) and collecting information (unmet). We ran the Submit discovery to look at collecting information. It’s the best work I’ve ever done.
• Building a team of user researchers
  The reason it’s OK for me to leave GDS is the team of researchers we built. We took risks and hired non-seniors. We trusted and supported them. Now they’re flying on their own. Could. Not. Be. More. Proud.

My biggest mistakes at GDS

Now that’s all out of the way let’s get down to these mistakes.

#1. Ignoring hypothesis-driven design for too long

I’ve always thought in terms of research questions. I was wary of hypothesis-driven design because I worried that hypotheses are ‘validated’ or ‘falsified’ with too little attention paid to the details. But I’ve learned that hypotheses are a great way to link problem-oriented researchers with solution-oriented designers and developers. 

#2. Failing to write up my most important research

After interviewing 150 service teams we presented our results. We got a lot of people excited. We took that momentum and carried it into a discovery. But what I failed to do was stop and take the time to write up those results for others to read later. I get requests pretty much weekly for the write-up. It’s still on my to do list. Huge mistake.

#3. Holding alpha workshops when I wasn’t ready

At the end of our Submit discovery we held design workshops with people from across GDS. We wanted to use their creative thinking to widen our alpha pool. But I wasn’t ready to hold these workshops. Instead of cancelling, I went ahead and ended up talking at these talented people for three hours. Wasting their time. Horrible.

#4. Leaving the Submit team after discovery ended

This one is hard to admit. On the Submit discovery I was the product manager on top of my normal role as lead user researcher. We finished with huge team momentum. But I decided to walk away and go back to my normal job - for good reasons - and I became part of the reason that momentum was lost. Momentum is a precious thing.

#5. Forcing researchers to recruit participants

Civil servants are our users. We can’t recruit using agencies. We can’t pay incentives. The best answer two years ago was to recruit people ourselves. But I failed to tackle this at a structural level. Now our 10 researchers spend too much time recruiting participants. I was scared of the size of the problem to solve. That’s a poor excuse.

#6. Doubling up our researchers without thinking

I introduced a senior and non-senior researcher to some teams. A move that let us recruit juniors, open career progression, respond to new projects, and be resilient when people left. But I didn’t think about how our researchers would get on with each other or split work between them. We’re humans. Humans are complicated. 

#7. Failing to record sick leave and annual leave

This one sounds like a little mistake but was a huge one. I pride myself on being a good manager. But I didn’t always log sick leave and annual leave properly for the people I managed. Knowing what I now know, this is unfair to the people I managed and undermines much of my other good work. I’m never making this mistake again.

#8. Talking down the work of our teams in public

My worst mistake to own up to. Part of my job is to critique the product strategy of our teams. This is a natural - if difficult -  conversation to have with the team leaders. But there were times where I let my frustrations show to other members of the teams. I upset people I cared about. I burned bridges I needed and they needed. I’m sorry.

#9. Failing to sell ideas to our senior management

The Submit discovery felt unstoppable to me. I thought it responded to a clear user need, had significant cost benefits and opened up strategic opportunities. But I got carried away by being too close to it and failed to communicate these things to our senior leadership. I’m still kicking myself for this. Because I know how to do this.

Other mistakes in my career

In writing up my biggest GDS mistakes I realised something. Many things that I did well at GDS were linked to mistakes I'd made beforehand. These are some of them.

#10. Writing discussion guides at 2am 

I did a lot of this at cxpartners. I used to say this was because I wanted to spend time working on the prototypes. But these days it feels more like a control-freak mechanism to ensure I didn’t have to take feedback into account. At its worst, my intern turned up to moderate his first research session to find me still writing a guide.

#11. Researching my designs in a leading way

I used to be both researcher and designer. I was an excellent researcher when I was testing designs made my other people. I found all sorts of problems. But when I tested my own designs they magically performed much better. Yes, I was a good designer, but no, this wasn’t the whole truth. Don’t mark your own homework.

#12. Presenting a huge list of problems to people

When I started out as a user researcher I wanted to hold on to every observation. Partly because I felt this was the way to prove myself. Mostly because I didn’t have any idea how to prioritise. But if you present a huge list of problems to a group they’ll smile, nod, and ignore the whole list. Much better to show them four or five things.

#13. Expecting people to work the same way I work

When I started managing people I was out of my depth. I tried to turn people into clones of me. Leaning over to edit documents. Holding them to timelines I’d set myself. But people don’t respond well to being managed like this. State the goal, let them go, trust them, and give useful feedback. Let them make their own mistakes. 

#14. Sharing the wrong things with people

Our society and culture encourages us to be transparent and open. To avoid being ‘two-faced’. I tried this with the people I managed. But it unsettled them when I told them things that were unclear or too far outside their normal world. Now I take this advice from a trusted friend - be judicious with information, emotion, and opinion.

#15. Being angry at others when out of my depth

At cxpartners I was handed an impossible project. I tried to deliver it and spent months being furious with the people who set it up. But what I didn’t realise was that I had the power to say no. That though anger is a useful sign that something is wrong, it’s rarely a useful thing to hold onto for any length of time. I’m still learning this.

#16. Binding my identity to my work too closely

Lots of these mistakes come from linking my self too closely to my work. I worry that saying no will stop people loving me. Once upon a time this was useful because I had loads of capacity. But these days there are far more requests than I have time to spare. Learning that the world doesn’t fall apart when I say no has been huge for me.

#17. Burning out in 2014

I've written before about burning out. I used to think that I could cope with anything work threw at me. But during the impossible project, trying to make a webfont work  at 3am, I broke down in tears. I went to New Zealand for four weeks and spent half of it recovering. I did counselling. I had coaching. I learned to deliver the impossible project. It took me six months to regain my confidence. Then I came to GDS.

So, yes, I’ve made mistakes

We all make mistakes. All of the time. Every one of us. It's unavoidable. 

It doesn’t matter that we make mistakes. What matters is whether we are able to recognise the mistakes we made. This is hard work for lots of reasons:

• It’s not always possible in the moment.
• It requires us to take a long, hard look at ourselves.
• It means putting aside our anger with others.
• It can involve someone being brave enough to bring something to our attention.
• It means we need to find space to reflect on things away from everyday work.

After we recognise our mistakes we can change things. Of course we will always make more mistakes. But at least we won’t be repeating the ones we made before.

I hope that sharing this prompts you to think about the mistakes you've made. And then, having done this, to try talking them through with someone you trust. I think you’ll be surprised at how helpful people can be when you do...

Let me know what you think on @myddelton. Thanks to Amber Westerholm-Smyth for prompting me to do this and Mia Kos for once saying we should talk more about our failures. Finally, I want to give a huge thank you to the wonderful people at GDS for giving me the space to make these mistakes and learn from them. I will miss you all.