In our last post, we argued that adding some basic user research skills to your data team’s toolbox can have a big impact on the usability of the data products you build. Ultimately, this enables you to add more value to the organization. In this post, we’ll dig deeper into methodologies you can start with and concrete steps to make your research program a success.
The five-step user research process resembles quantitative data experiments
How do user research practices work? Regardless of your specific research methodology, the process can be broken down into a series of steps that will be familiar to data teams focused on quantitative data experiments:
- Outline your objectives. Clearly map out the questions you want to answer that will be impactful to your team. For example, why are stakeholders not using the self-service data platform?
- Define hypotheses. Record the assumptions you hold about your stakeholders so that you can challenge them via your research. Stakeholders might not be using the data platform because they have a difficult time finding the dashboards they’re looking for. Or perhaps you think your users are overwhelmed by learning a new tool.
- Choose a method. What might be the best user research method to address your question? And what method can be achieved with the resources available (time, team)? We explore options more below. Following our example, user interviews or a survey could be very useful to understand your team’s usage blockers.
- Conduct the study. Build a research plan, create the materials you need for the study (such as an interview moderator’s guide or survey), determine who should participate and recruit participants, and gather the data using the method you’ve selected.
- Synthesize the data. Make sense of the data you’ve gathered to color in the knowledge gaps and disprove or prove hypotheses. Often you may have a small dataset, having interviewed a handful of users. Aim to find trends within that data and distill your learnings, but keep it at a human scale. Sometimes you can understand as much from a single user quote that gets to the heart of the issue, as you can from a countable trend. With your results, create a plan of action to integrate learnings into your data platform.
These five steps can form a loop. Once you have synthesized your data and applied the learnings to your product, you may want to perform another round of research to learn whether your changes have resolved users’ pain points. If so, you’ll start back with outlining your new objectives.
Choosing a research method that’s right for your data team’s questions
If you do a quick search for “user research methods,” you’ll find many (here’s a good overview of thirty four methods!). To help you get started, we’ll highlight a few that are particularly applicable to the types of questions you might have about your data products:
User Interviews
User interviews are semi-structured one on one conversations that enable the collection of deep insights from a small number of people. They are great for probing into your users’ backgrounds, general goals and motivations, and workshopping a solution to a problem. Prepare a discussion guide with the questions you have to standardize the information gathering process, but be authentic and engaged during the interviews. Ideally avoid closed questions (those with a yes or no answer) as you’re looking to gain deep answers.
User interviews are, in our opinion, the single most useful research tool for data teams. They enable you to take a focused moment to understand your stakeholders’ point of view more deeply. This both builds trust within that stakeholder relationship and illuminates insights you may not hear in day to day interactions. Bringing together a relatively small number of user interviews can quickly highlight trends.
User interviews are, in our opinion, the single most useful research tool for data teams.
You can even structure existing meetings with stakeholders to include a few thoughtful questions that help illuminate their pain points. This creates a lot of value and insight for your team without having to take time to specifically run a user interview process. Perhaps you already talk to stakeholders at the beginning and end of each project for requirements gathering and review, respectively. Those a great opportunities to include some more probing questions focused on their motivation and user experience, as well as the usual questions about the data itself and how the data will be used.
When should you use user interviews?
User interviews are a great first step in almost any research situation to more deeply understand users’ perspectives. They are particularly useful in the early stages of an open ended project – for example, when considering implementing a self-serve data tool for the first time. Deeply understanding how stakeholders currently use data and what their pain points are can help inform the vendor selection process.
Surveys
Surveys are a familiar method, in which you collect information from respondents via structured questions. For this purpose, we generally mean surveys administered via an online tool like Google Forms or Typeform. Surveys are helpful for collecting data from a broader group, where you are unable to talk to each user one on one. A recurring survey also enables you to track trends over time. To ensure surveys provide the greatest value to your team, focus on question wording to avoid leading users to a certain type of response.
When should you use surveys?
Surveys are useful when you have a fairly large group of people already using a product and have clear areas of focus where you want to understand their sentiment. For example, you may ask general questions of all of your stakeholders to understand trends in usage or pain points. We have written much more on data team surveys in a separate post.
Usability Tests
Usability tests focus on how easy it is for a participant to accomplish a specific task with a given tool. With usability tests, it’s important to focus the scope of the study (make it a very guided exercise) to keep users on task, avoid adding other variables, and address the key questions you’re interested in. Often a sample size of 5 users is enough to find the majority of pain points, and if your tool is used by fewer people, you can learn a lot from even testing with 1-2 stakeholders. Testing more users doesn’t appreciably increase insights.
When should you use usability tests?
Usability tests are a great fit for understanding how to evolve a very specific tool, like a dashboard, to better serve the needs of its users. For example, perhaps you’ve built a dashboard that enables users to filter for a specific promotion and understand a variety of metrics about the promo. It might be helpful to run a usability test with several different intended users to ensure they are able to easily and quickly understand how to accomplish their role-specific tasks by using the dashboard. This can happen before you take the dashboard live, or at any point thereafter to continue to evolve its usability. While you should focus first on understanding how folks use your dashboard or tool independently, these sessions can also end with some bonus training to empower users to be more effective.
Focus Groups
Focus groups are interviews with groups of a few users. As the study moderator, you would lead a discussion on a defined set of topics. This technique can be helpful as you can often yield creative responses with users bouncing ideas off of one another and building on the responses of others.
When should you use focus groups?
Focus groups are useful when you hear a lot of different feedback about your data product and you’re looking to define your problem statement. Speaking with a group of stakeholders can help you surface and narrow in on the key issues to solve. They also provide a setting where you can explore solutions with the group. This can be useful when problems aren’t yet well scoped. For example, perhaps you are hearing some generalized feedback that your company’s mechanisms to deliver automated data to clients has some challenges. A focus group can bring a cross-functional of affected users together to understand the root of the challenges and begin to ideate on potential solutions. Just be careful – the goal of a focus group is to understand the perspectives of users, not too decide on a solution!
A brief caveat
With any experiment or research method, there are limitations. Human memory has its weaknesses and sometimes participants think a detail is minor, so they don’t bring it up. Particularly with tools they have used regularly for a long time, sometimes users take certain attributes as a given and don’t think to mention them because they don’t think it’s possible to fix problems. It’s helpful to keep this in mind when choosing a method and constructing the questions you ask participants, and to probe a bit more deeply to try to really understand users’ perspectives.
Make your data team’s user research insights actionable
After you gather information via your chosen research method, outline a plan of action to integrate the feedback into your data product. For example, you might find that users struggle to understand what metrics mean, so your action item is to add definitions of metrics into your BI tool. Or perhaps you find that users have to use 3 or 4 dashboards to fully understand a single aspect of your business, so you plan to consolidate information from several reports into a single, more robust report. Or, you may find that users are just overwhelmed the minute they log into your BI tool, so you want to start a training program to help your users get comfortable with your tool’s basic functionality.
Ensure that your action items are achievable. If the feedback involves making massive enhancements, break it into smaller projects that gradually get to the ideal state. Keep the progress iterative.
Maximize the impact of your user research program
It’s all too easy to go through the steps of the user research process and not get the full value out of your results. You’ve spent the time planning out your research project, gathering feedback, and distilling the key takeaways into actions. There’s more you can do beyond integrating those learnings into your data platform. Here are a few key suggestions to get the maximum impact not only from a single research project, but from user research as a team competency:
Build research into your project process
Make this a muscle that the whole team is accustomed to flexing. When you kick off a large project (whatever large means in your context), have a default step at the beginning to understand why and a step at the end to confirm that you’ve really solved the problem. You probably have something in the vein of “requirements gathering” and “stakeholder sign-off” – you can adapt these processes to be even more user-centric and focused on motivations and usability rather than simply logistics.
Get the whole team involved
Invite your teammates to observe user research sessions and dig deep into survey data, creating a culture of empathy and deepening their understanding of users. This allows the team to see pain points in real-time. It also builds excitement around user research and formalizes the practice into your process.
Bring a research mindset to all your stakeholder interactions
Even if you don’t have the bandwidth to embark on a full research project, learning how to listen to the real needs of your users, turn feedback into incremental improvements, and lean into empathy are skills you can weave into everyday stakeholder conversations. It’s a huge benefit to interact with your users directly every day, so take advantage of that access to glean all the insights you can from them.
Borrowing best practices from user researchers can make empathy a super power of your data team. It can help you build better products and approach every problem from a user-centric perspective. Have you used any of these methodologies within your team? We’d love to hear about your experiences in the Slack channel!
