In our last post, we talked about the differences between linear and circular projects. The distinction is valuable because it can help data teams manage expectations and build processes that suit different types of work. Managing expectations is particularly important for the kind of work we do as data teams for several reasons:
- Data work is often a moving target (particularly exploratory or otherwise open-ended projects).
- This open ended nature creates unique stakeholder communication needs in comparison to more straightforward projects on other teams.
- Data work requires managing tradeoffs between rigor, curiosity, and urgency.
As you read our suggestions, remember that linear projects tend to be easy to scope and closed-ended, with clear, objective acceptance criteria; while circular projects tend to be open-ended and iterative, with later phases heavily determined by the results of earlier phases.
Classifying Projects Can Improve How We Work With Stakeholders
How can you put this concept to work?
First, categorize the project. Just because some data work is circular in nature doesn’t mean it all is. A good question to guide you is, “Is this project possible with the data, knowledge about the data, and tools we have?” If you know that it’s possible, though it may be complicated, it’s a linear task. If the outcome depends on some data-related unknowns that won’t become clear until you’ve put in a significant amount of work, it’s likely circular. It’s worth noting that this is not quite the same as defining the problem (e.g. “Is this what we should be solving?” or “Is this question clearly defined?”).
That said, most of the tips below can help alleviate uncertainty in both linear and circular projects. Investing more time in planning and communicating progress can ultimately save quite a lot of time and headaches. Here are a few places to start, and some of the tips from our modifications to Agile post may be helpful as well.
Setting Expectations with Stakeholders
- Help key stakeholders understand why some projects aren’t guaranteed to succeed, and why it requires so much work to determine whether they will or won’t. This is fundamentally different than most disciplines, so it’s hard to over-communicate it.
- Communicate uncertainty appropriately for the type of task. For linear tasks, you likely have a pretty good idea of the potential sources of uncertainty. Here are some good phrases to use for linear tasks:
- The portion of the report that shows sales by product line is straightforward, but I’ll need to investigate the format of our referrer data to know how long it will take to produce sales by referring site.
- That data source is supported by our extract and load vendor, so getting it into the data warehouse will only take a few hours, but I will need to see what the data looks like to understand how long the transformation process will take to make it easily usable.
- For circular tasks, you may not even know the stakeholder’s request is possible, let alone how long it will take. Here are some good phrases to use to clarify that:
- We will investigate X, Y, and Z metrics to try to determine why sales are up this month. Those might not give us a conclusive answer, but we can regroup to discuss the findings and determine next steps.
- There are a lot of options for how we could approach predicting that. Can I explore it a bit and get back to you with potential next steps? I’ll spend 2 days investigating our options so we can assess several possible approaches, with the complexity and likelihood of success of each in mind.
- Honestly, no one has delved deeply into our customer service data in quite some time. I’d like to start exploring the data set to determine whether we can answer those questions before committing to a timeline.
- Talk often about how much failure is palatable. Research isn’t always successful – failure is a necessary and valuable part of creating knowledge. Sometimes it takes 3 or 4 iterations to figure out the solution to a problem. Sometimes it takes 20. Sometimes the problem isn’t solvable with existing data and methods, no matter how much time you devote to it. Make sure stakeholders understand the risk of failure up front, and regroup at each milestone to talk about the outcome, whether it makes you more or less confident in ultimately solving the problem, and align on how much more time is reasonable to devote to this project before changing course.
Improving Team Processes to Mitigate Spin
- Invest more time up front in requirements gathering. Make sure you really understand what stakeholders are looking for, why, and with what level of polish. “Why” can help you suggest alternative, more straightforward, solutions. A 10-15 minute conversation up front can both save significant work time and improve the quality of the output.
- Create a clear, separate exploration phase for projects. Do time-bound research to get a better sense of how challenging the unknown aspects of a task might be, what can be easily answered, and revise your estimate of feasibility and potential timeline. This allows for open-ended investigation but puts guardrails on the task. This holds you accountable to delivering a clear recommendation in a reasonable timeframe rather than losing focus and potentially spinning out.
- For really open-ended tasks, try a design sprint. The sprint methodology is designed to take very open-ended projects, narrow in on a reasonable area of focus, prototype, and iterate. For mandates like “the company needs dashboards” or other not-yet-scoped projects, a design sprint can provide the necessary structure to align stakeholders and get something produced in a time-boxed period.
- Practice figuring out the smallest meaningful iteration for circular projects. Sometimes you can reduce a circular project to a series of linear tasks, at least for the early stages of the project. Identify what you can commit to and deliver it incrementally, then revisit whether future steps will add enough value to be worthwhile. This can be one of the hardest things for the rigour-minded types that tend to work in data. Learn the difference between “perfect” and “done,” and share your work early and often!
- You might not be able to share a completed dashboard or get a model production-ready in a week, but you can probably start by sharing a few initial insights and getting feedback there.
- A quick, directional answer can often provide 80% of the value of a much larger project. Keep open communication with stakeholders about the difference between what you can easily answer and what gets into more unknowns. Provide what is easy first, and then talk about the tradeoff between time and value for the rest.
- Consider whether a data product manager could help improve project scoping and stakeholder communication on your team. There are some limitations of a traditional product management approach applied to data, where the data analysts and data scientists have such a deep understanding of the data that it can be hard for someone else to take on a role writing stories for the next steps of a project. However, a good product manager can help focus the team’s work and clarify expectations, leaving more time for analysis.
Conclusion
Data project planning comes with a few nuances that can make it less predictable than other kinds of work. When it comes to circular projects, successful communication comes down to reserving time for research, ongoing communication about lessons learned, and identifying what meaningful output might like for each iteration. How does your team manage circular data projects? Come have a chat in our Slack channel, or join us August 18 at 3pm EDT for a live conversation!
