Hello Product Data Team, Goodbye Ad-Hoc Work

From 2019-2022, I was the data product manager of the business intelligence team at Unite, a sizeable European B2B procurement platform, where I guided our transformation from a “service team” to a “product team.” We went from implementing long lists of dashboards and reports for sales, customer services, and management to turning analytics inside the company upside down. When I left, we had just finished implementing a multi-year vision for the future of the company’s data platform.

We went from ad-hoc work for internal users of the business intelligence systems to proactively designing and building a few data products to help product, sales management, and the executive team make critical decisions better and faster through data.Going from a service team to a product team was necessary for us because our previous way of working wasn’t practical anymore.

In some situations, service teams are necessary and quite effective; data teams often adopt a service-oriented approach for a reason. A service model can work well in situations with clearly-defined problems and a need for quick turnaround, especially for some stakeholders. And if it’s working well, you will face resistance to changing the team’s direction. So, it is essential to understand when this transformation makes sense and when it does not.

The process outlined in the articles sounds easy:  “focus on users and be proactive,” but most data teams already do focus on users, especially when they are service-oriented. And they are proactive. They do come up with great ideas for new dashboards, and they proactively set up workshops and the like. Being a product-oriented data team, thus, is not primarily about just “putting your focus on different things”; It is about shifting from many disconnected low-value work items towards fewer high-value projects combined into a single long-term vision.

But this shift is hard and takes time. That’s why this post is about the process we came up with that helped us make the successful transition. I’ll start by describing the “pre-2018 world” as it was at Unite. We were a successful service-oriented data team that constantly delivered outcomes. 

I will then explain:

1. why we considered making a change to our work styles,
2. how we started small,
3. how we analyzed the results, and 
4. how we learned & adapted from them.

The data team before 2018

On the business intelligence side, the data team at Unite focused mainly on delivering dashboards and reports, building data pipelines, and maintaining the business intelligence system. The work came in through tickets created by internal customers in sales, customer service, marketing, and other departments.

As data PM, I prioritized these requests and decided what we delivered and when. The company then had about 500 employees, with about 400mm USD in revenue growing 10-20% yearly. The business intelligence team I PMed managed systems that served around 100 unique users each week. 

As an internal customer-focused team, we frequently talked to these end users to understand the requirements of a specific report or to gather more ideas for other reports. The team had no problem delivering on asks and built up significant technical and business knowledge. Thanks to that, the reports & dashboards were built quickly, and our stakeholders were happy.

The situation was great for a service-oriented team. Just like when you call your Apple customer service, the problems we received needed quick implementation. Our report turnover time was similar to a basic repair; finishing one new report took a few days.

Both the iPhone repair service and our service orientation succeed because of three factors:

1. The problem is apparent, clear, and well-defined. 
2. There is a step-by-step solution to the problem.
3. The tasks need quick implementation.

But in 2017, Unite underwent three significant shifts First, the company pivoted its business model including a subsequent renaming and rebranding. Simultaneously, the software department went from a monolithic software architecture to a full-blown microservices architecture. Finally, everything moved into the cloud.

With so many changes around the rest of the company, the data team sensed that we might need to change too. Suddenly, the three factors making the service orientation succeed didn’t seem to hold anymore.

Step 1: Take the hint

When I became the data PM for the team in 2018, the previous PM already realized something was off, so we talked about the situation. It became clear that the business change and the technological change pulled the rug out from under the data team.

The company implemented a microservices architecture to change the essential software components of its business quickly. That meant the business contexts started to change much more quickly. Suddenly, we didn’t have “step-by-step” instructions anymore because every report was based on a new concept or product we didn’t even understand the business lingo of. The data team was already behind as we received requests to analyze products we had never heard of and to ingest data we didn’t know the source of. Data teams need to understand the data they get, the context in which it is produced and meant to be used, and the language of the internal customers they deliver it to. If data and internal customer language are new and hard to understand, it becomes a significant hurdle to delivery.

In addition, the way the company worked changed. The company went from using data sporadically to using data in every major decision. As a result, the value of the information the data team delivered changed. Before, there were a lot of small tasks with small value. Now there were big tasks with big value in a fast-changing environment.

These moments- significant changes in the environment around the data team, in the business, in the software department organization, and in the way end users work with data- will create the space and opportunity for a workflow transformation. They may happen slowly and creep up on you, or there might be big bang moments. They might be hard to see or quite apparent, but, no matter how they appear, you must observe them closely and decide to act. 

This is an incredible opportunity for any data team – having your work become so visible and impactful is exciting, but it also means you have to deliver.

The previous data PM considered two different modes for operating in this environment:

1. Go all-in on the service model but adapt the services model to the fast-changing environment. Train specific domain experts for each major domain and have them stay up to date all the time. That would be one data engineer and analyst for sales, one pair for product management, and one pair for marketing.
2. Go all-in on a product model, stop implementing ad-hoc requests, and start to implement a continuous discovery and delivery process. At the time, it wasn’t clear what this meant, but it was clear what it did not mean: it did not mean doing disconnected work driven by ad-hoc requests that don’t fit together into a larger picture.

While we considered a process change inside the team, we also had many conversations about our new business model with top management and the head of analytics. It was clear from these exchanges that this new business model needed to have data infused into every part of the key decision-making positions, and it needed to happen fast. We decided that Option 2 was the only way to achieve our goals.

From my perspective, it was also clear that we would only make significant steps in the right direction if we stopped implementing small requests that individual end users submitted. I needed to align our team’s priorities with the new company direction, which meant serving some parts of the organization more than others.

Integrating data into decision-making everywhere was a significant shift. However, to progress on this shift, we had to keep working in one direction, not several disconnected ones. It meant we needed serious direction setting and prioritization.

Step 2: Try something

The idea of “Setting a direction, aligning the team” sounds like product management 101. But that doesn’t mean that management or your department head will approve of such a change of process inside the data team, let alone stakeholders that are used to submitting tickets and getting them implemented within a week. 

Delivering visible progress is the easiest way to convince management and stakeholders that this change is the right direction for the company. There are a bunch of things you can try to show visible progress. All of them should have two characteristics:

1. They should be small. You won’t get large changes approved. You only get fast results with quick and, therefore, usually small changes.
2. They should still be meaningful so that you can show actual progress.

Many of the things you should try will center around communication and planning. Depending on your previous situation, you might:

1. Implement a regular planning event with all your stakeholders, such as quarterly planning
2. Implement a backlog to show off priorities
3. Implement a roadmap to communicate strategic focus
4. Implement regular but infrequent meetings with key stakeholders to discuss major initiatives
5. Communicate your planned new way of working internally. For example, through a blog post
6. Propose new work items you come up with yourself that align with the company’s direction

By the time we decided to change, we had already implemented quarterly planning events, but those events didn’t feel like an improvement to the ad-hoc way of working. During this meeting, all stakeholders would request their list of reports & dashboards for the quarter, and we would prioritize them together. We went from having ad-hoc requests daily to getting them in one big batch at the beginning of the quarter. That didn’t feel like a huge improvement yet. 

So, I proposed something very different for the next planning event to the stakeholders. For the next event, I wanted to try several things, turning the planning process upside down. Everyone did see that a change was necessary, so everyone was up for a try. Here’s how it went down:

• I first thought through a list of things the team and I thought were attractive to the company.
• I asked all stakeholders to provide their “list” before the planning meeting.
• I aligned both lists and packaged them into large blocks, eliminating everything that seemed optional to the company.
• I had one event with the stakeholders to decide which 1 or 2 of them we’d do.

The outcome was a very different plan for the upcoming quarter. Instead of a long list of dashboards, we reduced our efforts to two significant initiatives. Even better: the stakeholders were on our side, and though some didn’t get any of their projects into the upcoming quarter, they did realize the importance of the work to the company.

I recall one key stakeholder I completely cut off for a quarter (and indeed multiple ones) saying, “That sucks, but you’re right; this is the best for the company. We actually will get by. We have workarounds; the salespeople don’t.”

Having significant initiatives aligned in one direction enabled us to be more agile and invite more end-user engagement. In addition, because we only chose 1-2 large initiatives, we left the room to ease the transition pain; we were free to implement a dashboard here or there whenever the end-user conversations indicated it would be helpful.

In the previous meeting, I had agreed to finish a long list of reports and dashboards, leaving little room to adjust the course. However, this quarter, we were free to adjust as we worked toward a bigger goal.

Step 3: Analyze

Once we had some early success with moving into a product operating model, I talked to my boss. Over a couple of conversations, we analyzed the approach. My manager and I both realized we needed a vision for the team. So I brainstormed and came up with the following:

To help employees make better and faster decisions thanks to data.

We could only truly analyze the proposed shift from this point on. This vision clearly showed that the service model wouldn’t work and a product model would be needed. The details, however, weren’t as clear. A couple of questions we dug deep into were:

1. Is this vision closely aligned with the company vision?
2. Is the vision closely aligned with the department’s work?
3. How are we planning to implement this vision?
4. Which people are we focusing on to make better decisions?
5. Do we need support to pull this off?
6. Are we taking all our stakeholders with us with this change?
7. How do we make sure we’re keeping our focus up to date?

In our case, going through these questions delivered two key insights. First, planning events every quarter sounded good but wouldn’t help us keep current. A quarterly planning cycle was too slow for the business changes we were undergoing. We already noticed that we needed to make regular adjustments throughout the quarter, but more extensive changes should also be possible, and we should not have to wait 1-3 months.

The second insight was more subtle initially, but it was one of our most significant shifts. There is one big question that internal data teams should spend the majority of time on: “Which internal customers are we focusing on?”.

There is a duality in most companies. We have two buckets of internal users of data with different weights and different quantities:

1. Lots of internal users of business intelligence systems, most of whom support decision-makers by providing information from this system to them (and of course, some are decision-makers, but they are smaller in number!).
2. Decision makers themselves also use these systems but always rely on support from analysts and other people to make their decisions.

Put differently, most internal users are only in a supporting function for making decisions. The critical decision-makers might not even use the system that helps them make decisions.

So we had to learn from this and adapt.

Step 4: Learn & Adapt

I decided to run a dual communication strategy to get the best out of both user groups:

1. I set up a regular but informal meeting open to all end users. Its purpose was to inform and gather information, but it was never a “planning” event.
2. The discovery and potential planning happened in infrequent but regular deep meetings with the key decision-makers. This discovery process directly involved other product managers, top management, the Head of Sales, the Head of Marketing, team leads, and other critical decision-makers.

Only after adopting this dual strategy did I understand the bigger outcomes we needed to achieve.

Communication with critical decision-makers was surprisingly easy to nail. It meant doing what every other PM does: focus your discussions on problems and alternatives, not your shiny solutions. Ask questions, understand pain points, and understand what these decision-makers want to achieve themselves.

The communication with end users evolved over time. Here are some of the strategies we tried:

1. Have demo days; we had them bi-weekly
2. Write a newsletter; we did a quarterly one
3. Run surveys; we did one with every newsletter
4. Establish a “community of data practice”; we did one every other week
5. Open up specific Slack channels for questions
6. Have a public clear and visible nearsighted roadmap.

We adapted our communication strategy over the years. Slowly, we learned the key ingredients to great product-oriented data work at our company:

1. a clear vision,
2. big chunks of work that have a visible and understandable value to the company,
3. Embracing the duality of internal customers and communicating with both groups (a lot!),
4. a focus on the high-level decision-makers when it comes to understanding customers.

We went through this process and transformed how the company worked with data. This change was only possible because we iterated. 

1. We needed to understand first why our team had a service model before understanding what had changed. 
2. We could only get our first change in our work process approved because of that understanding.

We could only start the large-scale change towards a product model by showing how well the tiny change worked. 

3. We could only succeed with the transformation by constantly analyzing and learning. 

This step-by-step approach underlies our transformation; you will need it, too, if you want to make this transition work.

Did your team go through the transition from service to data product team and made it work? We’d love to hear your ideas and experiences in Slack.