In today’s article, we are taking a step back from the technical aspects of data science. Instead, we are investigating the best approach to manage data science projects. Too often, projects are started with a focus on the technical solution. In other words: “how” to solve a problem. But data science leaders say it’s much more important to start with the “why”. Let’s dive in!
Start With Why
According to renowned data scientist and prolific author Thomas C. Redman, “the biggest successes of big data projects stemmed from softer factors such as a deep understanding of business problems”. He derived this insight from comparing notes on his own successes and failures over the years as a successful data scientist.
Data scientists in companies want to do work they are good at: extracting valuable insights from data sets. This is what they’ve been trained to do, and this is what their job description and hiring process are focused on. Companies also tend to isolate data scientists from company politics, because they are “rare” employees on the job market and therefore valuable. Naturally, leadership doesn’t want to bother them with managerial problems!
However, to produce successful data science projects, the entire team must understand “why” they are solving a particular problem. Everyone must be involved in the underlying priority of a project. This will lead to a singular focus on the decisions made. If your team is not aware of the “why”, they may end up creating a powerful and effective solution to the wrong problem.
How can you implement this approach as a project manager? By creating individual contributors on your team. An individual contributor is defined as “an employee without management responsibilities who independently helps an organization support its goals and mission”. Having everyone understand and believe in the “why” of your mission, and providing them with the resources and authority to execute that mission is what delivers success in data projects.
Girish Pancha writes: “A good leader never specifies the “how”. In other words, you need to define the strategy and motivation behind projects and give employees the right support, without micromanaging the method of execution. Giving your team the freedom to execute will create room for innovation. Leaders are meant to keep the focus on the “why”.
Centralized vs. Decentralized Reporting
If you’re giving autonomy to execute, you still need a good reporting system for the managers. Chuong Do writes that data science teams can adopt either a decentralized or centralized reporting structure. The best strategy depends on the company size. He states that most smaller companies rely on a hybrid strategy, whereby data scientists report to executives but are also close to business units, a technique called “embedding”.
Centralized data science organizations have data scientists reporting to a single head of data science within a company. Decentralized (or “integrated”) data science organizations have data scientists reporting to different functions or business units throughout a company. Choosing the right reporting structure depends heavily on the size of your organization.
Note that some data stack platforms allow you to have a hybrid centralized/decentralized reporting structure, meaning both executives and engineers will have access to progress and findings. This allows your team to execute the “how”, so the data science manager can keep the focus on “why”.
SOURCES:
Check out our amazing sources and their profiles!
Thomas C. Redman: Do Your Data Scientists Know the ‘Why’ Behind Their Work?
Girish Pancha: Leadership Is Like Engineering: You Need to Start with Why
Chuong Do: What is the most effective way to structure a data science team?