One of my first bosses, Chad Richeson, generously wrote the forward for #DrivingDataProjects. Using data to drive decisions was very new in the early 2000s, and we often thought of ways to help people understand the value of doing that. We used to brainstorm about how to make the concept of data more accessible to people. I miss those sessions, but I am glad he’s still just an email away.
One of those conversations made it into the About the Cover section of the book. As data consumers, we need to be both mechanics and pilots. We must know how to gather, cleanse, and prep—and present data, make data-driven decisions, and influence data. That is a very broad set of skills.
To be effective data consumers, we need to be both mechanics and pilots. We need to know how to gather, cleanse, and prep our data—as well as present, influence and tell effective stories using data.
With tell-me features and AI Tools, we forget the importance of and lose ambidexterity skills like managing details and thinking strategically. Additionally, the emphasis used to be on being able to talk to the box and not the people; now, we must reason with the box (and the people). Skills cultivating engaged stakeholders and executive sponsors weren’t emphasized as much but are now increasingly important. Those are radically different skills!
The emphasis used to be on being able to talk to the box and not the people; now we must reason with the box (and the people). Those are radically different skills.
Too often, we forget the fundamentals that lead to greatness. When I think back to who I learned the most from in the earliest days of Microsoft’s BI days, it comes down to two individuals: Chad Richeson and Greg Koehler. They deserve the lion’s share of credit for what Microsoft’s BI eventual efforts became. Before we had enough people to call a team and before we could prove a business case viable enough to attract a vice president to sponsor our efforts—all of that came after we had taken the risks and proven ourselves—they were mapping out the function of what eventually became the BICI team.
Greg’s early vision laid the blueprint for the data supply chain that served the online consumer division. Greg enabled consistent delivery of business data while also delivering consumer data by transitioning from ETL (extract, transform, load) to ELT (extract, load, transform). The choice between ETL and ELT depends on several factors unique to each organization, including data schema requirements, transformation complexity, performance, and budget constraints, to name a few--Greg was the one who navigated all that before anyone else was doing it.
Early on, we recognized Cosmos (our centralized data repository) as the only way to store and process the vast amounts of data we collected. The plan was presented to Satya in 2007 (then VP of Search Engineering), and he agreed to fund it -- $300M approved in one meeting! Satya saw the future even more clearly than we did. Eventually, we merged BI and CI into a single pipeline, such that CI powered BI. For the first time we could connect personalization to data science to executive reporting, and it unlocked arrays of new insights and even changed some business processes. We had built a data supply chain the business could understand and rally around as something that would serve its business strategy. From there, the team, the VP support, and the division momentum began to build momentum, support, and ongoing engagement.
Being part of those early efforts launched my career in data management and an interest in data as a common language between disciplines that changes the social circuitry of organizations. The Guide catalogs many of the lessons I learned throughout my career from those early days of laying that initial data supply chain and performing many of the activities along its path toward predictive data.
Being part of those early efforts launched my career in data management and an interest in data as a common language between disciplines that changes the social circuitry of organizations.
5 Takeaways from Driving Data Projects:
There is a human side to the data supply chain. We must become more accountable as data teams and stakeholders for how data is acquired, managed, used, and disposed of.
Working with data is not straightforward. The data supply chain illustrates how interrelated every technical platform and data project is. Learning the purpose and general activities in each layer—the underlying hardware, data quality efforts, analytical tools, business decision-making, etc.—an organization will perform as well as the lowest performing level of the supply chain.
Sponsorship is critical. Sustainable change (e.g., ongoing adoption, utilization, and proficiency) cannot occur with a single project manager trying to do the right thing. Sustainable change requires constant collaboration and partnership between the project lead, sponsor, and working group members.
Scoping problems to multiyear goals enables accountability against criteria for success. [You can’t run a data strategy from an IT backlog.]
Change must be constantly tuned. If we seek speed and scale and are too well funded, nothing will ever change. What works in one situation doesn’t necessarily work for everything—solutions must be tuned. We must find ways to slow down even at a fast pace to find the right frequency. Small and slow is transformational. We can learn to cultivate communities of practice where we belong, and the world can shift. This can happen in a classroom, a meeting, or a conversation--and our views can shift.