Impact Forward vs Infrastructure Forward: The Faucets First Philosophy

Rob Collie (00:00): Hello, friends. Before we get started today, I've got some quick housekeeping-slash-coming-events notes. I'm personally quite excited about our upcoming lineup of guests. We have four special guests, not just in the pipeline, but actually scheduled for recording. That's really important, including two Microsoft product team members, Miguel Myers, who works on the core visuals for Power BI. Now the visual canvas of Power BI is one of those places where innovation is potentially endless and not just innovation for its own sake, like actually, valuable innovation can continue there basically forever. So that should be a pretty fascinating conversation about the roadmap. We also have Chris Finlan, who's been on the podcast before. He owns Core Data Engineering Workloads in Fabric, with an emphasis on Spark. So from him, we'll get a really good non-Power BI perspective on Fabric. In recent episodes, you've heard a lot of Fabric is about that other side of the world, the other side of the world that we don't necessarily as Power BI people have as much familiarity with.

(01:01): So getting a glimpse behind that curtain will be very valuable. Plus, Chris has always been, and always will be a very astute and, importantly, very candid observer about everything in this space. So that'll also be definitely worth tuning in for. We also have a recording scheduled with Forrest Brazeal. I'm not even sure if I'm pronouncing his last name right. We'll find out when we meet, but it might be Brazeal or something like that. But you might know Forrest as the creator of that viral comic that went viral on LinkedIn a little while back about AI and sad engineers, which was really kind of cool. Something like that would honestly be enough to have him on the show, in my opinion. But when I dug a little bit deeper into his bio and some of the other things he's been saying and producing over the years, I found that he's very often that insightful and that funny.

(01:48): And once I saw that, I was like, Okay, this is definitely someone that I want to meet. And one of the beautiful things about this podcast is that I can use it as an excuse to meet very interesting people. So I'm really just giddy looking forward to that conversation. It can go in a number of different directions. I won't even be able to predict what they all are, but I'm sure they're going to be interesting when they happen. Last but not least, we also have longtime industry veteran and observer, Shawn Rogers in the queue. He's one of the first people I ever followed on Twitter back in 2009. Shawn's been an executive at many data software firms, including Dell, Microsoft competitor, TIBCO, and he's now CEO of a company called BARC, which is kind of similar to a Gartner or a Forrester Research. So that episode should contain a very high concentration of sage wisdom, which is something we always, always, always need.

(02:37): Okay, that's all coming soon. But today I want to talk about a philosophy that's really central to our entire approach here at P3. And it's not something that we've trademarked or anything like that. It's just a really good thing. It's one that we've mentioned many times on this show, but we've never given it its own standalone episode. And I think that's something that I very much want to do. So we call it a few different things, like on our soon-to-be-relaunched website, we have it called Impact Forward as our approach. It appears in our internal core values as the direct core value, meaning we are direct, but our favorite way to describe it is Faucets First. It's just fun to say Faucets First, get that alliteration. And it also invokes a really powerful visual metaphor. So let's dive in. I'll explain what it is, what it means, how it contrasts with other approaches, why it's really only recently been possible, and also why it's so much more valuable than its predecessors.

(03:39): So let's do the visual metaphor. Let's start there first. Imagine you're standing in a dry-ish landscape, and you're standing in front of a shriveled tree, a tree that's not doing well. Now the good news is there's a pristine natural spring nearby. It is like an oasis, and it's less than a football field away. It's not that far off. So how do you get water from there to the tree? Well, first imagine like a giant construction project. A team comes out and surveys the land, and then a couple of weeks later, some bulldozers arrive to grade out the ground, and then they leave, and another team comes in the next week and starts pouring a bunch of concrete slabs all over the place. And when that's done, even that is just the prep for the prep because soon after that, or soon-ish after that, another team comes in and starts building an elaborate and Byzantine system of pipes, it is the sort of thing that you would see at an oil refinery or a water treatment plant, just a big, big, big infrastructure, big metal.

(04:41): And this goes on for months, maybe even a year plus this whole thing before anyone gets back around to saying, Oh, yeah, right, water for the tree. It's just been pipes. No one's been thinking about delivering on the original goal. They've been building the infrastructure that was originally deemed necessary to deliver on the goal, but the goal's been lost. Really, no progress has been made against getting the tree some water. And things are just getting around to maybe delivering some water to the tree for the first time when you realize, well, the tree's been dead for six months. All right, now that might sound like a silly and tragic story, but until recently, all data projects had to operate on exactly that type of timeline, exactly that type of philosophy, and basically were doomed to yield exactly that type of result. And this was non-negotiable.

(05:30): You had to approach things that way. The software tools of previous generations required that you built up from the bottom. You had to build infrastructure first to even enable the idea of eventually providing some value to the business. And even though that was the way for decades, it was never the right way. It was just the only way, and it's entirely unsurprising that complete professional cultures sprang up around that methodology. It was expensive, it was inefficient, it was frustrating, but there really wasn't an alternative. Now, returning to the metaphor for a moment, imagine instead you see the tree, you see the water. Your first step is to run a flexible hose from the spring to the tree. Now, in the face of the previous philosophy, the incredibly infrastructure-forward, plumbing-forward philosophy, running a hose like that might seem like primitive and low-rent. But if you hadn't been indoctrinated into that historical methodology, the benefits of this are immediately evident.

(06:38): First and most obviously, the tree doesn't die while waiting for the infrastructure. If the whole point was to restore the tree to its health, tree not dead does seem to be an incredibly positive outcome by comparison. As you keep going though, even more benefits emerge. For instance, you actually get to test your implicit hypotheses. Number one, was this the right tree? Is the fruit of this tree the most valuable? Maybe there's lots of other withered trees around here, and you just sort of picked one. And even if that is the right tree, is water what it really needs? Or is it a soil quality problem? Imagine waiting for the year-long infrastructure project to finally reach some sort of momentum before you discover that either this is the wrong tree or water isn't even what it needed, and there's at least one more benefit that's definitely subtle compared to those first two but might be just as important in the final analysis, which is something that I think I'm going to start calling positive visibility.

(07:36): When I say positive visibility, I mean the word positive in the same sense as positive handoff, like clear and verified, that kind of positive. So for example, in the metaphor, once you start watering the tree and you start quickly getting the feedback as to whether or not this is what you're actually after, that is very, very, very non-ambiguous confirmation that you're either on the right track or that you need to change tracks. And while that might sound kind of like, so what? It's important to note that you don't have that, you don't have that positive visibility in the infrastructure-forward approach. You'd have to sit there and trust while they're building all these pipes that someday it will deliver on its promise. Even the fact that you chose to run a hose at the beginning. Now, if that starts to work and show results, if you decide that you need more water or you need a more robust pipe that isn't going to dry out in the sun or something, you can run that new pipe.

(08:32): It's just one, it's very direct. You have complete non-ambiguity, clarity on where that pipe needs to go. And when you turn it on and switch it over from the hose, you've been getting good and positive results to that point. And if you don't keep getting good and positive results, then you know immediately that there's something wrong with the new pipeline. Again, something that you wouldn't have in the infrastructure-forward approach. Okay, now, at the metaphor level, the watering a tree scenario, it's really clear which of those two approaches is better. You would never choose the plumbing-heavy, plumbing-first approach to that problem. You'd run the hose, you'd keep the tree alive, you'd gain all the information about whether or not it's working. And even if you decided to upgrade the plumbing at some point, you would do so with a tremendous efficiency that would never be present in the plumbing-first approach.

(09:26): But let's bring the metaphor back to real life, back to the data projects world. In a data project that follows the Faucets First philosophy, the tree that doesn't die while it's waiting for water. That's the business need or the opportunity that you identified in the first place, the reason for the project. And that opportunity is still going to be relevant, active when you start getting your first results. And so often, in the infrastructure-first approach, the business need has either expired or changed dramatically by the time the infrastructure project is ready to start delivering its first trickle of water. Moving on point by point through the metaphor, testing your hypotheses that this is the right place to deliver water and that water is what you need. Data projects, and in particular, BI projects are inherently projects of discovering the unknown. There are more twists and turns in a BI project in particular than there are in even building a sophisticated piece of software.

(10:28): The requirements are waiting to be discovered. The moment you start getting your first answers, you inevitably start realizing that you are asking the wrong questions. And that is like an immutable truth in my experience, it's the source of my saying that human beings don't know what they need until they've received what they asked for. Once you internalize that, you'll never trust another requirements document ever again. Okay, what about positive visibility into the project from the early stages? So as a stakeholder or a business leader, not as a techie for a moment, if, for example, you're seeing dashboards in the first week or so, you're so much better equipped to evaluate whether those are on target than you are at evaluating the early stages of some abstract data pipeline project. You're seeing the thing that you need and the thing that you understand that you can provide feedback on, as opposed to some mystical conjuring ceremony that they tell you is going to bring a genie, but you're not really sure, you've never seen a genie before.

(11:30): You got to take it on faith. So with that kind of positive visibility from the early stages of a project, you know you're not setting time and money on fire, you know you're not being taken for a ride, whether that ride is intentional or accidental, you know that's not happening. That means you have confidence, and your domain knowledge as a stakeholder can contribute in real time to the quality of the solution. The opportunities that you spot along the way are often just as valuable as even the original intent, and you're going to miss those if your business domain experts are sidelined by the plumbing first philosophy. You even start to understand the tech a bit better. You understand the reasons for things. So if someone comes to you and says, Hey, if we upgrade the data pipeline, like from the hose to a pipe in the visual metaphor, if we upgrade that data pipeline behind the scenes, we'll gain these tangible benefits X, Y, and Z, you'll have a basis for understanding that and a basis for weighing in on it.

(12:29): So the bottom line is that from a data project perspective, you get dramatically better results this way, better results, but you also get them much faster, at much lower cost, and at much lower risk. All right, if it's so much better. And it is, it is all of those things I just described, why wasn't it always like this? Why didn't we always do it this way? Well, the difference really is owed to really a complete sea change that occurred in the software tools of the current generation. And while Microsoft is not and was not the only player in that revolution, they are by far the most impactful, the ones that were most successful at it, the ones that made it a reality much, much more so than anyone else. And the revelation that is the Microsoft, the modern Microsoft platform, it does not get enough due credit, in my opinion, in the marketplace for making this new approach possible.

(13:23): In my timeline, Microsoft tools like Power BI, they went from being perceived as edgy and unproven one day to the boring and safe choice like the very next day. We had years of punk rock upstart type status, and then that went straight into boring. This is the default tool of choice. The market never took the time to celebrate how amazing these tools are. Freeing us from that infrastructure-first philosophy should have been a year-long Jubilee or something like that. And while that's kind of a funny and sad observation, I think that's actually a problem that we transition from one to the other without ever stopping and realizing as a market what we had. Now, this is not a problem for Microsoft, being the boring default tool of choice, hey, that's great. People don't have to think they just buy the software. The dollars spend the same.

(14:20): No one should cry for Microsoft that they didn't get their party. The problems here are actually for everyone else and for two reasons. Number one, the infrastructure forward way still hasn't died. Now it's no longer considered fun, or cool, or edgy to criticize the infrastructure-forward approach. And I still do it, but it was a lot more fun eight years ago. On the record, most people will tell you, Oh, yeah, yeah, yeah, we've totally switched to the Faucets First philosophy. They won't call it that, of course, but they'll say the right things. At the same time, though, we still see a lot of infrastructure-first, plumbing-first projects. Now, ironically, Microsoft often wins here too, because infrastructure-forward projects, not only do they take more consulting time, more implementation time, and cost. Infrastructure-forward projects often do tend to consume more software licensing. The software footprint and the software cost of an infrastructure-forward solution is often quite a bit greater than a Faucets First projects footprint.

(15:24): Now, I used to think otherwise, I used to think that getting the implementation cost out of the way would make more room for Microsoft to sell more software. But I think what we've discovered is that software is still fundamentally, in a micro sense, software is, especially Microsoft software, is just too affordable. It's a good problem to have, right? It's just too affordable. It's only the grand total that adds up. If you're going to sell a lot of software, you almost need there to be waste in the system. So chalk that up as another benefit of the Faucets First approach is that even your software bill is reduced. Now, the second reason why I think the lack of a market realization of just what a gift this stuff is is that there are still a fair number of people stuck on the starting line.

(16:11): The lack of a clear resonating now today is different message in the marketplace. A message that says that the old ways aren't the way anymore and you don't have to take the same risks, and you aren't priced out like you used to be. The lack of a message like that has a paralyzing effect and keeps these people stuck on the starting line or close to stuck anyway when they don't need to be. So until those two problems finally fade away into history, and we're finally living in a world where every project is driven directly by its business impact with positive visibility and infrastructure that is built in response to those needs. And until everyone who is currently today stuck on the starting line until they are unstuck and confidently, nimbly watering the right trees at the right time with the right mixture without all the costs, and delay, and risk associated with the old approach, until we reach that day, we're going to need to keep talking about this Faucets First philosophy.

(17:14): Okay, so wrapping up. Some questions to ask yourself, whether you're a business leader or a data practitioner, number one is your organization, would you say, honestly, behind closed doors, would you say that it is fully on board with a Faucets First approach? Another question. Are you seeing evidence that infrastructure-first projects are still maintaining their reign of terror? Are you seeing those things still happening? And thirdly, and perhaps most importantly, are you experiencing an unconscious paralysis because you're afraid that a particular business need or opportunity that you've identified, you're afraid that that can't be addressed without big costs, delays, or risks? If so, that's definitely an opportunity to get off the starting line because it turns out that even if it's not dashboards, if it's something else of a data project, there is confidently a Faucets First approach that can be taken to solve that problem.

(18:09): All right, that'll do it for today. A quick note at the end here, if you'd like to support the show, you can just leave us a review on your favorite podcast platform of choice, or you can even just tell a friend or colleague about the show. And we're always looking to expand our reach with this. We don't really advertise in any way for this podcast. So really, the only way that we expand our reach and reach new people is through word of mouth, through those reviews, through you telling other people. So if you like the show, it's a very, very, very kind thing that you can do for us. All right, catch you next week. And until then, as producer, Luke, says, have a data day.