A Simple Trick for (Occasionally) Predicting the Future 

Speaker 1 (00:00): Hello, friends. So the title to this episode kind of sets a high bar, the ability to truly predict the future where anyone to actually have it would legitimately be magic, a true superpower. And I want to be clear upfront, I do not possess anything like that power. I didn't foresee the rise to dominance of Google or Amazon or Facebook for instance even though I was around and paying attention when all of them first appeared on the radar. I didn't anticipate the transformative role of smartphones on business and society. I'm not a futurist. And frankly, even futurists aren't futurists because their hit rate is so low as to be noise most of the time.

(00:40): On rare occasions though, I've been able to see how something big was going to play out and to see it with sufficient clarity and confidence to actually act on it, to take career and/or business action with a clear picture of that future in mind. And then when that prediction was proven correct, people looked at me as if I were some sort of magical predictor of the future. It's less about the ability to see the future and more about not seeing something that isn't there, and I think it's something that everyone can learn. So it's not a superpower, let's call it a normal power. And given the looming disruption and opportunity posed by AI, I think this is a good time to share it.

(01:22): This will all be so much clearer with examples. So let me share the two most significant cases when I used this normal power to its greatest effect. One was in around 2010 when I saw how good the first release of Power Pivot was and what this was going to do to the BI industry. It was all so clear seemingly like in an eye blink, BI projects which had long been slow, expensive, and underwhelming in terms of their results were now going to become faster, more affordable, and more successful. The lower price tag on implementations was going to dramatically expand the size of the BI market with the number of projects executed worldwide each year going up by a 100X or maybe a 1000X or more, and consulting firms were going to have to adjust to this faster tempo. Rather than performing a handful of million dollar plus projects a year, they'd need to learn to execute many, shorter, and less expensive projects.

(02:20): And I didn't think the existing consulting firms in the space were staffed or organized in a way where they could actually make that transition. This seemed like a massive opportunity like a once-in-a-lifetime career moment. So I started the company that became P3, and here we are years later, proven to be very much correct. We're big-ish and growing despite being self-funded the whole way and despite having a business model that exploits neither our clients nor our employees. We've now also spawned a number of copycat firms following in our footsteps, and I also expected that to happen. In fact, we take that as a sign of our success.

(02:58): But even as successful as that prediction proved to be, it wasn't anything perfect. I didn't see the now obvious point that Excel, which was where Power Pivot lived, was a bad host, a bad container for this revolution, and that we were going to need something far more graphically compelling like Power BI in order to break through.

(03:17): I also didn't see that the traditional consulting firms would hang on as long as they have because somehow they're still largely alive and well today. But the broad strokes of the prediction were definitely accurate enough. I bet my career on the coming change, and it's 100% proven to be a good bet. We've defined the template for what this new style of firm can be, but I want to emphasize that it didn't feel like a bet. I'm not a gambler. I'm not even what you would call an entrepreneur really at heart. I wasn't looking for some big new idea for a startup. And I'd have 100% remained in my software engineering job if I hadn't seen this one specific opportunity. It just seemed really obvious with the low risk of being wrong.

(03:58): Now, before I reveal this mysterious simple trick that is sort of the key to being able to see these sorts of things, I think the second example is probably even more helpful to explaining it, so let's cover that quickly. In early 2020, when we were still referring to Covid-19 as the Wuhan virus and it hadn't really appeared yet in the US, the internal analytics, the dashboards that we use to run our business at P3, were indicating that we should be hiring more consultants to meet growing demand. Now, it's hard to remember what it was like back then, but Covid wasn't even a reliably top three story in the news in the US yet. We were still at least a month and a half away from it erupting as a crisis in the United States. But Kellan and I were looking at this signal from our dashboards and asking ourselves, "Okay, do we do what we would normally do in this situation, which is to trust the dashboards and go on a hiring surge?" And I was like, "Nah, I don't think so. Let's hold tight."

(04:53): A number of weeks later when we were right on the eve of Covid exploding to the front of our awareness, but it was still very low-key at this point, I was in what proved to be the last face-to-face business meeting I would have that year. One of the people I was meeting with was telling us how she'd be out Thursday and Friday of that week traveling to Texas for her brother's wedding. And before I could even stop myself, the words came out of my mouth, "Well, this is the last weekend for air travel for a long time." And the room kind of stopped and looked at me funny. Everyone in the room looked at me funny because this just seemed at the time to be a preposterously doom and gloom kind of thing to say. After all, it was still business as usual in the US of A on that day. I was their client, so they were polite about it, but I saw on their faces that they were humoring me.

(05:38): Of course, now hilariously filed under the heading of "You can't make this up," she and her family didn't even make it back from Texas. Flights closed down that weekend and they had to rent a car and drive back to Indiana in the sudden new reality of lockdown.

(05:54): So what was the secret? Why was I able to make crucial business decisions six plus weeks in advance of the world changing in a way that ultimately surprised most people? Again, it's not a superpower. It comes down to this. Most people look at the status quo, the way things in society operate at a given point in time, the way that your business operates at a given point in time, the way the marketplace operates at a given point in time, and they think that that current state of affairs is very heavy, like the great Pyramid of Egypt or one of the world's most massive cruise ships or oil tankers, hard to move, resistant to being pushed around. And that's a comforting worldview that the status quo is heavy, that the status quo is super durable, but it just isn't true.

(06:41): The status quo, in my experience, actually weighs nothing. It has no special status whatsoever. It doesn't resist change at all. It merely flows with change. So think of the status quo more like a kite on a string up in the air. And as long as the wind continues to blow in the same direction and at the same speed, the kite on a string will remain in a relatively fixed place in the sky. But why is it at that position to begin with? It's only there because of the current wind flow. There was, and is, nothing at all special about that point in the sky that attracted the kite. That's just where the kite settled in response to the forces around it.

(07:25): Now, if I were to then tell you that the wind has been coming from the south for the entire time the kite has been in that spot, but now that weather stations about an hour away from your location are starting to show a change in the wind coming our way and those stations are now reporting winds from the west, it wouldn't be hard to guess that the kite is going to move when those wind changes reach us. And depending on the level of detail we're getting from those nearby stations, we might even be able to fairly accurately predict where the kite is going to move to.

(07:56): Winds from the south, coming from the 6 o'clock position on the clock face, have had the kite kind of pinned at the 12 o'clock on our clock face if the kite is tied off at the center of the clock face. So won't winds that instead come from the 9 o'clock on the clock face push the kite closer to 3 o'clock? Of course, we would want to think things through before making that prediction with any confidence that it's going to move and maybe even where it's going to end up. We want to ask ourselves questions like these, like, are those weather stations reliable? Are they actually experiencing change wind conditions or is it perhaps a glitch in the system? Also, if we believe the data from those stations, is there any reason to suspect that those conditions will not make it to us? Will those conditions die out before they get here? Or will they change direction even before they get here?"

(08:43): And another question we want to ask ourselves is, do we see any other forces which might work to counteract those change conditions? If those wind changes do get to us, is there anything that's going to resist them? For instance, if someone is attached a drone to the kite and has the ability to kind of forcibly move it around in the sky, well, we want to factor that in, right? Now, let's apply this metaphor to the Covid example. What did we know in early 2020 when we were thinking about whether or not we should hire more consultants? We knew that the virus had been spreading for a month or more in China. We knew the Chinese government was taking it very seriously. We knew that there were mobile morgues being set up and those morgues were busy. We also knew that the Chinese government was making moves that had tremendous economic cost.

(09:27): Now, the Chinese government, let's face it, isn't known for caring about its citizens nearly as much as it is known for caring about its economic strength. And yet here they were shutting the entire country down, absolutely tanking their economy. They were literally welding people's apartment doors shut from the outside in order to enforce a lockdown. And we also knew that the virus, like all viruses, spreads exponentially, which means that things can appear to be moving slowly for a while before suddenly exploding higher.

(09:57): Now, all of that information was available to all of us. We all knew that was going on. That was a news story. It was just kind of happening over there. It was happening elsewhere, right? So I ran down the checklist, thinking about China as if the weather station's an hour away. First of all, were those reports reliable? Well, yes, they were widely confirmed by a wide spectrum of news sources. Now, was there any reason to suspect that this wouldn't spread to the USA? Would it not reach us? Well, no. Travel lockdowns weren't being enforced at all yet. And in fact, we were already getting isolated reports of infections in the US.

(10:29): Lastly, did we have any reason to believe that once it was here, it would have a different impact on us than it was having on China? And no, there wasn't any such reason. I was left with all arrows pointing to this, the US economy was going to experience a massive interruption, like a shock at least as severe if not more so than the 2008 financial crisis turned out worse, and that would lead to businesses taking defensive positions to ride it all out. We were going to see air travel lockdowns. And while adversity does tend to be very good for the data industry in the medium and long term, you have to ride out the short term spending clampdowns in order to get to the other side.

(11:09): So when you're done with an analysis like that, you're left in the position that it's overwhelmingly likely that the status quo is going to rapidly move to a different point in the sky. In fact, it would be a far more preposterous belief to say that things were going to stay put. So to put a bow on the Covid predictions, it wasn't some crazy foresight. It was just taking the information that was available and following it to its natural conclusion, which involves and requires understanding that the status quo is like a kite on a string in the sky and not something massive that resists change and resists movement.

(11:44): That was the same way with the prediction in 2010 of what we're now calling the citizen developer movement. The new software approach to be being pioneered by Microsoft was understandable by the Excel VLOOKUP and PivotTable crowd. Those people were a lot closer to the business problems than the isolated developer types who had been the implementers of BI projects in previous decades. In the hands of that audience, which by the way is a massive audience, bigger than any other developer audience in the world, in their hands, these tools unlocked a project pace that was easily 10 to 100X faster and more efficient than the traditional pacing of a BI project that we'd come to expect.

(12:22): Now, unlike in the Covid example, the observations I was having in 2010, they weren't really broadly available to the general public yet. So I was benefiting from a little bit of special access to the information for a time there. It had lot to do with my career arc and the path I had followed, and there was absolutely some luck that put me in that position, but it still suggested to me that the kite was going to move and move relatively dramatically. Was I getting a reliable signal? Yes, I was.

(12:49): I was a member of the Excel VLOOKUP and Pivot audience, and I had just finished re-implementing a BI project using Power Pivot that I had previously paid to have executed via traditional consulting methodology. And the results were overwhelmingly clear. It was way faster and way better, and a tiny fraction of the time and an even tinier fraction of the cost, I was getting much better results than I had from the first time around.

(13:15): All right, did I see any reason why this change wouldn't make it to the broader market? Again, no. Microsoft had just invested an entire release of its flagship BI product analysis services into Power Pivot. Too many careers were now tied to this risky move and they weren't going to change course. Plus, Microsoft always plays a patient game and figures things out for a while before changing course. Heck, they're still doing Bing.

(13:39): Lastly, did I see any forces in the market which would counteract this disruption? Well, I didn't see any, even though it turns out that there were some that emerged, like I mentioned before. The tie to Excel ended up being an albatross rather than the benefit that I predicted it would be. But the more important forces were still the incentives of companies and individuals in the world. Nothing is more valuable than accurate, actionable, and timely information. And if the cost of acquiring it suddenly drops by one or more factors of 10, market pressures are going to find that new way like water flowing out of a tank through a new and larger pipe that opens up. And those incentives, both of the market as well as Microsoft's, took care of the Excel problem starting in 2015 with the debut of Power BI.

(14:28): So in both cases, the Covid example and then the Citizen Developer 2010 Power Pivot turning into Power BI thing, the predictions strike me as no more inspired than saying a ball will roll down a hill if you give it a push or that a kite will move when the wind changes. It's just that by default, we as humans seem to have an override installed in our brains that says, "Nah, things are going to stay the same." It's a comforting lie. We tell ourselves to spare us from uncertainty. I get it. No one likes the angst of uncertainty. But the thing is, when we suppress information from ourselves about a coming change, we're not protecting ourselves from uncertainty. Quite the opposite in fact. We're just kicking the can down the road and making ourselves much more vulnerable to feeling insecure and scared when that change arrives anyway. A little bit of discomfort now spares us a much greater discomfort later.

(15:24): Just imagine the discomfort we'd have experienced if we had followed through on what the data was telling us in our internal financial dashboards, and we had run a new round of consulting hiring in early 2020. When lockdown later hit the US, we did see some of our clients put a hold on spending. We also saw our pipeline of new clients completely dry up. It went from robust to zero basically overnight. And these changes invalidated all of the predictions being made in that financial model. And if we had hired like we normally would, we absolutely would not have been able to keep those people on staff. That would've been awful to have people leave their existing jobs only to have those jobs evaporate, leaving them suddenly unemployed. Whereas if they'd stayed put, they'd likely still have a job. Not to mention the impact on our business. We'd have lost valuable time and money on those new hires and created an unsettling morale problem by showing our team that we could be so badly blindsided like this.

(16:22): We ended up in 2020 having to tighten our belts to get through the change conditions, but in the end, we kept everyone and even managed to grow our business by 7% that year. That wouldn't have happened if we'd followed the status quo and we'd followed through on what our dashboards were saying. We would've paid dearly for that extra few weeks of self-comfort.

(16:41): Imagine the discomfort I'd have felt all these years later if I hadn't trusted what my eyes were telling me about the BI market and someone else had eventually run with the idea of an up-tempo higher efficiency data consulting model. It's also worth noting in both cases that I ended up temporarily and kind of uncomfortably on the outside of the herd. If everyone else is leaning in on the comforting truth that the status quo is durable and you're the one saying change is coming, they're going to want you to quiet down or go away. Even if you're legitimately sincerely trying to help them, you are ruining the self-comfort that they've created for themselves. And so they're naturally prone to shoot the messenger.

(17:19): I was legitimately trying to mentally prepare that person who was going to her brother's wedding that she was potentially going to face difficulties getting home with her husband and her one-year-old child. And she wasn't mean to me in that moment or anything, but I was aware that the dynamic in the room changed and suddenly I was being viewed as some mad scientist type. There was a social awkwardness in that moment. I was confident enough in what was about to happen that I was willing to endure it and later be vindicated, but still, it didn't feel great in the moment. Of course, I only had to wait a few days in that case.

(17:49): In the prediction about the BI market, it took years. And since I was publicly sharing my prediction of the world's future state on my blog and elsewhere during those years, a fair amount of shoot the messenger energy did come my way, and it makes sense. Most of the community who was tuning into my blog at that time came from the traditional industry. And now here's this recently ex-Microsoft guy, me, telling them that their business model is at risk and that a new alternative way is springing up. Totally makes sense. And in hindsight, I handled those conversations, those exchanges poorly.

(18:23): My confidence in the future worldview never wavered, but the criticism still got under my skin. I guess I kind of just expected everyone to see the things I was seeing and reach the same conclusion that the kite was going to move. But they weren't seeing a kite, they were seeing a great pyramid because they felt invested in that status quo. I didn't understand at the time that that was the disconnect. I thought incorrectly that it was a disagreement over facts. And as a result, I got pretty emotional in response to these very intelligent people disputing things that honestly were kind of beyond dispute.

(18:56): So those conversations could have gone in a much more positive direction than they did if I had reacted better. I could have changed a few more minds if I had responded more gracefully. Rather than leaning in on the binary yes/no fight over whether the old model was in danger, I could have instead helped them see that opportunity was merely going to expand for them, which it did. And it turns out that many of my biggest critics in that era ended up today being celebrities in Power BI.

(19:21): Now, all those lessons are paying off again for me today in the context of AI-driven disruption. The wind is changing and the kite is going to move. It really hasn't moved yet very much anyway in our industry, but it's coming, both in terms of the services we deliver to our clients and the ways in which we deliver them. And thankfully, this time I'm responding more gracefully and less emotionally when I do get pushback when I share the picture that's evolving for me. If you want to hear what that picture looks like, make sure to listen to our recent episodes about AI, particularly the one titled My New Clarity and Confidence about AI and also Low Tech Industries and High Tech AI.

(19:57): So let's run the kite process on AI real quick. The way our company, P3, operates today is a kite in the sky. Our kite is in a different place from our traditional competitors kite because those firms have still somehow managed to preserve their exploitive model for now despite the changes in the way the tools work. But let's pretend for simplicity that all data consulting firms are represented by a single kite, and here comes a win change in the form of AI. All right, let's run the tests. First of all, are the signals real? Oh yeah, they're real. Generative AI chat bot interfaces represent a sea-change for all software because they're the first time a conversational interface has ever worked despite decades of prior attempts, this is the first time it's ever actually truly worked. And this conversational interface is probably going to be at least as big of a revolution for all software, as was the graphical mouse-driven user interface that we now take for granted in Windows and macOS, and also at least as big as the touch interface we now take for granted on our smartphones and tablets.

(21:03): Remember, the graphical and touch interfaces when they were first invented didn't instantly pervade all software overnight. Keyboard oriented, non-graphical versions of Lotus and Excel, for example, persisted for a long time before going extinct. So it's going to take a while for generative AI conversational interfaces to make their way into everything. We're still just on the starting line there. And think about this, not only did the graphical and touch interfaces make their way through all commercial software and not only did those interfaces open up completely new categories of tools that weren't possible before those interfaces were invented, they also eventually made their way into custom solutions built for and by individual companies. Homegrown internal line of business apps very much take advantage of graphical and touch oriented interfaces. And heck, even the Power BI dashboards built for your company rely on and take advantage of those interfaces. So generative AI chatbot interfaces? Yeah, coming soon everywhere and changing all the rules again.

(22:08): So next test. Are these forces going to peter out before they get to us? Hell no. Have you seen the initial price tags on things like Copilot? These AI assistive add-ons are more expensive than the original software itself. The software world of the early '90s with its magical ability to boost productivity unlocked profit margins for itself that few other industries could rival. And now the software industry is starting to suspect it can unlock a step change higher profit margin? No, they are not going to give up on that quest. This stuff is coming, and it's coming to everything. And most importantly, it is good enough that it's going to supplant at least a significant portion of how we've interacted with software up until this point.

(22:54): All right, final test. Is anything going to block or deflect these forces? I don't think so. It's not like companies are going to reject this sort of change. If they can boost productivity of their current headcount and/for get away with a reduced headcount, they're going to do it. Even the price tags on these AI add-ons aren't going to be a deal breaker, which is why the software firms are setting them so high in the first place.

(23:15): We're also never going to see any sort of government regulation block this sort of thing. Too much money's at stake. Plus, there's even like a geopolitical national security kind of rivalry here. The USA wants to outrace China and vice versa just to name one competitive pair. And there's no way any government is going to hamstring its own domestic companies in such a race.

(23:34): On net, I find myself in a very similar place to where I was when I was seeing the 2010 BI revolution and when I was watching Covid making its way toward our shores. You to accept that change is coming, it's going to happen no matter what you do. It's going to happen whether you like it or not. And once you get through that, suddenly things immediately start to feel better. It's not all bad news. Once you sift through the noise around AI and develop a semi-confident picture of its capabilities, which takes work as I acknowledge in the intro to that Low Tech Industries episode, if you then take the exercise of looking at the status quo in your industry as a kite in the wind, I bet you might start seeing AI-driven opportunities that you weren't seeing before. Because remember, disruptive change like this is not all bad news. It's bad news if you sit still and ignore it for sure. But if you deprive yourself of that short-term deceptive comfort, you'll quickly reach a new comfort, which is that you can surf this wave to new places.

(24:36): Yes, on the negative side of the scale, some of the ways you operate today might be at risk, but the doors that are opening up in the process far outweigh that on the positive side for those who are willing to look. Others will undoubtedly stay the course and get crushed by that wave. Sadly, it's true. But it's really truly not that hard to turn it in a positive direction once your eyes are open. It's a normal power and it's within everyone's reach.