Customizing AI for Your Business is Crucial But It Isn’t Hard

Announcer: Welcome to Raw Data with Rob Collie, real talk about AI and data for business impact. And now CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie: Hello, friends. In last week's episode I talked about how big tech firms are spending enormous sums of money on AI, R&D and build out, [00:00:30] and how that spending currently dwarfs, just dwarfs, the revenue that they're bringing in. And I want to follow that trail of breadcrumbs a bit because I think it leads us to something important for our own businesses and careers.

Now we know why the AI stuff is expensive to build, it's unlike anything the tech world has ever seen. In the past tech innovation merely required you to hire and pay a software engineering team. Now, that's not a cheap thing by any stretch. But if for example, we assume [00:01:00] something like Excel, one of the largest and most complicated software products on the planet requires a team of 100 engineers making an average of let's say $500,000 a year, that's a $50 million a year expense. Again, very much not chump change.

But AI R&D is very, very different. First of all, the researchers are incredibly expensive. Famously, a single researcher recently secured a $100 million comp package [00:01:30] spread over multiple years. So you could have something like the entire Excel team for two years, or you could have that one person for a few years. And that's just the beginning because there's tremendous hardware expense. You have to buy expensive GPUs like by the truckload. NVIDIA, who makes the bulk of those GPUs, is currently the most valuable company in the entire world. That's just because of how expensive GPUs are and how many are needed for AI R&D and also for ongoing [00:02:00] AI operations.

Then of course you also have construction expenses. You have to literally build data centers to house and run all of those GPUs. When you're done with that, then you get to the tremendous power consumption, the energy expense. A single rack of GPUs in a data center can consume 140 kilowatts of electricity. That's like running 140 high-powered microwave ovens or 80 hairdryers 24 hours a day, 7 [00:02:30] days a week. I charge my electric SUV overnight with 7 kilowatts so I could charge 20 of those electric SUVs in parallel with the power consumed by a single rack.

You don't have to do any of that if you want to build a software product like Excel. You, quote-unquote, just need to pay software engineers. There's no physical footprint, no massive server farms to buy build power and cool. Back in the AI [00:03:00] world, OpenAI has announced that it plans to spend $17 billion next year alone and then more than doubling that to 35 billion in 2027 and continuing to balloon to 45 billion in spending for 2028. You could run 900, 900, Excel-sized software teams for the amount of spending that OpenAI plans to commit in 2028. But it's no mystery where all that money goes, it [00:03:30] goes to all of the above.

So I think the more relevant question is on the revenue side, why isn't revenue keeping up with all of that spending? We keep hearing that AI is going to revolutionize business, that we're going to have digital colleagues, that drudge work is going to be eliminated and productivity is going to skyrocket, and that smaller companies are going to be gifted capabilities that formerly were only affordable by the Fortune 500.

Now, do I believe all of that? Do I believe all of that is coming and feasible? [00:04:00] Yes, I actually do because I have seen use cases that open my eyes and convince me. That vision of AI revolutionizing the workplace, that's not a myth, it's real. It's just that the world is slow to catch on. In fact, even though the AI firms are racing to spend vast amounts of money on ever better LLM models and capabilities, we don't even need the new ones they're working on to make this a reality. The use cases that have made me a believer are 100% [00:04:30] possible with today's LLMs. But crucially none of the use cases that I've been seeing, real-world use cases that have completely convinced me that AI is for real, none of them were off the shelf use cases.

What do I mean by off the shelf? I mean just using the AI as is, like logging into ChatGPT and starting typing. By now you've probably seen the ads from ChatGPT aimed at personal use. What should [00:05:00] I cook for my date to show her that I like her but that I'm not too desperate or clingy, or how do I get to the point where I can do 10 pull-ups by the end of the summer? That is what off the shelf looks like. Help me with things that are basically the same for me as they are for everyone else without requiring much, if any, customization. Those ads on TV during football games are basically an admission that use cases like those are the only use cases that work off the shelf, [00:05:30] and these are very much not business use cases.

Publicly available estimates indicate that like 60 to 70% of OpenAI's revenue today comes from individual consumers just like that. But they're still hemorrhaging money, I mean, not that they really care, but they're hemorrhaging money because business use is going to be the ultimate driver of AI revenue. Just like business use is the driver of most software and cloud revenue today. Business productivity is what's really going to make the AI [00:06:00] cash registers ring.

But even though business use is the reason for all of this massive investment, OpenAI is currently choosing to advertise use cases on TV that boil down to help me impress girls. A time-honored pursuit for sure, but not exactly worthy of all that investment. And this highlights another critical difference between the AI revolution and say the spreadsheet revolution of the 1980s and '90s. Excel worked great off the shelf. [00:06:30] Sure, you needed to learn how to use it just like you kind of need to learn your way around ChatGPT a little bit, but you didn't need to build any custom frameworks around Excel to make it work. You didn't need to give it instructions or teach it about your business other than just learning to write formulas.

By contrast, LLMs like ChatGPT at first glance are breathtaking in their off the shelf form. They're both easier to use and far more wondrously powerful than Excel in a lot of really critical eye-opening [00:07:00] ways, and that gives us the false impression that it's going to be just like that for business use. So then we sit down with it all optimistic and start asking it for help with things like optimizing production schedules and we just get nowhere. It becomes instantly apparent that it's both really dumb about the particulars of your business and also that educating it, via lots and lots of typing or dictation or copy-paste, is going to be a lot more manual work than it's worth, [00:07:30] and then that experience gives us the false impression that AI is no good for that sort of thing.

So we bounce off of AI for business use and we go into a holding pattern, and that is why AI company revenues aren't keeping up with their spending. It's as simple as that. Business use cases don't work off the shelf and we don't know where to go next. So to break out of that holding pattern and to help others break out of that holding pattern I've taken to describing [00:08:00] these AI systems as being magic LEGO bricks. They do some really cool things on their own in the off the shelf case when you go to Chatgpt.com, but they really shine when you use them as part of a bigger LEGO model. And those bigger LEGO models that you build are made mostly of regular bricks, simple non-magical stuff.

Now, I keep coming back to this particular use case because it's one of the few that we're allowed to talk about publicly, but the vendor bot that [00:08:30] Bill Krolicki and team built at International Packaging, which we covered in a previous episode is a perfect example of this. It uses GPT-5, it uses the magic LEGO brick, but it doesn't use Chatgpt.com as the way to access the magic LEGO brick. It uses a lightweight system made out of regular bricks, Power Automate scripts making requests to Power BI to determine what materials are on order with upcoming due dates. [00:09:00] That's very much regular old non-magic LEGO brick stuff. And then it writes emails, these are form emails populated by the information retrieved from Power BI. Again, no AI, just regular LEGO bricks. And then there's a human in the loop, a human being briefly reviews each email before pressing send. It doesn't get more traditional than a human in the loop.

When email replies come in to those emails requesting, hey, is our stuff going to be on time? There are again, [00:09:30] regular LEGO brick type systems watching for it, just watching for emails from certain people with certain subject lines, no AI magic there either. But then when such an email is detected, the regular LEGO brick system that detected it sends each email reply to GPT-5 along with a detailed set of instructions, instructions that were written once and reused every time, instructions that ask GPT-5 to decide whether the email reply [00:10:00] indicates that the materials are going to be on track to arrive on time. So they get back from GPT-5, a yes, a no, and a degree of certainty. That's it. That's the one place where the magic LEGO brick is used. And then regular LEGO bricks, again, take it from there, ultimately resulting in certain individuals at the company getting notified of potential problems.

That isn't hard. It's not rocket science, it's not AI research, but it definitely is not [00:10:30] off the shelf and that's the gulf we all need to cross. Once you've gotten comfortable with off the shelf usage just using chatgpt.com or whatever, you're kind of left wondering what's next, and it's a big air gap jumping to your first custom solution. The custom solutions aren't hard, but there's no clear transition path.

To feel a little bit better about this remember that even something as comparatively simple to understand as spreadsheets took many years to reach widespread adoption, [00:11:00] the better part of a decade even, if not longer, because it's so hard to imagine something until you've seen examples of it working. Some companies back in the '80s just bought PCs for their employees, installed spreadsheets and hoped for a miracle. That's kind of what is happening now with companies buying AI subscriptions for their employees. But off the shelf usage of Excel was an on ramp to the miracle. One person at the company could achieve an off the shelf breakthrough and then show their colleagues and [00:11:30] it would spread from there. Whereas off the shelf AI usage doesn't lead to the same business breakthroughs. There's no spark. It's so much harder to prime the pump with AI. But that doesn't mean the gap is impossible to cross, nor does it require you to reinvent yourself because, again, customized usage of AI is mostly about regular LEGO bricks that mysterious.

Internally here at P3, we're now developing a suite of customized AI agents that are going to radically [00:12:00] improve our internal business processes across a wide spectrum of departments and functions. And guess what? Most of the construction of these customized agents, customized workflows, most of it is regular LEGO bricks with an occasional usage of the magic AI, LEGO bricks. I'm really keen to share more about these things once they're in production and early indications are very, very exciting, but I want to wait until we're overly sure of success before trumpeting them here.

As soon as we reach that [00:12:30] point though, we'll be very eager to share because all it takes is a handful of customized use cases to make the light bulb go on in your head. Spotting those customized use case opportunities at your company is the most important muscle to be developing right now, and the best way to do that is just to learn from examples. So stay tuned because it won't be long. Until next time.