Jensen Huang’s Reindeer Games, Agent Frameworks vs. Fully Custom, and Rapid Impact vs. Technical Debt

Announcer (00:04): Welcome to Raw Data with Rob Collie, real talk about AI and data for business impact. And now, CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie (00:20): Welcome back, Justin, to the recording that you had to leave and come back to. We do have something we want to talk about today. We want to talk about agent frameworks. But first, we have to address... I'm sure this is a plague on the entire world right now.

Justin Mannhardt (00:36): It's awful.

Rob Collie (00:37): This is inflicting tremendous productivity loss all over the world. It's got to be some recent Windows update, where whatever meeting software you're using tells you that your camera is in use by another application, and there is no other application that's using it. Windows is telling the meeting software that the camera's in use. I thought it was a Teams bug, because it hit me in Teams.

Justin Mannhardt (01:02): Same.

Rob Collie (01:02): But it just hit you in SquadCast.

Justin Mannhardt (01:05): By now, it's hit me in multiple video platforms and with multiple cameras, including my external webcam and the integrated one on my laptop.

Rob Collie (01:15): Okay. So absolutely something in Windows. Windows broke something.

Justin Mannhardt (01:21): And somebody was telling me you can go into Windows settings somewhere and you can see what app is currently using.

Rob Collie (01:27): Oh yeah.

Justin Mannhardt (01:27): Nope.

Rob Collie (01:27): Yeah. I think it was me that told you that, I think it was me, because I was trying to solve the same problem. And I go and I turn everything off that's supposed to have access, and still in use. So if you're out there and you are being plagued by this, we feel you. And I know that people out there are hearing going, "Oh my god, I'm glad it's not just me."

Justin Mannhardt (01:48): I'm going to deep breath, I'm going to go through my calendar for next week, I'm going to add five minutes before every meeting.

Rob Collie (01:54): For reboot.

Justin Mannhardt (01:55): I'm going to restart the computer before every meeting.

Rob Collie (01:58): Well, you know what we could do? We could write something that all it does is every five minutes check to see if the camera is in use as far as Windows is concerned. When that indicator goes red, you reboot.

Justin Mannhardt (02:10): Yeah. I've got to get Claude on this problem.

Rob Collie (02:13): There's an agent for that.

Justin Mannhardt (02:15): And then, I'll throw it out open source, like camera fixer-uppergus. You know, open source.

Rob Collie (02:19): Not open source, no, no, this is a subscription. Shut up and take my money.

Justin Mannhardt (02:32): 99 cents a use.

Rob Collie (02:32): Yeah, seriously, take my money.

Justin Mannhardt (02:32): I'll make 100 bucks before the end of the day.

Rob Collie (02:34): Think about the savings to productivity and frustration. I don't know what the right price is, but it's not tiny, it's not a micro transaction.

Justin Mannhardt (02:41): Are you tired of showing up to calls with your prospects and can't get your face on the screen? What's that worth to you?

Rob Collie (02:48): Well, plus, it also starts to erode the culture of people having their cameras on. "Well, Jimmy doesn't have his camera on. I'm not going to have my camera on." Next thing you know, you're all just staring at blank screens again. Not good. I had another banter-y topic. Let's talk about NVIDIA CEO Jensen Huang. When I say that name, NVIDIA CEO Jensen Huang, you can probably already, some of you anyway, can picture him. But if you haven't seen him, he's on stage in his black leather bomber jacket, and he's on top of the world.

(03:22): So many of these big tech companies are in a very insecure state right now. If you're CEO of Microsoft, you're CEO of OpenAI, you're CEO of Anthropic, yeah, it's good to be you, but you have a lot of stress. There's existential dread in your life. We all know that not all of these big AI players are going to survive and have a chair at the end, not at this trajectory. There's too much money sloshing around and there's only so many seats. The musical chairs game is going to end at some point. Some people are going away.

(03:58): And in the meantime, Microsoft, they're not going away, they've still got a lot of dread too, because everyone understands why I'm paying a premium subscription to OpenAI for AI because they have the model. Why do I pay a premium subscription to Microsoft for access to APIs that I already have access to? In terms of psychological field position, Microsoft has really good field position in some ways, and in other ways, their neck is exposed.

(04:23): Everyone is feeling angst and uncertainty and dread, except for Jensen Huang of NVIDIA, because it doesn't matter. His company sells the chips that all of this runs on. It doesn't matter if OpenAI goes under, who cares? And in the meantime, everyone trying to spend each other into oblivion means that they're buying the hell out of his product, he's the beneficiary of all of this. But it occurred to me as I was writing a chapter of my book recently, the last couple of days, which by the way, this chapter is... Maybe it's one that I'm just going to completely drop. It's a good chapter, might just be too much information, might be too much interruption to the flow of the book, but I can't cut it down until I've written it, so I just have to get through it. It's explaining what is a GPU and where did it come from and why is it graphical, if it's AI, why is it graphical.

(05:17): So it just occurred to me how funny this is. If you rewind five years and you're at some Silicon Valley dinner or charity gala or something, all the fancy people are there. Jensen Huang, they're going to have him sitting at the equivalent of the kids' table five years ago. He's the one whose company makes gamers' frame rates better or gives them better textures or better glow effects. "Well, look, we run through water and it looks more realistic." Shut up, sit down, we're over here at the adult's table talking about serious things. He's neural networks. It doesn't matter what his net worth... Well, no, no, not yet. All that stuff was still kind of fringe-ish. It was still there, like, okay, so there was something kind of cool. But until OpenAI and ChatGPT dropped the world on its head, he made chips for gamers and for Bitcoin miners. Again, I know you're wealthy, Jensen, but you're not serious people.

(06:23): Oh, how the turn tables. I mean, he has got to be loving this. I'm positive they left him out of the high status Reindeer Games. Now, who's eating out of whose hand? It's like, uh-huh, yeah. We had a client many years ago, I was meeting him somewhere in Indy, He happened to be visiting Indy, I hit one of the famous Indianapolis potholes. Indianapolis has some of the worst-

Justin Mannhardt (06:49): Oh, man.

Rob Collie (06:49): ... absolute worst potholes in the country. I think they can take the Pepsi challenge with anyone. They probably have the worst potholes in the entire country. The number of rims and tires that were lost by the Collies alone in Indianapolis to Indianapolis potholes is insane.

(07:05): So anyway, I hit a pothole on my way to meet him, and it bent the rim, flattened the tire, parked it in a parking lot, called an Uber. After we have our lunch and we hang out together, he drove me back to my house in his Bentley. That's the one and only time I've been in a Bentley. I'm very much in the non-Bentley owners club. Because he was a Miracle Ear franchisee, he owned a lot of Miracle Ear franchises. He was like, "Yeah, I mean, it gets me a Bentley. But trust me, this doesn't get you a lot of respect at the country club."

(07:35): Jensen was living the equivalent of that. I'd never thought about that. I'd never thought about once you reach a certain level of wealth, people care how you made your money. Talk about a problem I've never had to really deal with. But the irony of Jensen now being the only one with a seemingly unassailable field position... And I know others are working on chips, I've seen some things about chips, new chips coming up. But really, this is a high barrier to entry business.

Justin Mannhardt (08:04): Oh, extremely, yeah.

Rob Collie (08:05): This is not software. This is fabrication of just ridiculously complicated stuff. And if you've been doing this for 33 years like NVIDIA has, yeah, come on in, bring it on. You'll start to see Jensen being like Neo in The Matrix, or no, no, no, it was Morpheus with the little beckoning flick of his hand, like, "Bring it, bring it."

Justin Mannhardt (08:29): Oh yeah. I saw he was doing a product reveal, and there was this big rack of servers that must have been like eight feet tall, and he was holding this thing in his hand that's the size of a small laptop, and it was like, "This thing has like 10x the computing powder of that whole rack." I was like, "You guys are real smart."

Rob Collie (08:50): Yeah, yeah. Seriously, they're dealing with real world constraints, material constraints, physics. They measure things in nanometers, nanometers, and it's single digit. What's nano again, is it a millionth?

Justin Mannhardt (09:05): One billionth.

Rob Collie (09:08): One billionth. A nano is a billionth. Holy hell. The wires, if you want to call them that, the wires in these chips are down to single digit billionths in width.

Justin Mannhardt (09:17): Huh?

Rob Collie (09:19): Yeah. We made fun of the Rudolph song recently, one foggy Christmas Eve-

Justin Mannhardt (09:27): Jensen Huang came out.

Rob Collie (09:28): Jensen Huang was like, "Oh, my gaming chips, that's what you need, huh? Okay, well, Yahtzee." Now, it's like the rest of them didn't involve him in their Reindeer Games, he now runs the Reindeer Games, they're his games.

(09:43): All right, switching to the real topic. We've been hearing a few things. So first of all, we're seeing many, many, many agent frameworks springing up, everyone's got one and some people have more than one.

Justin Mannhardt (09:55): And what's a non-tech biz leader-friendly definition of an agent framework?

Rob Collie (10:02): That's a great question. So given our guiding principle that successful business AI is custom AI, so it has to be customized, and if you're going to customize it, are you sitting down and building all of the non-magical Lego brick parts of the solution, are you building them from scratch in raw code, or are you leveraging some framework that makes some of the really common functions that you would have to write over and over and over again, makes them table stakes?

(10:40): From that point forward, there are many different flavors of agent framework. There'll be ones that claim to be low code, no code, where you're literally stacking blocks together on the screen, like I'll just drag one of these on Visio or something like that. And then, there's others that I'm sure you're still writing code, but there are lots of parts of the solution that you're not having to code over and over again. Again, Jensen doesn't care. He doesn't need to offer agent frameworks. It's everyone else that needs to offer them. These are coming from everybody, from the AI players themselves, as well as from people like Microsoft.

(11:13): We're also encountering stories from the field, particularly at large enterprise shops, where they have an AI solution that they want to build, they have a workflow that they want to build whatever, an AI agent or an agentic system or workflow, use whatever nouns you want, they want to build a thing, an AI thing, and they're making a really big deal about it. They want it to be done, in a way, almost heavy infrastructure way, where they're going to take maybe months to do it, many months, and maybe even working with big software vendors to get the big software vendors to add things to their platform so that the enterprise customer doesn't have to have much custom code running. Enterprise IT in particular knows the drill. Every line of custom code that piles up in their operation, basically, it goes into the liabilities side of the IT balance sheet.

Justin Mannhardt (12:11): Correct, yeah.

Rob Collie (12:12): And so, their job gets harder, and the surface area for failure gets larger, and they know that no other executive is ever going to ever understand that, they're just going to know that something broke. So it's the classic trap that IT finds themselves in, it's like, "No one praises me when the lights are on, but when the lights go out, I get destroyed." They know that the less that they're responsible for, the better. This has been beaten into them. They come by this honestly.

(12:41): The business is coming to them, because it's not IT's idea, the business is coming to them saying, "Wow, we really need X, Y, Z." And we look at what XYZ is and go, "We, P3, no problem, we'll have that done in a week., In a week, and it'll be solid, it'll be robust, it'll be amazing." But yet, this has become some massive, almost political thing between this enterprise customer and a big player like Microsoft or a big player like OpenAI. It's wild, all this pressure building behind this dam. There's people like me that are just raising my hand going, "Hey, folks." Remember in the movie Die Hard, they're trying to shut down the power grid, and there's the one guy who's down in the manhole?

Justin Mannhardt (13:31): Yes.

Rob Collie (13:31): The big wigs are standing there talking like, "No, I can't do that without authorization, blah, blah, blah. It's going to take forever to turn off this grid." And then, the guy in the manhole is raising his hands like, "I can do it right here. No, really, I can just do it right here. Right there, there's the switch." I kind of feel like that guy. And so, in the same way that IT is over-indexed to the, "We need to be careful about what we're responsible for," I'm much, much, much more, and I've always been, more on the other end of the spectrum, like, "No, let's go do something that advances the business."

(14:07): And so, I look at these situations that I'm describing, they're just on hold for months at a time, trying to figure out if they can do this in a way that heavily minimizes IT's responsibility for the custom solution, because it has to be custom, it has to be custom. Does custom mean a bunch of brand new Python code or whatever? I don't know how this resolves, because the organizations that say, "You know what? Let's go empower the business, let's go make the business better, and we will accept that that changes our whole equation in terms of what we're responsible for," I think those people are going to gain a lot of advantage.

(14:52): Now, whether it comes back to bite them is a separate question, but in the short-term, they will gain a tremendous amount of advantage. And if you're more mid-market listening to this, I think you're going to have a lot clearer sense of this, like, "I can turn this on and things get way better." I don't think most mid-market companies are going to have this same angst about turning on the magic.

Justin Mannhardt (15:16): I think there's some familiar psychological things going on here, the different scale of impact. A couple of thoughts that have been kicking around my brain the last couple of days. One is that, psychologically, it reminds me a lot of trusting the cloud. It took a long time for the world at large to trust data in the cloud and the social proof that needed to be established. And you mentioned, yeah, but the impacts are way different. I think that's true, but I think mentally, that's still what's going on, is that confidence is viewed as a risk by some of these ID departments, because yeah, they're responsible for protecting their company's information and it's scary, on one hand.

(16:00): On the same token, when I take a step back and I understand... I try to understand. Understanding AI is difficult. I try to understand what's going on with LLMs... Big news this week, both OpenAI and Anthropic both dropped new models, context windows getting bigger, capabilities getting broader. The solutions that involve these systems are inherently fluid and custom, and they're inherently getting better and more capable. And I think you're right, the teams that find a way to just embrace that and get early, boy, is that important.

(16:39): And I think for leaders, I've talked to a number of companies over the past several months, this would be a challenge I'd have is if the AI mission has been delegated down into your company, you might have a problem. You yourself probably need to figure out how to get that visceral understanding of what's happening. We're way past chat.

Rob Collie (17:05): Yeah. Talk about a how the turn tables moment. I'm going to take a moment here and feel good about being us. If you're CEO of some Fortune 500 company, you are wired in a way that I am not. You are able to do things that I am not able to do, whatever they are. No, I do not belong in that kind of a role. But at this moment, I understand what they should be doing about AI way better than most of them do, I'm just confident about that. It's very rare for me, of my self-deprecating nature, to say such a thing.

(17:44): But oh yeah, if I were allowed to have a conversation with any random enterprise Fortune 500 CEO in private, where they wouldn't have to be afraid of being judged by their peers or whatever, they would really benefit from that conversation, and it's very rare that there is anything like that. We're in a situation now where I, Rob Collie, would be valuable to them in that conversation.

(18:06): And most of the time, even in the world of Power BI, when it was brand new, cutting edge, not really. I still understood it better than them, but it wasn't going to be that helpful to them to have that conversation. It'd be like, "Oh, you were talking about the dashboard thing." Okay, that's one tiny little part of the overall problem. AI is not one tiny little part of the overall problem, it's huge now. It's like our Jensen Huang moment.

Justin Mannhardt (18:29): Yeah. This is an interesting thought experiment. Another way that I've described a framework to people, an agent framework, is we're all familiar at this point with something like a ChatGPT, we can go talk to it. A framework is something where you've got an LLM, tools like code, resources like documents, and maybe a bunch of things and maybe a bunch of different LLMs, and they're working together. And let's say everything keeps getting better, the LLMs keep getting better, the tools keep getting better.

(18:58): And so, let's say there's an LLM that's really smart about contract law and it's got a tool to write a contract, and let's say it writes that contract and it's not consistent with the way your company's going to do it. Okay. Well, what you're going to end up doing is you're going to go tell the contract law agent, "Hey, here's how we do it." And it's going to change some code and change some rules, and now it's going to be custom. People are like, "Oh, we use the OpenAI thing," or, "We use the whatever thing." That's not solving for this need to get in there. You're going to have to get in there. And I do think things will be getting smarter and smarter and smarter, but it might be easier for you to get them up to speed, but you were just saying an example of, "Oh, we'd do that in a week." It's already really insanely powerful.

Rob Collie (19:50): Now, what sort of debt does that incur? Let's say we go into this Fortune 500 company, that has this AI workflow problem, that they're negotiating and having high-level meetings with big software companies and vendors. They're going to drag this out for six months before they even decide what they're going to do.

Justin Mannhardt (20:08): There'll be four more releases of Opus by then.

Rob Collie (20:10): Yeah. And again, I'm down there in the manhole cover saying-

Justin Mannhardt (20:15): "I can do it."

Rob Collie (20:15): "I can do it right here." So let's say we did it our way, and we even deliberately went out of our way to not think about the maintainability of it, which we wouldn't do. Let's say we just shut our ears and eyes to any maintainability problems. Look, all that we care about is getting this solution running and lighting the business up, just transforming the way that they work in an incredibly positive way. What debt would we be incurring? There's an oversight thing. Does IT even know that that agent exists?

Justin Mannhardt (20:45): Correct.

Rob Collie (20:46): It's like the spreadsheets that used to be, still are, on people's desktops, running businesses, that IT is completely unaware of what the spreadsheet is doing.

Justin Mannhardt (20:55): Now, with Copilot.

Rob Collie (20:56): Yeah, yeah. Back to agents and frameworks. Okay, so this custom solution that we build in a week, again, if we completely close our eyes to maintainability and all that kind of stuff, by default, it's not going to show up on IT's radar as, "Here's one of our agentic workflows. Here's one of our portfolio." You want to be able to, especially at an enterprise scale, you're going to want to be able to go and see-

Justin Mannhardt (21:22): What you have.

Rob Collie (21:23): ... your portfolio of all these things like this that are running, you're going to want to know if they're all still working.

Justin Mannhardt (21:29): Who's using them? For what?

Rob Collie (21:32): You're going to want to be sure that they're following some best security practices. If we spin up, and again, we're not this irresponsible, but we could, we could spin up an agent, give that agent behind the scenes the keys to the kingdom in a way that's irresponsible, it can see too much and isn't impersonating the user, so that whatever user, it's their credentials being used to make sure that they can only see what they should. You could write it in a bad way, you could mess that up. Are they drifting? Are they being used? Are they being used successfully?

(22:04): Even just a monitoring and awareness problem, which again, I think if you shift gears to a mid-market operation, a mid-sized company, you're not going to care so much. You're going to know that you have three or four really good, powerful, awesome things that are radically improving your capabilities, and three or four is not that many to keep track of. You know why Godzilla is actually impossible is the cube squared law? As something gets bigger, its weight increases with the cube of its dimension. Bones and things are only stronger by a power of two. And so, by the time you get an animal as big as Godzilla, it would need to be all bone to hold itself up. There wouldn't be any room for muscle or things that make fire or anything like that.

Justin Mannhardt (22:52): Nuclear radiation.

Rob Collie (22:53): Yeah. Otherwise, it would just collapse. There is a size problem at enterprises that mid-sized firms just really fundamentally don't have to confront. I can get all of that. Even the stories we've been hearing, they're not even necessarily about agent frameworks, they're about hoping that built-in functionality is going to handle it. So it's rounding to zero custom. I can understand the agent framework thing, you want responsible custom. This other thing, like, "Ph, please, please, please, let this be built in. Let Slack's built-in AI just address this need out of the box." I'm very skeptical of that, very, very, very skeptical.

Justin Mannhardt (23:32): We talked about this idea on an episode many, many months ago, in a lot of ways, the AI problem is very familiar to the BI problem. I don't need reporting in all my systems, I need analytics that brings it all together. I don't need AIs in all my systems, maybe for little use cases, that's fine, but I really do want something or a system of things that is cohesive in a very friendly and fun to use, easy to use, effective to use way. Off-the-shelf, it's hard.

Rob Collie (24:04): The chances of off-the-shelf success are even lower in AI than they were in BI, and in BI, they were zero.

Justin Mannhardt (24:11): Yeah.h Negative chances.

Rob Collie (24:14): I mean, it's just that in BI, you could lie to yourself that you were successful. "Well, we've got reports, we've got them."

(24:20): "And do the reports give the right numbers?"

(24:22): "Yeah, they give the right numbers, so check."

(24:25): But if the AI don't work, it don't work.

Justin Mannhardt (24:28): Yeah. What I'm seeing now is this pretty big difference between what I've been describing as productivity vanity points, which is sort of where I was in the early, going, "Oh, I can use ChatGPT to help me write a document, write an email, do whatever," to what's actually really capable when you can invest the time into custom, that gap is huge.

Rob Collie (24:56): Yeah, it's wild. Because you were such an aggressive and thorough adopter of that first phase in a way that for a while I was standing on the outside looking in on that going, "Justin appears to be becoming a new species, and I don't really quite understand it." And then, there was this whole additional world beyond that. The world that you were occupying before, that was legitimately impressive, is a molecule in the other world.

Justin Mannhardt (25:29): We could have stopped advancing the technology two years ago and it would have still been amazing, but we didn't.

Rob Collie (25:34): Yeah. No. Well, it's really just like our understanding of things. Whatever LLMs were available two years ago, there were still APIs to them. Like recent guests, Juan Garcia, Tuio-

Justin Mannhardt (25:47): They were doing it.

Rob Collie (25:47): ... they were achieving immense success. They had unlocked this, "Oh, it's a custom thing." And most of custom, in fact, all of custom is not LLM, it's regular Lego bricks, not magic Lego brick. They had to make some very, very, very complicated regular Lego brick structures to make it work back-end, and those structures would be and are much simpler today, but it's still the same basic game. Even two years ago, with your massive personal success, there was this other world that was already possible that we weren't really tracking yet.

Justin Mannhardt (26:21): Have you been introduced to the concept of the capability overhang? Have you read about this?

Rob Collie (26:27): No.

Justin Mannhardt (26:28): It's kind of like what we're talking about now, it's just interesting to mention, because I think there's a paper out about this. Basically, it's the simple idea that the capabilities of the technology significantly overhang the vast majority of ways people are using it.

Rob Collie (26:43): Oh yeah, Okay. This totally makes sense. It's by-

Justin Mannhardt (26:47): Many magnitudes.

Rob Collie (26:48): Yeah. It's ridiculous, this overhang.

Justin Mannhardt (26:51): I feel like the length of that overhang is increasing, because the pace at which these things are getting better is moving faster than people can adopt ways of using stuff.

Rob Collie (27:03): Let's test that for a moment, because in an episode not that long ago, we also talked about how even though the LLMs continue to advance, the advancement of those LLMs is really not really the thing... People are already not taking advantage of even close to 1% of what was available last week. So why should we be excited about this week's release? I still kind of have that vibe. The length of the overhang is growing, because the LLMs are still advancing and people have not caught up, I do agree with that, but I still don't perceive it as it's advancing by orders of magnitude, like yesterday, people were like 100x behind, and today they're 1000x behind. I think it's more like they were 100x behind, and now they're like 103x behind. What is your take on that?

Justin Mannhardt (27:51): I still maintain it's not as hard to catch up as we might think. If you're not totally current on everything-

Rob Collie (27:59): Right, yeah, I agree with that too.

Justin Mannhardt (28:00): That's still true, so I don't want to come off as scary.

Rob Collie (28:04): There's an unlock, but until you do the unlock, you're at the I'm getting 1% of the value out of AI that I could be. I think your number can go from 1% to 80% pretty quickly. But if you're not unlocked, if you haven't figured out this whole custom thing and what that really looks like and reach the point where you can be clear and confident about it, that's the issue.

Justin Mannhardt (28:28): Yeah. Some examples that, and they're very recent, little thing, it's a little thing, but I actually posted on LinkedIn last week this humorous take on how I'll catch Claude in a mistake and I'll point it out, and it'll be like, "Oh, I'm so sorry. I won't do that again." I'll be like, "Oh, you silly bot, you're going to do it again."

Rob Collie (28:46): "You absolutely will."

Justin Mannhardt (28:47): "Unless we update something about the thing you're following." So Opus 4.6 came out like yesterday, so now that's running in Claude code, and I'm going through one of my things and I said, "Hey, I didn't like how this happened." And it was like, "Oh, cool. Let me proactively go update my own instructions and I'll update my..." I was like, "Oh, you do that now."

Rob Collie (29:12): It updated Claude.md?

Justin Mannhardt (29:15): It updated its Claude.md File. I have a custom set of slash commands and scripts that do some of these things I do. It was just like, "Oh, I need to fix all this so I don't make this mistake again."

Rob Collie (29:25): Yeah, that's huge. I've had to tell it a lot of things, like, "Okay, now put this in Claude.md So that you remember in the future that you actually can do this or that you shouldn't do it this way."

Justin Mannhardt (29:39): So I posted that on LinkedIn this morning. I was like, "Well, that aged well." But then, another thing I was hearing from some people that do more code work, software development work, how they used to be in this state where they would work meticulously with Claude code, maybe in Sonnet 4 or something like that, to build a plan and review the plan very carefully and make sure it's right. Now, they're like, "I don't really do that anymore. It's more like the understanding of what our role is is changing a lot in how we need to work with these things." And so, I don't think that takes anything away from the custom need at all.

Rob Collie (30:20): No, I don't either. There's an answer to this, I just don't know it. I wonder... Let's take this example of Claude code now going, "Oh, I should update my instructions." Is that because of Opus 4.6, or is it because they updated the non-LLM parts of Claude code and added an additional tool that said-

Justin Mannhardt (30:41): Yeah, I don't know.

Rob Collie (30:41): ... "Spot recurring problem, write it to Claude.md"? I've built agents that do exactly that. I've literally just said, "Okay. Now, let's put another tool in the hands of the LLM coach so it can even spot that it has the possibility of doing this." And that suddenly levels things up, even though I have not switched which LLM is being used. So it could be both, right?

Justin Mannhardt (31:09): It could be both, yeah, I'm not sure. It's weird, it's exciting, it's scary, it's fun. It's all of the above. And so, when we're talking about custom AI and putting it in your business, I think if you're the leader, on the leadership team at a business, the sooner you can set aside a day to go deep or get some help going deep or reach out or whatever, for you to understand the delta, I think it's going to ultimately inspire you. I think it's not going to scare you about being behind or anything like that. I think it's going to inspire you to, "Oh, this is different."

Rob Collie (31:46): Man, I want to finish my book. That's the tone of it and that's this mission is to do exactly that for people.

Justin Mannhardt (31:52): What are you sitting here talking to me for? Go type.

Rob Collie (31:56): The content beast must be fed.

Justin Mannhardt (31:58): Feed the algo.

Rob Collie (31:59): Yeah, that's right. Well, plus, as we've said many times, we learn things in these conversations. This is one of the opportunities we have during the week to set aside to talk about things like this. We're not going to have a meeting just to sit around and jabber.

Justin Mannhardt (32:13): People hate those meetings.

Rob Collie (32:16): Well-

Justin Mannhardt (32:17): We don't.

Rob Collie (32:17): ... you and I wouldn't. As long as we get to control who's there. It depends on who's jabbering. This is really valuable. But yeah, you're right, I really need to be finishing that book, don't I? Someone we know, when he's writing a book, often goes to a writer's camp.

Justin Mannhardt (32:37): Okay.

Rob Collie (32:38): It's like a retreat.

Justin Mannhardt (32:39): Like a Jon Hamm Mad Men kind of-

Rob Collie (32:42): Yeah, yeah. So there's no phones, no Wi-Fi. It's just a place to go and write.

Justin Mannhardt (32:47): That's cool.

Rob Collie (32:48): You can't be reached. And it's a complete fiction, it doesn't exist. It's just his advanced out of office. He's literally sitting at the same desk he's always at-

Justin Mannhardt (32:59): But he just tells everybody-

Rob Collie (33:00): ... but his auto reply says, "Hey, I'm at Shady Acres Writers Retreat out in the middle of nowhere." And I bought this when I first saw that he was at this retreat. A few months later, he let me in on the secret. There is no such place. So beautiful.

Justin Mannhardt (33:23): Dang, I was thinking I'd ask to go next time.

Rob Collie (33:25): That's actually how I got him to admit that it wasn't real. I'm like, "Can I come? I'm working on a book." This is years ago. "This sounds amazing." You see him for a moment just going, "Should I tell him? Okay, I'll tell him." Well, with that, I guess I'm off to the writer's retreat.

Justin Mannhardt (33:52): Stretch those fingers, man. Good luck.

Rob Collie (33:53): All right, Justin. Have a good rest of your day.

Justin Mannhardt (33:55): All right.