Book PR, Fourth-and-One, the AI Knowledge Cliff, and LinkedIn WTF Moments

announcer (00:04): Welcome to Raw Data with Rob Collie, real talk about AI and data for business impact. And now CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie (00:20): Welcome back to day 35 of the Rob Is Sick podcast. After recording last week's episode with my daughter on Thursday, I spent Friday in the ER trying to get answers.

Justin Mannhardt (00:31): Oh, this thing is crushing you.

Rob Collie (00:33): It is awful. And I did not get answers. They ran all the tests. I did get the traction that was appropriate for my situation. I'm rooting for pneumonia and they're like, "Oh, good news, Mr. Collie, your chest x-ray came back clean." And I'm like, "Damn it."

Justin Mannhardt (00:51): Something well known, please.

Rob Collie (00:54): Blood tests, CT scans, x-rays, swabs, all kinds of stuff, "No, we don't know what's wrong with you." There are treatments that I'm on, but we'll see. That's one KPI that we've been keeping track of. We're a numbers kind of podcast around here. And the other one is pages written, I think I'm up around 220. A little bit of a stall since last week, but some of it has been going back and editing.

Justin Mannhardt (01:19): Refinement. Yeah.

Rob Collie (01:21): In terms of percentage complete, it's misrepresented. We're submitted to the book trade. We've hired a PR firm.

Justin Mannhardt (01:30): Cool.

Rob Collie (01:31): So a really interesting study in conformity and safety. Years ago, it was really clear to the analytics crowd in football that you should go four on fourth down. Fourth and one, you should go for it. But coaches, it turned out, weren't optimizing for maximum win percentage. They were optimizing for don't get fired. When you go for on fourth and one and you don't get it, that comes back on the coach. Those sorts of decisions come back to haunt you and increase the chances of you getting fired even though on net, that's the behavior that you want from your coach. It took a long time for that to break through. And now it's when they don't go forward on fourth and one, everyone just rags.

Justin Mannhardt (02:19): I mean, the analytics now is something like if you're past the defending team's 40 and it's like less than fourth and four or something, you always go for it.

Rob Collie (02:27): It's like so clear. I don't know. 10 years ago, even though the analytics were that clear, the behavior was 180 degrees opposite.

Justin Mannhardt (02:37): Right.

Rob Collie (02:38): So I've had a very similar, interesting experience with the orthodoxy. And you know how I am with orthodoxies. Orthodoxies are my favorite thing. You know?

(02:47): So I had this past experience of having written a book and written it in my weird ... It's weird for a book. It's weird for a business book. It's weird for a technical book.

Justin Mannhardt (02:54): It's unique.

Rob Collie (02:55): It's very friendly, verbal, familiar, informal. And having it do really well. In fact, having people say, "That's what we wanted. Oh my God, I can't believe, well, why doesn't everyone write like this?" I got away with it because I didn't have Microsoft Press editor straining all the personality out and turning it into humorless tech book voice.

(03:16): So this really interesting experience where I've talked to so many PR firms that specialize in books and authors. And I've had some really great conversations with some of these people. And I'm talking like exchanging emails with the guy who promotes Jim Collins' books. We're at the top of the profession, and we're interacting and things are going great. And then I send them the manuscript and they ghost me. Now-

Justin Mannhardt (03:51): For this book?

Rob Collie (03:52): Yeah. No, not all of them.

(03:54): And so, oh man, past me would have been devastated by this, that kind of rejection. But because I had this previous experience of writing a book in this informal, approachable voice that again is non-conformist, but just really better. No one really wants to read the humorless business book, the aloof sort of like, "I'm better than you," voice, that all of these books kind of occupy. No one really wants to read that, but that's just the safe thing.

(04:26): And because I've written a book like that that outcompeted the quote unquote "official books", even though we went through a independent publisher, I have the grounding now to know this is okay. But for them, it actually kind of does make sense, because for them to go to their network ... They have a successful business. These people are pipelines, they're factories. They don't have a problem. They don't need to take risks. Their number one job is not to mess it up. And so I get it.

(04:58): And so at one point I had to go back to my buddy editor, Eddie, and say, "Hey, Eddie, we need to go look for a different type of person. We need mavericks with gravitas." Anyway, so one thing we did mention is that I did find a publicist and we found this company on the not fucking around aisle.

Justin Mannhardt (05:15): All right. Oh yeah, I do remember that part. Now that you said that quote. Yeah. I don't know if this is a fact, but I was like, hmm, that might be the first F bomb on Raw Data in a long time.

Rob Collie (05:25): Tom needs to sneak them in as just a matter of practice.

Justin Mannhardt (05:29): It's like, hey, let's make the edits very difficult.

Rob Collie (05:35): I think it's to the point where it's kind of like the PG-13 rating. If you only have one in the movie, you can still give be PG-13, but you have two and it's on R. I don't think we're going to put the E on this episode, unless we do it again.

Justin Mannhardt (05:52): We'll have to see what happens next.

Rob Collie (05:54): Yeah. So how you been?

Justin Mannhardt (05:55): I've been great. Had a little trip this week, got some sunshine. So we recorded two weeks ago and I don't think I was technically officially done with my office remodel. I had like one or two things left to do. I am 100% done now. Oh, man, you'll appreciate this for the number of times we've struggled with this, whether it's recording or like in meetings. I have ethernet.

Rob Collie (06:18): Aw. Hot damn.

Justin Mannhardt (06:20): I got my big desk up here and I've got all my creature comforts. It's a good thing.

Rob Collie (06:25): Congratulations. That's got to feel great.

Justin Mannhardt (06:28): Spring break was awesome. Kids were off last week. We did a bunch of fun stuff. Did a Minneapolis foodie tour. So every day we went to a cool restaurant or spot that we never been to before with the kids, just trying to expose them to different types of food and stuff. And we had such a fun time and wore them out. The Monday, first day back to school, they were zonked.

Rob Collie (06:49): Really? That means you worked hard.

Justin Mannhardt (06:51): Yeah.

Rob Collie (06:52): Any particular food experiences that they were just blown away by or like, "Yuck, dad"?

Justin Mannhardt (06:58): Fortunately, no yucks. Their favorite place was, shout out to the Minneapolis crowd, called Nico's. It's a taco joint, and so they make really authentic style tacos. They got elote, and all the sides, and all kinds of different stuff, and some really creative tacos. The name of it escapes me, but I had a vegetarian taco that had cactus in it. There's a word for the type of cactus that you can eat, and it was phenomenal. So that place hands down would go back to. But we did a French burger place. We went out for ice cream. We went out for brunch one day. Went and got some ramen. It was awesome. It was a lot of fun.

Rob Collie (07:37): Just that conversation, I think I need to find my way back to that katsu burger place that we ate at that one time.

Justin Mannhardt (07:43): Oh, dude. That's-

Rob Collie (07:45): That was like a religious experience.

Justin Mannhardt (07:48): If you want to schedule 15 minutes to brainstorm reasons to go, just let me know. That place was fire.

Rob Collie (07:56): Yeah. All right. Well, so we chatted via text a little bit. And I wanted you to react to someone's gunning for your job, my daughter.

Justin Mannhardt (08:07): I know, man. I was thinking, wow, this could be a father-daughter thing. No, she was great. Seems like she's got a solid head on her shoulders, man.

(08:18): First thing, from listening to the episode, when you listen to someone that is in that stage of life that's only really been exposed to something for what, it was just a couple days, like 48 hours from when you hooked her up to when you sat down to record, things that I think people like you and I who have been in this wrestling with AI for a few years now, you either take for granted or you forget about. The first one's kind of funny, which is remembering that all the cool kids refer to ChatGPT as just chat.

Rob Collie (08:49): I've only encountered that twice recently. It's so weird.

Justin Mannhardt (08:53): It is like the F word. It's like a noun, a verb, an adverb, an adj, it's all the things. It's like, "Oh, did you talk to chat about that? Or let me chat that."

Rob Collie (09:00): This is the second time I've heard that. My friend Kevin did it and I didn't think of it as a trend until I heard Ella say it. And I did sort of subconsciously catalog it like, "Oh my God, is that what people are doing? They're calling it chat."

Justin Mannhardt (09:13): They're calling it chat.

(09:15): This is a banter tangent, but then I also had an experience recently that reminded me how annoying GPT5 is. I have to rant about this for 30 seconds. We were doing the OpenClaw thing. Anthropic just came out and said, "No, you can't use your subscriptions for things like this. You have to use the APIs." And I said, "Oh, maybe this is a good reason to jump over and just use some of the open AI models. Hey, let's just see what it's like."

(09:43): So I did that and initially I'm like, "Oh, this is kind of cool." And it's hardwired into their system instructions to end every message with some dopamine addictive continuation, "If you'd like, Rob, I could turn this into a 10 point plan." And so I call that out thinking about people like your daughter's age, and everything we've learned about social media, these things can just sit there and just be like, "Justin, if you'd like, we can go deeper, and deeper, and deeper, and deeper." And versus I work with Claude all of the time. And I'm like, "Hey, can you go do this thing?" It'll just be done.

Rob Collie (10:23): And you heard her say, "Claude is better."

Justin Mannhardt (10:27): Yeah.

Rob Collie (10:29): I decided that a while ago, but I don't go back and re-update. I'm not that guy that goes and constantly reevaluates, and re-benchmarks and everything. So it's pretty validating for me to hear like, "Oh, you're coming this way, and we still think it's better."

(10:43): Anthropic really, there's something about their culture that is really kind of aligned the right way. They just kind of get it in a way that the others don't seem to. It's hard to put your finger on it, but it's significant.

Justin Mannhardt (10:57): And I kind of convinced myself of something that's not true when I was going through this experience. You're like, "Oh, all the major frontier models, they're all really, really good." Sure, maybe from a benchmark evaluation capability, that's true. But you have to remember that what you sign up for in these subscription plans is not purely the model. You're signing up for the model, an application layers, a bunch of system instructions and route trees that inform how that product experience is for you. And so after a while I was just like, "I'm not doing this anymore. I'm biting the bullet and we're going to spend the tokens for these other workflows."

(11:35): I don't know that I can call it chat, but I was really happy to hear your daughter acknowledge that step change.

Rob Collie (11:42): She said, "It's only like in the last couple of weeks, dad." Like, I hadn't even heard of Claude until you started talking to me about it. And the last couple of weeks, it's like she's starting to hear about it a little bit on campus. Did you hear the reason why she's hearing about it, is that apparently stuff written by Claude is less likely to be flagged as written by AI. Because the things that are being trained to detect AI are probably disproportionately being trained on chat, on ChatGPT, right?

Justin Mannhardt (12:08): That's a good segue because that was one of the other things that stood out to me, mainly because I had a coincidental experience last week where I met someone that works in higher ed. They're dean of student services at a regional college. They were asking me about what I do in our business and that problem was the thing they immediately went to. Said, "Oh yeah, we're trying to figure out how to use AI to help with AI detection." And I was just like, "So hey, just my two cents, some of these things aren't very good at doing that."

(12:41): And this is maybe the rabbit hole that's interesting for us is I do worry about how many educators you hear talking about they're more worried about detecting if AI was used versus the quality of the end result, or the quality of the thinking, or the quality of the learning, or the quality of the skill development. You sort of see this even in LinkedIn a bit where people are just like, "Ah, if I see an em dash, you're out." There's this big distortion field going in there. And I think higher education, I think this is disruptive for them because students have these tools and they can use them maliciously or to help them, depending on the choices they make. But I think academia, in my experience, they just moved so slow, Rob. Their ability to adapt to this, I think is a real challenge from almost the bureaucracy of it all.

Rob Collie (13:32): Another orthodoxy. We've been doing it this way forever. We're tenured. Really, frankly, most academics have been shielded from the realities of life. It's not just a bubble, it's a Nerf bubble. I experienced something akin to this when I left Microsoft, and like, "Oh, wow, look at this. The harsh, real world. There's no safety net."

(13:59): There's a point in that podcast with Ella where she's really, really, really astutely picking up on something. She's like, "The off-the-shelf product doesn't know your class, doesn't know your professor, doesn't know your book," and all that kind of stuff. And I start laughing. I'm like, "Yeah, but with Cowork, you just drop the book in the folder." And again, if you do that two months before graduation, it's cheating. If you do it one day after graduation, it's productivity.

Justin Mannhardt (14:26): That disconnect is wild.

Rob Collie (14:28): Yeah. I mean, what are we preparing people for? This is just really, really, really smart.

Justin Mannhardt (14:34): So I thought that was coincidental because I had this interesting conversation with this person about having a different perspective on detecting for AI use, everybody's using AI. The future is using AI. That's where this is going. So this idea was connected to the way you set Ella up from the jump, which is something I found my way to on my own journey, which was Cowork is not going to think for you. It's going to help you execute and get things done. And it's going to help guide you in what you need to think about and put down.

(15:10): And you referenced the idea of human in the loop, which we've talked about a ton over the last few years. I'm starting to wonder, human in the loop might convey the wrong idea. I've been working on a ton of things where it's not necessarily that I'm in the loop as much as there's this clear division of responsibility.

(15:31): And you can still think about it as the loop, but people are thinking about workflows as, how do I get the whole thing turned on for AI? To use Ella's job search as an example, there were parts you felt were important for her to own, like her voice in a cover letter or her accomplishments being stated the way ... You could see a scenario where someone's just like, "I just want AI to be able to do all of that." And you see that on LinkedIn all the time. It's like, "Oh, I got this magic thing where AI does everything."

Rob Collie (16:01): They're basically saying, "You know, folks, I've really figured out how to really mail it in." And y'all need to just listen, like no, we don't.

Justin Mannhardt (16:13): There's an easy trap. I think your first encounter with AI or your first encounter with a specific use of it where you see it and go, "Oh my God, it wrote three chapters of my book. How impressive." But then you sit with it for a while and you realize like, "This isn't my voice."

Rob Collie (16:28): And it's not even all that great. Even Opus 4.6, as amazing as it is, there are times when Eddie, my editor, Opus 4.6, Cowork with all my instructions, is pushing back or saying, "I think you need to make this change," but I'm not getting it.

(16:44): And I say, "Okay, help me get it by drafting what you think I should say instead." And that helps me, "Okay, now I see what you're trying to say. Okay." But the first time you glance at that paragraph, it's like, "Oh, that's perfect." The temptation is, "Let's just use that paragraph." When I stop and I read it, I go, "Not only is this not my voice, it's not even really, truly the idea I want to convey."

Justin Mannhardt (17:14): Yeah. Well, it's like not even your opinion.

Rob Collie (17:16): I'm like, "Okay, I'm grateful for this clarity because I didn't understand the abstract version of what my editor was telling me I needed to fix. So now I see, okay, now I get it. There is something I need to fix, but I am not going to fix it this way." There is this moment that hangs in the balance. The devil is sitting on your shoulder going, "Yeah, it's good enough. Just use it." This is where I'm going to get a little bit elitist. I honestly can't imagine most people resisting that voice. I really do think most people are just going to go with it.

Justin Mannhardt (17:49): When I think of people that are at the stage of life like Ella, your daughter, and what I was saying about the annoyance of ChatGPT, there's something to be worried about with how the cheap dopamine hits of it feels like I'm being productive, it feels like I'm moving forward. Oh my gosh, I got all these ideas and I can keep moving. This didn't happen for me overnight. I'm not going to claim I realized this from the jump. But where I am at today, I'm spending a very considerable amount of my time thinking and writing by myself about, whether it's things we want to do in our business, or in my personal life, or just I want to write something into post or whatever, to have that volume of me coming out is actually getting me, I think, better results once I engage with AI because I know it's grounded in my opinion, like you're saying, the idea I actually want to try and convey and see if it resonates with people.

(18:56): And I think that would be maybe just an urgent message to people coming up in their life, don't let that go because it feels good in the moment to be like, "Wow, I studied so fast," but you really didn't. And you've had this experience where you can kind of one shot like an app or a vibe coded thing. You get 80% of this awesome idea and you're just like, "Oh my gosh, oh my God." And you realize there's all sorts of little nooks and crannies where it's just not quite what you really were going for. But there's still a ton of opportunity for, this is what I believe today, we can still be really valuable in the process despite what might be happening in the greater economic conditions of trying to replace jobs and all that.

(19:40): I can't remember who brought it up first in your conversation between you and Ella. I think she was talking about how she was expanding her job search and looking beyond her field of study. Where have people in this major ended up in different professional tracks? I just think the value of what I would consider durable skills, being able to come up with ideas, and have ingenuity, and be able to push ideas forward, and have an opinion, I just think those things are going to be really, really important moving forward in this whole thing. And I was really impressed that someone graduating college sort of had that intuitive read on things just 48 hours into using Cowork. It was really impressive.

Rob Collie (20:24): I'm her dad, and it surprised me too. It's not like I've been interacting with her on intellectual topics on an ongoing basis. I don't know. You could listen to that podcast and get the impression that she and I talk like that all the time. We do not. That is an exceptional conversation between she and I. That is not how it normally goes.

(20:46): I had two motivations with that idea of grabbing her. One was, well, I got a hole in the calendar. Justin's not available and so I need something to talk about. But the other one was really just like this time capsule. I just had this instinct that 48 hours in was a really good time to show other people because you and I are not 48 hours in.

Justin Mannhardt (21:08): That was a long time ago.

Rob Collie (21:09): I thought it would be like a really interesting time capsule/on-ramp for people.

Justin Mannhardt (21:15): I just think there's some deep stuff there with how people are coming into the workforce, and kind of with the way they think about themselves, so it was a great conversation. If you didn't listen to it, you should.

(21:26): And then also just sort of because you've been out in front of her a great distance, some of the early lessons about you can give these systems feedback and here's how you do that. You sort of forget every time Claude wakes up, it's like a fresh little AI that doesn't remember anything, and it needs access to tools and information. And that's sort of another way of the investments I've made into curating knowledge bases and processes. If I didn't have any of that, I'd have a different experience every single day with this thing.

Rob Collie (22:01): Not just different, also like incredibly diminished.

Justin Mannhardt (22:05): Yeah.

Rob Collie (22:06): I call it the knowledge cliff in the book. So there's three classes of knowledge. There's the stuff that's built into the pre-training of the LLM, the stuff that it knows without even having to search the web, which is, by the way, astounding how much it knows. It knows stuff that Wikipedia doesn't know without having to search the web. It's crazy.

(22:30): But then you don't even notice it really anymore when it has to search the web. There's this tiny little step down from what's in the pre-training to what's on the web. You don't care. It takes a little longer to come back. You don't even really pay attention to it. You don't even really notice necessarily when it's searching the web versus when it's not.

(22:48): But then there's that third step, which is everything else not available on the web, and that's a lot. You take this little first little step down from pre-training to the web, and then the next step down, you just fall off a cliff and splat.

(23:03): So if you want to really hypersimplify how to make AI useful, you've got to fill that gap. And so everything is in some sense like struggling to fill that gap. All of these things, every one of these applications is like, "How do I get closer and closer to your information?" Try to make it out of the box, try to make it off the shelf. Cowork is pretty amazing at it.

(23:28): I got attacked on LinkedIn for ... It was just so confusing to me. I posted a screenshot of someone that I know that I told him to use Cowork and he's like Slacking me from Eastern Time on a Friday night saying, "Oh my God, oh my God, oh my God."

Justin Mannhardt (23:56): In that initial just seeing fire moment.

Rob Collie (23:59): And it's 8:00 PM. I'm at Target. I'm buying deodorant and this guy is slacking me from the Eastern Time Zone. And he's an early to bed guy. He's usually in bed by 11:00 PM. But he's like, "You know what?" And at one point he said, "I have to stop because I'm not going to be able to sleep." And this authentically happened. I took a screenshot of it. I redacted his name and identity. I put it on LinkedIn and said, "Some middle-aged CFO type trying Cowork for the first time."

(24:28): One of my former acolytes who was really in on the last time I was talking about a revolution, the Power BI revolution that benefited him, he was all in on that. I mean, he came for me hard, like, "Not you too, Rob." I don't even really know what he was accusing me of. How does me talking about someone else's experience with Claude Cowork even benefit me?

(24:55): One of the problems here is that people tried out chat and said, "Oh, that's AI." And so they just got stuck there. So I think what's going to happen next is people will try out Cowork and go, "Oh, okay, that's AI. That's all there is to it. That is it. That's all I need to know." And so in a way, Cowork is going to be a competitor for our company's services because it's going to become the status quo that people are comfortable with. By promoting this thing, I'm kind of like working against my own interests. Dude, what's my angle? There's a real resistance. There's a divide developing.

(25:38): And then at the same time, someone chimes in on a different post and they're like from a financial planning and an FP&A background. So you'd think they were one of our Power BI fans. He says, "I really like your podcast." And I said, "Okay, when did you start listening?" And basically he started listening during our AI phase. He wasn't a carryover. So it was like we're losing people from the Power BI world that are talking to me. Again, I don't know how many people are like this, but this one guy was really, really vocal. I've even responded to one of his criticisms before on the podcast, it turns out. And he's like feeling betrayed, like, "Et tu, Rob?" while at the same time, we're picking up other people. And I told this guy, "I get it. You're afraid for your job." "I'm not afraid for my job." And I wanted to come back with the Yoda, "You will be." You know, right, if you sit still, man.

Justin Mannhardt (26:33): I think you can feel it in some feeds. And I think LinkedIn at times is a place where everybody's just like, "Why is everybody trying to sell me stuff?" And they just assume that's what everybody's doing.

(26:43): And AI, yeah, it is scary. It's exciting. It's both. And listen, there's a lot of very powerful, loud voices that are broadcasting unrealistic visions of both hype and doom that aren't helpful.

Rob Collie (27:01): But those are what the algorithm wants. I talk about this in the book. These are the messages that float to the top. The ones that tell you you're doomed, you click on. The ones that tell you AI is a sham, you click on. Because this is the thing you're wrestling with. The reasoned middle takes, the ones that are actually helpful, they don't draw the clicks at the same rate. They don't draw the engagement.

(27:26): They finally exposed Facebook, like if someone clicked the angry button, it was worth five times as much as a like in terms of the algorithm. Folks, we really did that? We put a 5X premium on anger. Why would we do that to our society? The same sort of thing is happening on LinkedIn. It's not quite as extreme as the 5X anger thing, but it's still, these are just two sides of the fear coin.

Justin Mannhardt (27:54): Yeah, 100%.

Rob Collie (27:54): But yeah, so next week we'll talk a little more about Cowork. I'm sure we'll have other things to talk about. All right, man. Well, until next week.