AI Agents for the Manufacturing Industry, w/ Interpak CFO Bill Krolicki

Announcer (00:00:04): Welcome to Raw Data with Rob Collie. Real talk about AI and data for business impact. And now CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie (00:00:20): Hello, friends. We've been talking a lot about AI lately in general terms, things like why it's more of a data problem than we all expected, what its strengths and weaknesses are and how much better it performs when you get narrow and specific with the task you've assigned it. Those are very valuable and important things to discuss and understanding those sorts of things helps us all build a clearer picture of what AI can do for us. Which helps all of us start to identify places in ways in which it can be used to improve our businesses. That general lens, what can it do and how does it work is both valuable and necessary, particularly for an AI and data consulting firm like ours. We need to understand it in the general sense. Our client's needs and workflows for AI span the entire gamut of what's possible. We need to be ready for anything.

(00:01:08): But there's a second lens we can use in parallel, which is to just start looking at lots of examples of where AI has already been successfully deployed. Going through a collection of success stories is also a good way to build those how can we use this stuff muscles. Now I don't think the second specific examples lens is any more or less useful than the first lens. Certainly here at P3 we need both. But ultimately I think the second lens is probably the way that this stuff reaches the mainstream better. Humans learn by example. Humans learn by pattern matching. It's just that we're still really early in the AI revolution. A year from now, we're probably going to be inundated with success stories, but right now the industry is very much in that priming the pump phase. So it's time to do our part with the examples thing. And here at the Raw Data Podcast, we're going to start increasingly sharing success stories, both as we create them with clients and as we discover them in the wild.

(00:02:01): Today's episode delivers on that new commitment in a big way. Our guest comes to us via a private conversation that started on LinkedIn, which then quickly morphed into, "Oh my God, we've got to ask him to be on the show." And fortunately for all of us, Bill Krolicki, CFO of packaging manufacturer, Inter Pack, enthusiastically chose to accept that mission. He's got a textbook example of AI success to share with us. And when I say AI success, remember I will never mean off the shelf usage of ChatGPT kind of success. True AI success means the customized, tailored specifically to a business workflow kind of success. And VendorBot, the supply chain assistant agent that they've built at Inter Pack, 100% checks that checkbox.

(00:02:46): I love VendorBot because it's super, super easy to understand. It doesn't matter what industry you're in, this one will make all kinds of sense to you. It's at least a coin flip in fact that you're going to see some immediate parallel use case in your own business even if you're not in manufacturing. And if you are in manufacturing, chances are really high that you're going to go, oh yeah, we need VendorBot. Another thing I really liked about this conversation was Bill's humility. Here's someone who really is ahead of the overwhelming majority of his peers and who is positively devouring AI-related content and education on a daily basis. But he very much thinks of himself and speaks of himself as just like everyone else. Well, folks, he's not like everyone else. He has a clarity of picture here that's like in the 99th percentile. And he has a full-time resource at his company to help him build stuff.

(00:03:35): So please keep that in mind while you listen. You shouldn't put the pressure on yourself to expect that you should just be able to run out and replicate his success like tomorrow. Please don't let this conversation make you feel like you're behind. Bill is just ahead, way ahead. And to be clear, helping people be more like Bill is rapidly becoming our whole reason to be in business. We're stoked about helping people achieve these sorts of things. In fact, at the end of this conversation, we talked about another scenario, the budget bot or budget-nator, which is even a little bit out of Bill's reach at the moment.

(00:04:08): And that's led to some offline conversations about how we might help Bill make that one a reality, even Bill needs help. Your first job is just to start getting better at spotting use cases. And if you think you have spotted a use case or use cases and you want to talk about them and see if you're on the right track or if they're feasible, please feel free to hit me or Justin up on LinkedIn. No pressure, no strings attached. It's our job to talk about this stuff, which after all is exactly how this particular episode came to happen. With all that said, let's get into it. Welcome to the show, Bill Krolicki, how are you today, sir?

Bill Krolicki (00:04:44): I'm doing great. How are you guys doing?

Rob Collie (00:04:46): That's fantastic. Before we dive in, why don't you tell the audience who you are, your job title, where you work, and then how you came to be where you're at?

Bill Krolicki (00:04:53): My name's Bill Krolicki, I'm the CFO for International Packaging, which makes boxes for say like Home Shopping Network or traditional engagement ring jewelry box like clamshell, open up, the metal ones or cardboard ones and that type of thing. So we make a lot of those boxes right here in Rhode Island. And in terms of how I got into data stuff, I've just been a data guy since I was a kid. And remember, it was like sixth grade, I think they had. I think it was these old deck things and doing the basic print hello, go to 10, print a low 20, go to 10 and watch the paper all spin out as it prints the thing out. So I just remember that. I was like, "Wow, this is neat." Always had, as you said, the data gene. And then I got into Power BI five, six years ago and started with your book. That was the thing. Went to Amazon, they said, "You read them as for Data Monkey book to understand Power Query and then your book for the DAX. And I got them and it was like, wow, this is awesome.

Rob Collie (00:05:57): You got the Holy Macro Books bundle. Bill Jelen's Publishing company, Mr. Excel's publishing company named Holy Macro Books. Just one of the best naming twists of all time. Love you, Bill. Both Bills. Yeah, well, I appreciate that. The majority of people who were in Power BI today don't know that I wrote a book.

Bill Krolicki (00:06:14): It was the thing. Anyway, I'm somebody who can learn from a book, so I got it and it was great. It laid out the whole framework and how to star schema and all that type of stuff. And then when you go see other people's stuff and it'd be like, what the hell are you guys thinking? This is a mess.

Rob Collie (00:06:30): I really, in hindsight, I'm very, very, very proud of that book and how humane we managed to make. Which was another advantage of working with Bill Jelen as the publisher, is that I wasn't forced to voice the book.

Bill Krolicki (00:06:42): Right, turning it into something stale.

Rob Collie (00:06:43): All the personality, all of the softness to it, all the warmth would've been squeezed out by MS Press for instance. So you never really know even what the author's intent was versus what the publisher enforced when you're eventually consuming a tech book. But we're here to talk about AI primarily, but let's do the Power BI bedrock first. When you say you got into Power BI five or six years ago, were you built the same company as you are today?

Bill Krolicki (00:07:09): Yeah.

Rob Collie (00:07:09): So give us a picture of the before and after. What do things look like before Power BI and after Power BI and what kind of difference that made in your business? Because BI and mastery over your data and awareness and all that, invisibility, all of that is still very, very, very much a relevant going concern these days even though AI's getting so much attention for good reason. Let's not skip over that foundation because I think it also sets the stage really nicely for AI going forward.

Bill Krolicki (00:07:36): I think the Power BI foundation is still actually really useful for AI going forward. I mean, we talked about the stuff that we've done lately. But when I first showed up at Inter Pack, I showed up as a CFO and they had just moved on to a new ERP system, it's called Global Shop. The system is very good for manufacturers, but it's highly customized and it can be a little bit difficult to work with. And so it offers the potential to answer all of your questions and you'll be able to track every single little thing. And then suddenly everyone discovers like, oh my God, that's really hard to do, or that's a lot of work, or that's not how we did it in the past. And so then you make all these compromises so you can just actually get the products out the door, but then you lose the whole visibility into the system and then everyone's unhappy for a long time. And so I'd shown up in the middle of the, "I thought this was going to make everything better, and it's not."

Justin Mannhardt (00:08:34): This is how every ERP implementation known to man has gone, by the way, just this perfect elevator summary, Bill.

Rob Collie (00:08:42): Gartner needs its own curve for this, right? The trough of despair.

Bill Krolicki (00:08:45): Exactly. The software is very good for manufacturing, but the financial reporting aspect of it was pretty weak. And so it was a lot of export to Excel and then you had to clean it all up. And then we were consolidating multiple companies and it was just really tough to deal with. Plus you had very, very poor drill down capability to explain, well, why did travel expenses go up this year versus last year? So I started off trying to, okay, if I can pull the data, I can get the data out of this system and then pull it up into Excel, I can do stuff with it, that'll be great.

(00:09:26): But then there was too much data and it was tried to do a whole bunch of V lookups or whatever. And that was like, well, now it's crawling, so that's not working. And then I stumbled onto Power Query and then I was like, "Wow, holy. This solves all of my problems." And I was dreaming of this probably 20 years ago, something that could have this kind of power to it. Downloading these files and pulling them up and you got to do all that ETL stuff and it's like, wow, I just do it once and then it's all gone. Oh my God, it was just so simple. So I loved that.

Rob Collie (00:10:00): Can I inject a quick aside here?

Bill Krolicki (00:10:01): Yeah.

Rob Collie (00:10:01): You said 20 years ago you envisioned this, right?

Bill Krolicki (00:10:03): Yeah.

Rob Collie (00:10:04): 2001, 2002, I was obsessed with this feature idea for Excel that I called Data Merge. That really is Power Query. I just didn't know that Power Query was what the solution would eventually look like. So I had this really ambitious idea for all this end user transformation, repeatable transformation type of stuff. And I was constantly pitching it and I was constantly failing to get traction. Leadership wouldn't listen, say, nah, whatever. And I was very frustrated by it that no one would ever buy any, I don't think they even really saw the value of it, which was the most frustrating part.

Bill Krolicki (00:10:41): Other people were probably like the alternative was do it in VBA. And so I had learned VBA in the '90s a little bit, and then it gotten away from it and whatever. And they came back and was like, "Oh, I got to do VBA with this." I wasn't good at it and it was just struggling.

Rob Collie (00:10:55): Gross, yeah. Yeah.

Bill Krolicki (00:10:58): And then as I said. So at this point where I'm trying to wrestle with VBA and then suddenly I stumble across Power Query and Ken's book and it was just like, oh my God, this is what I was dreaming of and it's so much easier than VBA.

Rob Collie (00:11:11): So much easier, yeah.

Bill Krolicki (00:11:11): It's like, wow.

Rob Collie (00:11:13): And I had the same, oh my God, reaction to Power Query when I saw it. And my very next reaction was, oh, I am so glad that I never got people to buy in on my data merge idea. Because having seen what a comprehensive solution to that problem looks like, it was so much deeper of a problem than what I was giving it credit for at the time. I recognized the need and it was frustrating that no one else recognized the need, but they were kind of right to say no to me, but for the wrong reasons. They had to go and invent a language first, a whole programming language had to be invented before they started with the graphical elements. I'm like, oh yeah, I wouldn't have done that. So Power Query was first, then you get into DAX and data modeling after that?

Bill Krolicki (00:11:58): But it was funny because I was just doing the merges and the joins and all that. I read that book and I read your book. But then it was like, goddammit, I can solve everything with Power Query. In accounting when you're doing all these reconciliations and you're matching things up, it was doing a great job of that. But then at some point we were working on a reconciliation project. It would take 15, 20 minutes for Power Query to crunch through the thing. And then I realized, you want to change the dates or something like that. It's like, I got to wait 20 minutes. I think this is where DAX comes in. And so then started really making the push into Power BI.

(00:12:40): So then it grew out of running the accounting department, but then I could say, oh, well, the sales reporting was really horrendous. And so it's like, oh, this very easy to just click on the buttons and you can see by customer, by rep, by whatever you want to see. And it was like slice and dice and visuals and all that and put that out there. And so then started to get into the sales group. But it was funny that the owner, he's one of these people talk about Data gene, but before PCs, he's one of these people who can just consume these massive spreadsheets of just raw data, just numbers.

Justin Mannhardt (00:13:15): Seize the matrix.

Bill Krolicki (00:13:16): Right. That's the way he wants to see it. So it's like you give him a chart, he's like, "What's that for?" But he looks at the whole thing and so he has that magic power where he can look at it and go like, "Well, why is that cell out of line with the other..." But he's the only one who's like that, normal people.

Rob Collie (00:13:37): Yeah, that's a total outlier. Look closely at the back of his head, is there a CPU plugged in back there somewhere?

Justin Mannhardt (00:13:43): Right.

Bill Krolicki (00:13:43): So when I first came up with it for the sales, he was like, "I don't need all these charts and this colors and conditional for, he was like, ah. But then what happens to the head of sales then starts getting it and kept going to him and saying, oh, this segment of the market's going up a little bit. And he's like, "How you known all this?" He's like, "Oh, that sales report that Bill wrote in Power BI, I can see all these things." And so then he starts to get into it and then later on if the thing ever breaks, then I'm getting an email, "Hey, how come the sales report isn't working?" He's paying attention but in the beginning he was like... As I said, it's just because he has the natural talent to just absorb massive amounts of data.

(00:14:23): The accounting department made things so much better and then I had the ability to drill down. I worked with Imca Feldman, I know you did a podcast with her. So she's the one who helped us build our financial model because the financial models are a little bit more complex than a straight-up simple star schema type of thing. So she put that together for us and then I was like, "Wow, looking to drill down, I can do all this." I was going wild. And was working with Matt Allington, his course, following the Italians, all the guys were great. And so just kept on learning more.

(00:14:53): And we had a guy here, he was the SQL guy. It was so difficult to get him to learn Power Query because he was like, "I know SQL. I don't need this Mickey Mouse little wee interface to do it." It's like no interest. Somewhere along the line, I don't know, I convinced him to take a look at it. And then after a little while he was like, "Oh, wait a second. This is really, really easy. This is easier to do than some of these things I was doing was SQL." That was a big win because then once he got on board, mechanical engineer, but once again he has the data gene. So once he got on track, he's like [inaudible 00:15:30]. Now recently he had to do some stuff in Excel and used to live in Excel and he says, "Oh my God, I just can't work in Excel anymore." His massive transformations.

Rob Collie (00:15:41): And Excel is really making a comeback in a lot of ways. They're reinventing themselves. And I haven't really been keeping up, they're just doing so much over there. But that would be a whole podcast of its own. We probably should bring someone from the Excel team on to talk about that. One curiosity I have, so you're CFO, is there also a CIO at your company?

Bill Krolicki (00:16:01): No.

Rob Collie (00:16:02): The usual dynamic, right? The CFO is already up to their eyeballs in data by necessity, and that also means up to their eyeballs in Excel and other tech. And so the CIO type of job, whether you like it or not, it just kind of lands on your desk.

Bill Krolicki (00:16:20): Yeah, we're a small, medium-sized company. That's what it is for a lot of these size companies. The CFO oversees the ERP system, that's where all your data is and then a lot of the networking stuff, sometimes you have outsource people to assist, but you get stuck with all the data management and then tend to have some of the skills that are good for doing the analysis and goes from there.

Rob Collie (00:16:43): So two episodes ago I did a solo podcast titled: You are the AI Cavalry. Where you is the data people, the data gene people. And I think you are a prime example already of that emerging archetype. That there isn't some other AI cavalry coming and even if they did show up and claim to be that cavalry, which we'll find is that they're not nearly as effective as someone coming from the data side, the business side, understands things already like how the business works, growing into AI, this long journey that I've been on of discovery almost like come full circle. And I'm like, "Oh, AI is just a data problem." We had just some conversations again a couple of weeks ago, I think, about some of the things that you all are up to. We were just very impressed with your level of understanding and also sort of how far along you are.

(00:17:39): You wouldn't necessarily know that. I think everyone sort of assumes by default, most people assume that they're kind of lagging behind in whatever it is, whether AI or whatever. But we get to see a lot of the world, your denominator is one, you're seeing one out of one. But we see a lot, we're sincerely very impressed with your level of competence and understanding already. I think you're very much a leader in your space. You wouldn't necessarily know that unless you compare notes with lots of other people, but we can see it.

(00:18:09): All right. So let's just start with one of the workflows that you told us about that I just think is just such an awesome example of the practical application of AI. It's not super, super complicated, but the impact is immense. And it's the one about your production schedule. When you go to manufacture things, you have a schedule for manufacturing them, but if all the raw materials and ingredients, et cetera for manufacturing, that packaging hasn't shown up yet. You can't manufacture the things that you plan to manufacture on that day, and that's very, very disruptive to your business. Age-old problem, so many things have to come together. Any one thing isn't there and the plan is ruined. 95% is the same as zero.

Bill Krolicki (00:18:55): Especially for our stuff. It's like you don't have all the components, you can't make it, you can't go forward.

Rob Collie (00:19:00): We have everything but the glue.

Bill Krolicki (00:19:02): Right. You have no glue, you're not going to make the box. Like you said, the problem is that people are like, ooh, AI, and it's like, oh, it's just going to take my job and it knows everything and it's so smart, but the reality is it's not that smart. So once again, I say think about it like it's an intern, so you get somebody from college who doesn't know anything, but they're smart enough so they can learn and then they hopefully you show them this is the process and they can stick to it. And that's what it was.

(00:19:29): And so for us, as you said, the basic problem was we have these vendors, they're delivering stuff, we send them a purchase order, so they have the PO, they tell us, "Oh, it's going to show up on the 15th." And so we schedule our production saying, okay, the materials will be here and then we can start working on it. But the issue was that sometimes they were late and a lot of times they wouldn't tell us until the 15th. And so it's like, "Oh, we got everything scheduled and we got all these people here on the machine and now we got to run around. Well, we can't work on that. What do we work on instead? And which ones do we have all the materials on?" And granted, Power BI is what we use to determine all that. But the issue was, okay, this is a pain in the butt.

(00:20:12): And the purchasing people are supposed to call them up each week to say, oh, this is what's supposed to happen next. You're supposed to deliver this stuff next week. Is it all showing up? And follow up. But that's a lot of work and nobody wants to do all that. And so they kind of do it. A lot of times the answer's like, "Yeah, it's all on schedule, what do you think? If it's a problem, I'll call you." Of course they don't, but you're doing it and they hearing that. So then it was like, okay, let's put AI to work here because the AI can go and it can look at the open purchase order table. And it sees, okay, these are the purchase orders that are due next week and I see who the vendors are and it looks up and it knows who the email address of the person. So then it sends them an email with the table saying, here's the items that we're expecting you to deliver next week. If it's going to be late, let us know. And so it does that. Now actually, the thing is with the AI, you don't really need the AI for that, you use Power Automate and Power BI to do that.

Rob Collie (00:21:07): The automated sending of an email that's data driven, it's saying these are the things we're expecting from you next week or whatever. You could use AI for that, but it's overkill and probably not as reliable in some ways.

Bill Krolicki (00:21:19): But that's the key thing what you're saying there because we pried initially to like, oh, it's just there. We give it access to the open PO table. And AI, you're so smart, go to work and figure it all out. And it would get all confused and it was a train wreck.

Rob Collie (00:21:34): Which harkens back to last week's episode where we're talking about overwhelming these things. You got to give them the benefit of focus.

Bill Krolicki (00:21:41): Because that's what we did, is we basically said, "Okay, this is overwhelming the thing. Well, let's just build in Power Automate. And it goes out and Power Automate can go do that query of the Power BI database and come back and it comes up with the table of here it is." So we just handed off to the AI. Here's the table that you need. And so then it's like, okay, don't have to ask it to figure that out. Power Automate hands it over to it. Then we send out the emails. This is where the AI comes in. If the people respond, an email comes back. And so the AI, our bot is looking at the emails coming back. And so if somebody says, oh, yeah, it's not going to happen, it's going to be three days after what you're expecting. It can look at that, which is obviously vague, but you know what it's saying.

(00:22:27): But the AI will also understand, oh, he's talking about it's going to be three days late, it's due on the 15th, now it's going to be the 18th. And it will recognize that that is going to be late. And so the purchase manager is also going to see that. But this one, the AI seeing it and recognizes it. And then the thing is that once again, to get the purchasing manager to say, an expert, and go like, oh, this is going to be late. Let me pull up this information. I need to know is this scheduled? Who's the customer for this finished good product? Will that be late? What's going to happen? They have to look up a whole bunch of information. But now what we can do is the bot now knows we have a lateness problem. And the bot will go pull up and go into the ERP system and can once again pull up all the bits of information that we need, just like it was figuring out who's supposed to deliver the next two weeks.

Rob Collie (00:23:19): I love this.

Bill Krolicki (00:23:19): So it pulls all this information together that's key information for the purchasing manager and gives it to her so she doesn't have to look it up. This is all the information that she would've had to look up by herself anyway, but the bot already did it for it. And with Global Shop sometimes it's like you got to go into three or four different screens and maybe you do an Excel export or you got to write it all down or things like that. So it's here it goes, it goes and it does all of that consolidation, gives you all the information that she needs to answer the question of what do we do about this.

(00:23:51): And so the next step that we're putting in is we'll get a button with do you want me to do something about this because she's in the loop, notified of the problem. She can look at the email, say, oh, yeah. Because there might be a situation where she goes, no way, I'm going to call that guy up and I'm going to chew him out and I'm going to get him delivered on time. And so no need to worry. But other times she's like, okay, yeah, I know, we have no power over these guys whatsoever. So if they say it's going to be late, it's going to be late. And then it says, well, do you want me to update the PO due date in the system? Oh, that's good. So click the button and then the bot goes, once again, it launches a Power Automate thing, which will go into the system and go into the different screens that you need to do and click save and update and all that type of stuff to get it updated.

(00:24:35): So once again, the purchasing manager doesn't need to do that. We needed her brains to assess what's going to happen, but we don't need her to do the manual grunt work of wrestling with the ERP system to figure out how to update, that's doing that clerical work. So that's a waste of time. So you get that and then you do other things like, oh, well, do you want me to send an email to sales letting them know it's going to be late and why? It's like, yeah, sure. Boom, boom. So it's really exciting how it can pick up. And then there's one other aspect that's subtle, but I believe it's in there, which is sometimes, as I said, you have to do this analysis. Now you know 90% of the time the answer is A. 10% of the time it's B. I got to do a lot of work to say, oh, it's not A, this is one of those situations where it's B, but it's a lot of work. So the tendency is to kind of go, oh, we've got that problem.

Justin Mannhardt (00:25:31): It's probably A.

Bill Krolicki (00:25:32): I started the research process. It looks like an A. It's just an A, let's just assume it's an A. And it turns out it was actually a B, but you didn't go through all the hard work to confirm that. And it comes back and it bites you and everybody is all pissed off and it just turns into a total nightmare. So that's the type of thing. It's like, oh, get the bot, the bot doesn't care. It's like, great, it'll grind away, it'll do all that analysis, pull all this, go into the 10 different screens it needs and combine them all and put it in because you've coded that type of analysis that it has to do. So once again, that's either a Power BI query that it's launching or it's a Power Automate thing that is doing, but the AI knows, oh, I got to do that and sends off that aspect of the problem.

Justin Mannhardt (00:26:14): I love this so much. Well, first of all, Bill, you're describing the realities of basically every manufacturing organization I've ever encountered or worked at myself. Supply chain challenges, materials showing up on time, we've got temporary staff on the floor. Now what do they do? You've got the gal in purchasing that she can go just bend somebody in submission over at the vendor when you need it. So I love these stories.

(00:26:42): But the problem is very specific, I think that's really a key for leaders out there. If you're still in a vague space of like, ah, we'll just put AI in this part of our business, you got to get specific. And specific maybe loses some of the sex appeal of AI, it's helping us monitor late shipments from our vendors. So getting that specific, but then also the specificity of where the AI is coming into play. Rob and I, we talked last week, that you really need to think about what are the things you actually want AI to actually think about. Like in your example, you don't want it to think about how to go find the POs, that's a routine process, hand that off over to some other piece. You don't want it to have to think about how to find the salesperson or those things. Give it all that tool links because you want it to spend all its brainpower thinking about, I got this email, is it a problem?

(00:27:42): There's a lot of subjectivity that AI's really good at there. Or yeah, I do want you to write the email to sales like, okay, how should I write that in a way that's going to convey all the right information and the right tone? And so I just think those things are so important that we don't just assume AI is the right tool for all these other parts and we're being specific as we can about where we sit it in the process so that it shines and does what it does the best. This example is so good at articulating that, I just think it's so good.

Bill Krolicki (00:28:15): But yeah, no, I totally agree with you, which is that workflow focus is where you can get some real concrete value. In this example I said you're taking away a bunch of the clerical type of work, whether it's AI or some of the automation tools. They can do that, but you're keeping the person in there and so you don't waste all their brain energy wrestling with the ERP system or trying to pull the thing. It's like they get the data that they need and then they can make decisions and that's where they start to get better and better. If they're spending all their time making decisions and figure out this is how to handle it, this is how to handle it, they can get better and let the bots handle all the other stuff.

(00:28:55): And on the flip side, just like you say, I think a lot of people, I think sometimes on the programming side there's more of this. Where they'll be like, "Oh, great, we're going to dump all the information into the AI and let it answer all of our questions like it's the Oracle." To some degree with our PO. It actually does have kind of that capability with the chatbot. You can say, hey, which POs are due next week and which one are import orders? And don't count that one. It'll go through that. But what I've found is at least right now nobody uses it that way. Who does that? It's the purchasing agent. I already know how to get the information out of the ERP system. I don't spend this time talking to the bot hoping that gets it right or making it's... Especially, I think this is a generational thing. 20 years from now, everyone will be like, what the hell you mean? You trying to get stuff out of the ERP? You just talk to it and it tells the answers. But our average age year, something like 55, so the people are not used to talking to their screen and expecting to get a good answer out of it. Our own experience talking to Siri tells you it doesn't work with you well.

Rob Collie (00:30:05): On the flip side though, when this headless agent that's basically determining whether or not production is going to be on schedule, whether you're going to have all this stuff, have you met much resistance to adoption of it? To me, it would seem like it's just taking so much really unfun work out of people's lives that it almost wouldn't matter what an individual's personal technology adoption curve looks like. It's just like, oh, there's so much better. I would expect this to be relatively low friction adoption. Has that been your experience?

Bill Krolicki (00:30:36): When you're on this focused workflow approach, the friction is really minimal because the bot's suddenly handling it. It's like, oh, great, I don't have to do that anymore? Oh, great, I don't have to do that anymore? Oh, great, I don't have to do that anymore? Oh, yeah. But it's still checking in with the person, hey, this is what I see. What do you want to do next? And so they're like, they're still in control, but all the grunt work they don't have to do anymore. So they're okay with that. But as I said, when it's just like I've uploaded all our purchasing information into the bot and then expect the purchasing manager to ask it questions and stuff, she just doesn't think that way.

(00:31:10): The only place where it might have value, but we haven't gotten there yet is say someone in the sales department, they know that the order that they want to go out is dependent on some material coming in from the vendor. So they know my thing can't go out unless that material comes in on time, but they don't know how to get the answer out and purchase it. They just have to call purchasing, "Is this stuff on time? Is this stuff on time or when's it showing up? Or blah, blah, blah, blah." So that's irritating for the purchasing person. Whereas now if you had the purchasing bot, that salesperson would just ask, the material that I need from such and such a vendor, is it showing up on time? Is it going to be late? It knows how to go into the ERP system, pull it out just like the purchasing manager does. But the salesperson doesn't know how that works, doesn't want to learn how to do it, it just wants to ask the bot. And the bot is always happy to give an answer, whether it's right or not is a different thing. But the purchasing manager may be less excited about constantly answering the salesperson to tell if there's a problem I will tell you, stop calling me, I have work to do.

Rob Collie (00:32:13): Getting people to change their workflow is very difficult unless it's just overwhelmingly obviously better. When we designed software and we had sort of a general principle against surprises, you don't surprise the user in general of software. You make sure you don't do that, it's not good. Unless you're overwhelmingly certain that it is a positive surprise and an overwhelming percentage of circumstances. We'd make exceptions for really positive surprises with a very, very high probability of it being positive. But that was the bar, otherwise, no surprises. It's the same thing. One question I had listening to all of this was you've had this problem, it's just an age-old manufacturing problem, is this stuff going to be here on time? You've had it for a long time. Had you made any attempts to automate this process before AI came on the scene?

Bill Krolicki (00:33:06): We didn't automate this process. There's a similar type of situation on the accounts receivable side, which we did do, which had to do with it's been 60 days, you haven't paid your bill, and so it would send out an email. So we did something very similar, to some degree that was the beginning part, which was, okay, we want to automate the process of sending out the messages to people. But on the other hand, some customers, we don't want to get a chasing letter because they're like our top customer and we know what's going on and we have an agreement with them or whatever. So instead, what we have is each week the bot goes and says, okay, here's all the customers who are over 45 days past due. So here's the customers and then it asks us, head of sales, anybody on this list, you don't want to get an email.

(00:33:56): And then he'll say, "Oh yeah, don't send it to that guy." And that guy types in their names and then says, "Okay, send it." Everybody else gets an email from the bot saying, hey, you're past due, this is the invoice. This is when it was originally due. This is how many days past due you are. If it keeps going, we're going to shut you off or whatever it says. But it just automates that process. And so that was just with Power Automate. Every month we have a review of the outstanding accounts receivable.

(00:34:23): And I remember we got a Power BI report for that. It was at least a page and a half long, a screen and a half long of always reviewing. Okay, what's the story with this guy? Once we started doing this, it's collapsed to just half a screen. So it just like the number of people that we have being late on their payments. Where as I said, when it starts to get over 60 or over 90, that's when we start to get agitated. It's just dropped dramatically. It's like, wow. And it's just the bot. The head of sales is like, "Wow, this is great. I spent 30 seconds a week to tell it which ones not to send an email to, and then it goes out and wow, the money just keeps coming in."

(00:35:03): Tied into it. We also do other stuff, like the moment somebody flips over to 60 days, it's 60 days past due, we're sending an email to the salesperson and the head of sales and the head of accounting person. So they all know, oh, get more aggressive on chasing this person. The day that it hits 60, the notice is sent out to everybody. But then also it's watching to see if that person pays and then it'll send a notice out to everybody, oh, good news person paid, it's all taken care of. It's kind of doing that monitoring. Obviously you do it just by printing the reports and stuff, but this way it's just really customized and as the moments happen it sends it out. And if they hit 60 days, all the key people start to get their own emails about it saying put the pressure on.

Rob Collie (00:35:48): That bot you just described, the accounts receivable, overdue bot, and this, what do you call this other one?

Bill Krolicki (00:35:54): VendorBot.

Rob Collie (00:35:56): Okay, VendorBot. So the supply, we have the stuff, the VendorBot. They're so similar, but one of them you did before AI and one of them you waited until AI was on the scene to start. And my hypothesis is you just kind of knew that the second one, the VendorBot was made possible by AI in a way that it wasn't possible before. So starting from that hypothesis that AI made it possible. Your first line of exploration with this was to have the AI do a lot. Where you landed was the AI is serving this tiny little coupling piece. It's like this last link in a long chain or a middle link in a long chain.

Bill Krolicki (00:36:37): It's a middle link in this one, but yeah.

Rob Collie (00:36:40): Yeah. And it's really just almost like a function that is going to be late, yes, no?

Bill Krolicki (00:36:46): Yep.

Rob Collie (00:36:47): And this ties in with something I've been really just delighted about lately, is that things that we used to think about as operations on structured data can now be with the help of LLMs and GPUs, we can now perform these very, very similar kinds of operations. The numerical version of this would've been like, is days late greater than zero? But you can't do that on an email.

Bill Krolicki (00:37:12): That was why you needed the AI. And email is coming in and you don't know what the people are saying, but the AIs can read the email. And if I would understand that he's saying it's going to be late, the AI is going to understand that it's going to be late. It's going to be late by three days. And this is where you talked about with your hockey league type of stuff, where it's like, wow, it's amazing, it can pull this stuff. Where it says three days late and it realizes that the 15th is the original due date and add three days onto that is the 18th is what it's talking about. So it can do that type of thing or as I said, sometimes people can save things in vague ways. But the AI is smart enough to realize that, whereas if you tried to hard code it, you'd never be able to get to it.

Rob Collie (00:37:53): Everyone communicates differently. The conversation is in a different place with these people. They're in different mindsets, everything. So the AI in this case is like this shock absorber on the variations of the English language in this case and all of the meaning. But it really turns an English email into something that can have an if.

Bill Krolicki (00:38:15): So like I said, it turns into the yes no function. It's basically it's just saying, here's the email, is it telling us it's going to be late? Because a lot of times they'll respond and they'll just say, "Nope, everything's good." And it knows, okay, everything's good, that means it's not late. Whereas it said, oh, we've got a problem here. We're looking at more like the 31st. It knows, oh, that means late because originally 15, now 31. Oh, okay.

Rob Collie (00:38:36): What a cool journey knowing that AI made it possible. Thinking that AI made it possible, let's use the AI intensively sense, but coming to the conclusion that it's just this little link, it's a crucial link.

Bill Krolicki (00:38:51): You've talked about it before in your other things, which is things like the AI is actually not very good at a lot of times numbers and stuff. It's very good with language, so it's not numeric. It's okay, it's just a little vague, it can handle that. But if you're asking it to do the calculations, it's not quite as good at that. And if you want that determinative reliable results, that's where it's like, okay, let's get some code on that part. And that I can code well, but the AI, that's where it can orchestrate and know hopefully, and this is where we're experimenting, we're still figuring things out. But at least in my mind, the framework is if I want deterministic the right answer, you can get some code for that. That's where Power BI is really useful. I've got a model. And then once again, even the AI can help me write the DAX query or whatever it is that I need to get the answers out. And then there it is, okay, there's the little query that you got to send out. You'll get an answer, and then the AI can go from that, but you're relying on the DAX code, you're not relying on the AI probabilistic, oh, that sounds good or not.

Rob Collie (00:40:02): The hybrid of CPU software, traditional CPU driven software like Power BI and Custom Code and Power Automate and all those sorts of things playing like a symphony or a duet with the LLM? So many of our systems are going to be hybrids of the two.

Bill Krolicki (00:40:18): Where I think it's really valuable once again, is that workflow idea because if it's like we're going to go kind of through this standard process. So theoretically, once again, in this beginning phase at least, the AI can guide things along or you can lay down a rough path for the AI to follow. And the people were like, "Okay, great, the AI can handle that. I don't need to handle that. That's easy." And it's the AI smart enough to be able to handle those easy things and then just the hard stuff is left for me.

Justin Mannhardt (00:40:48): This is a good cautionary tale, like a slippery slope I've experienced myself or I've seen others, where they'll watch AI in whatever configuration, let's say it's working with a database of any kind. And they'll watch it. Oh, I asked it a question and it determined it could write a query. It wrote that query. It sent that query to the database and then it got an answer. And they'll see that behavior and they'll think, oh, this thing could just write the queries every single time. But the way AI works, so you can ask it the exact same question and it will not write the exact same query every time. It just won't.

(00:41:26): The LLM provider, they'll release a new version and now the behavior's different. It can help you figure out, well, the queries might want to add to my database or to my model, but once you get those things, you want to lock them in place and put them in the tool belt for the AI system. So it doesn't have to reason through how to write that thing, it can just go grab it off the shelf and use it. I just don't think we can fall into that trap of just like, oh, it'll always figure out what to go ask and how to ask it. In some cases you want that, in other cases, you want, like you're saying, Bill, no, I want you to do exactly this search in Global Shop. This is the search.

Bill Krolicki (00:42:03): Yes, right.

Justin Mannhardt (00:42:03): No other search.

Bill Krolicki (00:42:07): Don't figure it out on your own.

Rob Collie (00:42:08): And Power BI models provide some of that in the form of this measure formula is the same every single time. So even though the LLM when it needs to go self-serve some information from that Power BI model, it does have to write a DAX query.

Justin Mannhardt (00:42:23): Yeah. But it's not figuring out how to calculate days sales outstanding on its own, right?

Rob Collie (00:42:28): The surface area for variation is significantly reduced by that formula being hard-coded and only executed a certain way ever. And the more obvious you make the model, the less chance that the query is going to be improperly formed as well. So there's an art form to this. Bill, when we had our call before, did we demonstrate to you at all anything with our P3 AI site?

Bill Krolicki (00:42:55): No, you didn't demonstrate it, but I was betting that you guys got to be coming up with great stuff. Geez, you got to have some really good people there who's able to apply it because that's the thing, that's the difference with you guys. You guys are really good because you have... Sometimes the coders, they miss the bigger business purpose and they just fall in love with the code aspect of it. And then you have some people, I know the business process, I have no idea how to apply the AI or the code or whatever. But you guys are filled with all these people who are that a lot of those in-between people who are like, no, I understand what the business user wants and I can organize it on the back end for the coding to get them what they need. And that's part of your faucet's first methodology, but that's the way to go. Especially with this stuff, you can develop it so quickly. And with the AI, I would think your programmers have got to be super turbocharged because it's like, oh, now I got to do that. And the AI is writing this stuff so quickly, creating huge amount of value for customers.

Rob Collie (00:43:56): There's two parallel tracks going on here. One is using AI tools to accelerate our traditional workflows, the BI deliverables, delivering those in an AI powered fashion. And then there's also us getting into these custom AI solutions like the one you've described for yourself. We're building those sorts of things for ourselves, and we're starting to run some pilot programs with our clients as well. The internal usage that we've got on a couple of things already and the capabilities are really interesting. You mentioned there was one scenario, I forget what it was. But it was like ideally people would be able to self-serve, are things going to be on time for this customer?

Bill Krolicki (00:44:39): The salesperson knows that the vendor has to deliver that on time in order for it to go out to the customer. So they're trying to keep an eye on what's the status.

Rob Collie (00:44:48): That would be a perfect example of the scenario we call chat with data. Get answers that are available in structured systems, maybe across a mix of structured systems. Maybe it's in a Power BI model, maybe it's in a Power BI model plus some things. Even just translating that question that they have into, oh, I need to go find it. In this Power BI report somewhere is a bridge too far. That's when they reach for the phone. And I completely get that, I completely get that. Being able to sit down and just ask a question of a chat interface that has access to those data sources that can go and answer that question would be amazing. There is a little bit of a retraining there. Picking up the phone to them does seem like the easiest way, but maybe the person on the other end of the phone can then use this tool.

Bill Krolicki (00:45:37): Right. As I said, the purchasing manager will say hey, we've created a purchase bot that answers all these questions, stop bothering me, just ask it and you can even just turn the microphone on and talk to it, it will answer your question there.

Rob Collie (00:45:45): So we've been tracking Microsoft Copilot in this space and we've talked about that a lot on the podcast. And there's new versions of it coming that I think are going to be amazing. And in the meantime, we've built our own and that's one of the things that we could demonstrate to you on P3 AI. And at the moment it's more capable by quite a bit than the Copilot from Microsoft. They're going to release big chunks, they're going to improve by leaps and bounds. But at bare minimum, what we've got is a preview of where they're going to be, but we also have the ability to customize it. The entire chat interface, everything is something that we have control over. Any feature or capability that's going to be missing in Copilot for a while, well, we can get that going now. So this is both kind of a proof of concept of the future, but also I think it's going to have some legs for us and our clients in parallel with Copilot, we're going to be doing both. So that'd be one really, really good example to show.

(00:46:45): But then the ability to hybridize the information that's available to one of these chat agents. So giving it access to some Power BI models, but then maybe there's also one SQL source that is good to cross-reference with. Just as interesting, I'm just going to say even more interesting. But just as interesting is the ability to have it plug into databases of text instructions. So it's similar to custom GPTs and ChatGPT, but way better honestly because you have such granular control and you can also mix and match. Certain agents might want to load X, Y, and Z, and other agents load X, Y, and W, because they have different purposes.

(00:47:26): We have a copywriting agent that enforces our brand guidelines. There's even one that when I sit down to write, I can say I can load an additional database, which is write like me. It's been trained to write like me, and I still interact with it a lot. I don't just go with what it's got, but it's nice to be off of that starting line. It's already writing in a voice that's familiar to me that I don't have to go change all this wording. I don't talk like that and all that kind of stuff. The ability to plug into any source of information that you want to control the interface and what it loads from the beginning and to monitor it, control it, like it's pretty cool. We're pretty excited about it.

Bill Krolicki (00:48:08): I'd love to see how that works. I'll tell you the idea that I want to work on next. Maybe you guys will be the ones who could actually execute on it, budgeting. So here's a standard process that every company goes through is the budgeting. There's some problems with the budgeting in terms of sometimes the finance department does the budget and then passes it out to marketing and all the other people. And they didn't really create it, so they might give some pushback on some stuff. But they don't have any ownership for that budget. On the flip side, you can try to get them to create their own budget, but a lot of times they're not really good at it in terms of digging through here's the data and here's the Excel formulas and how to override it and that type of thing. So that's the problem is like, okay, I want them to own it, but it's too much work for them. I could try to sit there with them, but then even for me, writing all the formulas, it gets messy really quickly.

(00:49:04): The vision is like this is that you have your budget bot, so the budget bot and say you're doing the zero-based budget or something like, okay, I'm doing this GL category, I'm doing the travel budget. Show me what I actually spent on this GL category, the travel category last year by vendor. And then you can go and you get that simple table. That's just pulling from your Power BI model, it's a simple little pivot table. So you get this. Now you can see by vendor which month, all that type of stuff. Then you can start doing stuff where you can talk to your budget bot and say, oh, okay, for that hotel's expense there, that is going to be the same as last year, but increase it by 5% for inflation. And then over here, this expense with that vendor, it's the same as last year increased by 5%, but in March we're going to make a single purchase for $50,000. So you got to add that on in there.

(00:49:59): So this is where you could have, the person could have the conversation with budget bot. It could give them, here's all your historical information so that they're basing it on something. They can be talking about it. Based on what they're saying, it'll make the calculations of, okay, here it is up 5%, here it is with that $50,000 in March. Here it is with this. And the budget bot keeps track of your reasons behind you gave the budget, why you said in March it's going to be higher by 50,000 bucks. Because what will happen is you get into next year, you give the person the budget and you're doing a here's budget versus actually get into March. And they're like, well, I don't remember why the budget is what I said. And then here, budget bot would've remembered and kept the notes that said, oh, you said it was going to be up by 5% in March, you said you were going to be doing this, so that's why you're seeing this kind of blip in the budget or something like that.

(00:50:59): So that's the type of thing where on the one hand there's a component of it that's in terms of hard numbers. You could be pulling it out of Power BI in terms of here's historical stuff. The transformations you're asking it to do are not terribly hard, but a non-Excel person could talk to it and have the thing generate. And then as I said, it saves the file and then it'll get pulled up into the Power BI model under the budget table. And there it is, going forward next year, you're able to have these really tight actual versus budget comparisons and the people can look and explain. So when you asked them why are you off a budget, they'd be able to look and say, oh yeah, well last year I budgeted this and it actually turned out to be twice as expensive as I thought, or something like that. Whereas before they went, "I don't remember. I don't know. It's what it is."

Rob Collie (00:51:48): It could capture both the hard answers to the questions as well as extract some summary comment for each decision sort of like why it is what it is. It can act as a bit of historian as well, which is really, really, really, really, really compelling.

Justin Mannhardt (00:52:05): budget bot never forgets.

Bill Krolicki (00:52:07): Exactly. You can try and do it with Excel, but it's hard to get all those kind of components in there with the formulas and everything. Whereas with budget bot, it can handle all those calculations. It can understand what the person's asking for. They don't have to be a finance expert, but they can give their explanations and reasons on why they're budgeting the way they are. That in itself is a big pain when you're trying to do the budgets and people reviewing them and looking at it.

Rob Collie (00:52:34): If I understand correctly, the way this would work is that budget bot would interview each person essentially. And budget bot by looking at historical data and historical budgets, would already have a framework for all of the questions, at least by default? Yeah, yeah.

Justin Mannhardt (00:52:52): Like a process for the interview.

Rob Collie (00:52:53): It's just almost like a four loop, first line item, right? This is what it was last year. Give them the context. This is what it was last year, do you anticipate changes to this and then blah, blah, blah, blah, blah.

Bill Krolicki (00:53:03): And it can give suggestions, do you want to increase it by a percentage? Do you want to increase it by a fixed dollar amount? Here's the total amount and I'll just evenly spread it over the months. I can give you different options that you can do and then it'll just go and do it. And then like I said, then you can also open up thing where the person say, yes, it's like that, but for this particular month it's going to be different because of this. The bot would be smart enough to be able to understand that, but if you're doing it in Excel, it's a pain in the butt because like, oh great, I can write the formula and copy it over, but now I'm going to break the formula flow and this one is hard coded and then later on it turns into a nightmare.

Rob Collie (00:53:40): Checkpoints throughout this process or even near the end of it, maybe. There could be a final human in the loop type of thing. It could say, okay, let me recap what I think we've got just as an extra shock absorber for misunderstanding or whatever, and the opportunity for correction. I think this is definitely something that's buildable.

Bill Krolicki (00:54:01): You guide it through. And then as I said. At the end of the process, whatever, whether it's doing it iteratively through there because it keeps putting it back. Like, okay, you said it went up 5%, here's what it would look like if you increased it 5%. Oh, but you've forgot about March. Oh, okay, blah, blah, blah. And then just lie you say, the end it goes, okay, so what we've got here for the budget is this and these are all the points that you made about it and these are the numbers. You good with that? And if you're good, I'll go and turn it into an Excel file and put it in the budget SharePoint site. And so next year it's all there and the manager can review it and all your explanations for why you're doing what you're doing, you're there. It could make the process so much easier because you just talk to it and say, yes, no, I don't like that. Oh.

Rob Collie (00:54:45): At each step it's doing something small, right?

Bill Krolicki (00:54:48): Right.

Rob Collie (00:54:48): It's asking a small question based off of a small piece of pre-existing information and then critically it's writing a small amount of information somewhere at that point. So that it's not having to carry all of this, it's not being overwhelmed by the entirety of the conversation at all times and it's carrying just enough of the conversational thread forward. It's good, but it's not going to do the numbers that add to 7 and thinking that it's adding to 14, which is what happens when it's just overwhelmed.

Bill Krolicki (00:55:18): You said this is the kind of conversation to be quasi-structured, so then you can give it even better, if it says, do you want me to increase it by 5% or some percentage point, then you're even telling it. Here's if they say yes and the number's going to be something, put it in there. Here's the formula to use, or something like that. So once again, just like you're saying, you're not overwhelming it and then there's another question and can help to reset and each question passes on, okay, here's where our numbers are right now and these were the reasons that we had, and then it keeps carrying itself forward.

Rob Collie (00:55:52): To further illustrate the point when it summarizes back to you, here's what I think we've got?

Bill Krolicki (00:55:58): Yeah.

Rob Collie (00:55:58): It's not pulling that out of its conversation history, it's pulling that out of the database it's been writing to so that it's not at risk.

Bill Krolicki (00:56:07): Right, exactly. It would be increase it by 5% and it now knows, oh, there's there and I put that in my reason database and for this DL number and this date or period or whatever.

Rob Collie (00:56:19): All right. When do we get started? We're ready to run.

Justin Mannhardt (00:56:20): Yeah, yeah. Let's line it up.

Rob Collie (00:56:23): budget bot coming up.

Justin Mannhardt (00:56:25): It further emphasizes how to think about the value chain with AI. Everyone has examples in your life where you value another party, whether that's a person or a computer system, being able to understand you and then to translate that into the right things. Think of something like in your home, your air conditioners on the fritz. Well, I don't know how to fix the air conditioner, but I value having someone that will understand me explaining that it's broken or what it's doing and they know how to translate my problem into like, oh, we got to fix this. I've built these types of systems, Bill, the FP&A systems for planning and these complex write back systems. And there's so many instances where we're trying to train users how to navigate the tables or the screens.

Rob Collie (00:57:18): Yes, yes.

Bill Krolicki (00:57:18): Extremely large.

Justin Mannhardt (00:57:20): And they're just like, "Gosh, dang it, Justin, I'm just trying to spread this number from here to here. Where do I click? What do I type?"

Rob Collie (00:57:28): You right click and you got to hold your pinky off the mouse when you right click. That's different than when you have the pinky on the mouse.

Justin Mannhardt (00:57:34): They just want someone to understand what they're trying to say, but then also having an AI system that understands, well, Bill back in the finance and accounting team, they want to run this on an accrual or on a deferred basis or whatever your rules are. And so it knows your rules and what they're saying and it just makes it-

Bill Krolicki (00:57:53): Oh, that'd be wild, yeah. Really.

Justin Mannhardt (00:57:53): Makes it work, right? And so that's where I think we've got such a unique opportunity to... Listen, AI is going to change the way we work a lot, but just to make it easier for people to interact across functional areas of a business.

Bill Krolicki (00:58:07): That's the thing. It's like you're going to the sales guy or whatever, head of marketing, they got to put together the budget and then they'll like, "I hate doing this." And they do a-

Justin Mannhardt (00:58:16): Ah, budget season.

Bill Krolicki (00:58:17): ... half-ass job of it. And then instead you get them and you can guide them through just work with the budget bot, it will walk you through the process and you'll come through with it and then they go like, "Oh, that wasn't that bad. I didn't have to wrestle with Excel. I didn't have to do all these formulas. It made it easy." And that's the type of thing, just like you said, it's that cross-functional thing. I'm not an accountant, I don't know how to do all this stuff.

Rob Collie (00:58:37): There's so many things in life that are the equivalent of telling adult professionals that they can't get up from the table until they've eaten their vegetables.

Justin Mannhardt (00:58:48): Turning in your forecast spreadsheet is one of them.

Rob Collie (00:58:52): AI, we eat the vegetables.

Bill Krolicki (00:58:59): Well, one of the things that you've talked about before on other podcasts that I think is related to this. You've talked about the issue a lot where, okay, we get Power BI and I set up a whole bunch of reports and I set up a whole bunch of dashboards. Then you'll look at statistics and people never use them or it's the type of problems like, great, I get this one dashboard that's really universal, but it has a gazillion little filters and clicks and things that you got to do and then it's hard for the person to interact with or they forget that the problem that they run into that dashboard answers, they only run into it once every three months or something so they forget it's there.

(00:59:39): In this case, this is where the AI once again can come into place where they start to get used to say, no, I just asked the AI to do it and it knows all the dashboards and it knows how to interact with them. It knows whether it can guide me in terms of like, okay, well which date range we talking about? Oh, are we talking about this product line or that product line? We're doing this or that? And it can guide them through verbally through the filtering selection options process on these dashboards and knows here's the dashboard that you want to go to for this type of problem or it can quasi guide them on that. So that's where that problem that you talk about in terms of people design all these dashboards that do all these things and then they stop using them because they're either too complex or they forget it exists and sometimes they forget it exists because it answers a specific problem that only happens twice a year.

Rob Collie (01:00:28): And even the way that you name them can throw people off over and over and over again. It's just another example of human beings formulate questions, needs, desires in a certain way in their heads. And then all of this software in the world needs to absorb that interaction in a particular way that is very computer-y, it's very techie, it's arbitrary. It's not the way, it's the joke again, "Did you try shift right click?" I mean, "What are you talking about?" The more that we can meet people's needs in the form that they show up with, whatever they've described the problem or the question or the task in their own mind. If that description, if that is sufficient to get them on the path to success... The budget bot interview is another great example of that. But it's like they just sort of sit there and react, it's even better.

Bill Krolicki (01:01:24): Right. I don't have to know how to work with Excel, I don't have to do this. And then even if the finance department puts an extra little guidance or something like that, they can help them out on, "Ooh, did you think about this or what about that?" Or something like that. And they're like, "Oh, yeah, I forgot about that." "Ooh, yeah, I'm going to hire two people, I didn't put that in the budget, did I?"

Rob Collie (01:01:43): And by the way, that interview hack, the LLM, the AI interviewing you is one that I've kind of learned to use even in my off-the-shelf usage of tools like with ChatGPT or even some of our customized chat agents on P3 AI. I used to sit down and say, okay, right off the bat, I need to tell this thing everything about this problem and describe the whole thing to it so that they can then help me answer my question. But now sometimes I get lazy and it's really good lazy. I sit down and say, okay, I've got this kind of question, but I need you to ask me what it is you need to know to help me. Then we're off and running with that interview and I could just be that relatively low mental energy. I'm just answering questions, it's just the energy cost of it is so much lower that way.

Bill Krolicki (01:02:35): That was the big thing with AI. Remember you were talking about, and I completely agree, where sometimes you're like, oh, well I'm using the AI, but the amount of time I spent doing the project or whatever writing up the thing was probably about the same amount of time that I would've spent if I started from scratch. But the big difference is you get to the end and you have so much more energy. For me, I don't do a lot of coding, so I'm not very good at it and the syntax and all that type of stuff. That's where AI was the greatest gift when I'm like, "Wow, it's doing all this stuff and I get to the end and an error message pops up and I just feed it back in and it gets there." And it's like sometimes when it got stuck, it kind of went off the rails and it took a little while, but I'm like, "Wow, I still feel so much better." Whereas in the old days, if I was trying to fight my own way through there, I would get to the end and I would just be exhausted.

Rob Collie (01:03:32): Drained, yeah.

Bill Krolicki (01:03:32): Whereas this way it's like, "Oh wow, I figured it all out." It's like, "Geez, I wish we'd figured it out faster, but it got there eventually."

Rob Collie (01:03:38): It's for the next challenge. I used to joke in the classes that I would teach is that we often cost projects. There are things that appear on spreadsheets, time and money appear on spreadsheets. Those go into the how much of this project cost us, but suffering never appeared on any spreadsheet anywhere. And ideally it would because the psychic injury or exhaustion that you conflict on a team is important. Do you moralize how worn out are they? And there's no way to cost that. It's neat to see this concept coming back around it and in such a positive way. It still probably is faster to write something with the help of my Rob agent than without it. But oh, in terms of energy cost, it's like 10 times better.

Bill Krolicki (01:04:28): They say with AI, it raises the floor more than the ceiling. I'm trying to do something and I'm not a good programmer and I don't know this stuff. But it works at the end of the day and I'm like, "Wow. I never would've been able..." Especially if it was like, oh, it was setting up plug code in the Linux thing. It's like I have no idea what's going on here, but it's like, wow, I got there at the end, I have no idea how, but it took me there. It took a little while, but I'd never be able to do that on my own.

Rob Collie (01:04:56): Well, I think there's different types of applications of AI and one bucket raises the floor and the other bucket raises the ceiling. So for example, this copywriting agent that knows our brand framework, knows our history, knows everything about our target customers and knows about our ethos and our approach and everything, it knows a lot about us. We've quote-unquote taught a lot about us. I honestly think that that is a better copywriter and better marketing brainstorming partner than what you could pay for if you had an infinite budget. I think it's better than anything we could pay for, which is crazy, that a company of our size now has access to something that you could pay a million dollars a year and not get the same quality of output.

(01:05:49): It's really telling, and this isn't criticizing anyone, this is just showing you just human limitations. It's really telling that the same brand firm who helped us come up with this framework and it's a really good framework and we couldn't have done it without them. They're really, really good. That when you ask them to write copy for us, they drift back inevitably to their own personal brands. Even they have a hard time putting on a brand framework that they built. Now they use the same tool that we do, that's trained in large part on their output, on their professional output. It's just amazing. That one raises the floor and the ceiling because now we can produce things with lower friction, so we have a little bit more throughput. But the quality level is Madison Avenue level, it's like a Fortune 500 firm actually can't buy better than what we have, which is so wild.

Bill Krolicki (01:06:50): Right. That's where I was saying before where sometimes I go to do the analysis properly, you got to do all these steps to find out, yeah, it was what I thought all along and then people can get lazy. Where this is the type of thing with the AI, it's not going to get lazy, it's just going to keep on doing again and again and again, it's like, wow, the quality of our work we do is great because I got this AI bot who doesn't complain and shows up on time every day and works 24 hours.

Rob Collie (01:07:14): It does remind me of the monologue from the Terminator, it doesn't get tired, it doesn't feel remorse and it will never ever stop until your budget is done.

Bill Krolicki (01:07:28): But if you're riding on it's back then it's pretty good. It's like, oh, yeah, keep going. Keep going, Arnold.

Justin Mannhardt (01:07:36): I think Rob just nailed it for you, Bill. You have two classes of AI solutions at Inter Pack. And some of them, they get the name something bot and the other they get something nator. So I think you go with Budginator.

Rob Collie (01:07:50): Budginator.

Justin Mannhardt (01:07:55): Scheduler Bot works, but definitely Budginator.

Rob Collie (01:07:58): Budginator, yeah. We started off with the B-300. We're now on the B-3000. Were there other scenarios that you sort of have in flight, other projects, other AI ambitions?

Bill Krolicki (01:08:13): There's lots of different ideas, but once again, to execute on them takes a whole bunch of time. It takes that combination of, I need somebody who actually knows the process who's willing to work with us to say like, okay, well, here's the process and dah, dah, dah. Then on our IT side, the IT guy's able to figure it out on the negative side. We don't know how to do all this stuff, so we're figuring it all out. But on the positive side, it feels like, oh wow, we're on the cutting edge because nobody else knows how to do it either. I said, when you were giving your podcast and you're talking about context windows and all that.

(01:08:48): Our guy here, Nick, I talked to him the next day and he was like, "Oh, yeah, I was running into the context issue and we burned through all our capacity on our F-2 thing, and it was like dah, dah, dah. It's exactly what you were talking about. So I'm like we're all running into it. And I listened to you." This is the difference, when I was learning Power BI, I was like, "Oh, wow." These guys were all the gurus who you listened to, and you assume they knew everything. But now it's like, wow, I'm listening to all those guys, I listened to Brian Julius and Sam McKay and they're just figuring it out, they're not that much further ahead than I am.

Justin Mannhardt (01:09:18): Just like the rest of them. Yep.

Bill Krolicki (01:09:22): So that's exciting, but it's also hard in terms of like, nope. And the stuff is changing so fast. I like listening to Brian Julius and Sam McKay on their podcast because you listen to them and how they're like, "Well, three months ago I thought this, but now I have a total different perspective on it I didn't understand before, but now it's this way. And oh my God, I thought this was the greatest thing ever." And three months later, "Oh no, no, no, it's Gemini, gemini is the greatest thing ever. Oh, no, no, no." It just changes so fast.

Rob Collie (01:09:55): One of the ironies that's just sort of floating in the back of my head the whole time we're having the conversation, the average age at your company is like 55. Maybe that is the sweet spot. I know you've got one productionized workflow at this point, but it's a really good one. It's a really, really, really good one. And it does, it puts you in a fraction of the 99th percentile, especially for firms your size, maybe that this average age at the company is actually like... I'm saying this selfishly as someone who just turned 51.

(01:10:24): So maybe it's us. Maybe we are the long-term ongoing exposure to the energy cost of all of this manual shit. The motivation to do something differently might come out of that. Another way to say it is, I think it's more mindset than just age, just generational. I understand that the younger generation is going to grow up chatting with ChatGPT. I was watching advertisements on football this weekend for ChatGPT of a teenager who wants to be able to do 10 pull-ups by the end of the summer. And ChatGPT is developing a training program to make him stronger. It is a consumer.

Bill Krolicki (01:11:13): [inaudible 01:11:14]. Like a gym instructor ChatGPT.

Rob Collie (01:11:17): But a friendly one. That is constantly telling you that you're stronger than you actually are, pumping you up. Anyone who's in school right now or just got out of school, it's like, "Yeah, we were using it to death, it was writing our papers." And they've developed a consumer relationship with this stuff even in their personal lives that maybe the rest of us wouldn't have done. It's really professional reasons that dragged me in to AI. And then I discovered it's useful in my personal life. The younger crowd might be the reverse. They started in their personal life essentially and then it just bleeds into their professional. But if average age is an impediment to adoption, imagine what you all would be doing if your average age was 49. I mean, you guys are...

Bill Krolicki (01:12:01): I think paradox is kind of what you're saying there. You got a good point where actually the thing is sometimes it's the people who have more experience and more over the broader knowledge of it, that's where their value is. Their value is getting subtracted by having to do a whole bunch of these manual tasks that are subtract from it, but it's their skill that they've acquired over years of looking at it in their own pattern recognitions and then that can really augment it. So it's like, great, I can take away all the grunt work away from you, and then just really start to utilize your ability to analyze it and say, oh, I know what that problem's caused by, I know how to solve that. I know what that is, and I know what's going on there and use this solution, use this solution. And the bot goes out and then just executes. But you still need that person where it's valuable to have that person with that extra knowledge. I think that's why sometimes in the medical profession, they have things like, oh, radiologists or whatever, the guys who read the x-rays are all going to disappear. Instead, it's like, yeah, it doesn't work that way.

Rob Collie (01:13:01): Not yet anyway. We go back to the scenario where the salesperson picks up the phone to call and find out if X, Y, Z is going to be built on time, manufactured on time. If that person making that phone call is 25 years old, and you give them a chat interface instead, an AI chat agent for that. They're going to go, yes, I much prefer that to picking up the telephone and calling another human being, right?

Bill Krolicki (01:13:27): Then you're going to get on the other side a bot that's answering the phone. It's going to be the bot saying, hey, late, you guys going to be on time this week? Oh, this one's going to be late, I'm sorry.

Rob Collie (01:13:39): Yeah, yeah. Why not, right?

Justin Mannhardt (01:13:40): You guys seen the video of the two AI bots talking to each other in some bit language or whatever?

Rob Collie (01:13:48): No, I haven't.

Justin Mannhardt (01:13:49): I'll send it to you, Rob. Basically, it's like a bot calling a hotel to make a reservation and they go, "Hey, I'm AI, you're AI. Why don't we talk in this other thing?" It's basically like [inaudible 01:14:00].

Bill Krolicki (01:14:00): Talking binary.

Justin Mannhardt (01:14:02): Because they I can compress the meaning to just so much.

Rob Collie (01:14:06): Yeah. You know what? Just send me the vectors, right?

Justin Mannhardt (01:14:09): Yeah.

Rob Collie (01:14:13): Whatever text you send me, I'm just going to take it and turn it into vectors anyway. Let's just cut out the middle man, just send us the thousands of numbers.

Justin Mannhardt (01:14:20): Here's the vectors.

Rob Collie (01:14:24): A little nerdy there from a moment, a little AI humor, if you will. But yeah, Bill, seriously, I think you and your team should be very, very proud of where you find yourselves. And not just in terms of current achievements, but your mindset. Your mindset. You think about where you are today as a noun, how you operate, how you think is a verb. The verb is so much more important than the noun. You're in exactly the right groove verb wise. And I think that is the most valuable way in which I would say that you're at the front of the line amongst your peers, even more so than just measured by what you've currently got deployed. You could rewind six months, you have nothing deployed yet, but if you have this mindset, it doesn't matter, you're going to win. We'd love to talk to you about what is it, Budginator?

Justin Mannhardt (01:15:11): Budginator.

Rob Collie (01:15:14): No matter where that conversation goes, we could at least talk about it because I do think it's totally doable.

Justin Mannhardt (01:15:20): Totally.

Bill Krolicki (01:15:21): That'd be great. But this is these types of things where I think AI, I think about electricity 150 years ago. Electricity is like, oh, wow, well, what do we do with this? And it takes a long time for somebody to come, I could make a dishwasher with electricity. It's like, okay, it took a long time for somebody to be like that wasn't exactly what I was thinking of when I came up with electricity.

Rob Collie (01:15:43): It's funny to think of it that way, right? What about a TV?

Bill Krolicki (01:15:44): Exactly.

Rob Collie (01:15:44): What?

Justin Mannhardt (01:15:45): What's a TV?

Rob Collie (01:15:49): What's a TV? Well, Bill, thank you. Thank you, thank you so much.

Justin Mannhardt (01:15:54): Yeah, thank you, Bill.

Bill Krolicki (01:15:54): Great.

Rob Collie (01:15:55): What a great conversation.

Bill Krolicki (01:15:56): Yep. Well, good talking to you guys. And like I said, good luck with Budginator.