Tales from the Five Percent: Tangible AI Success, w/ Tuio’s Juan Garcia

announcer (00:00:04): Welcome to Raw Data with Rob Collie. Real talk about AI and data for business impact. And now CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie (00:00:20): Hello, friends. Every week here at Raw Data, we receive multiple unsolicited emails pitching us on having a new guest on the show. And most of the time, the suggested guest in the email is a flat no for us. And there are several flavors of flat no. One group is the scammy influencers. The people you see on LinkedIn with deliberately provocative content that proves to either be lacking in substance or honesty or both. Nope. Then there's another group, people who are so clearly selling a product and only selling a product that there's no way we'd be able to have a conversation that's useful or enlightening for us or the audience. Also, nope. But every now and then, and I mean like once or twice a year tops, there's a ringer, someone truly interesting. And those rare exceptions, like today's guest, Juan Garcia, are the reason why we keep reading those emails every week.

(00:01:18): Because the occasional diamond makes it worth sorting through the otherwise steady stream of noise. Juan is co-founder of an insurance startup in Spain called Tuio. Now, they didn't set out to be an AI powered insurance company. They set out instead to be a data-driven and modern insurance company. But then after ChatGPT hit the scene, they added AI into their operation. So they're not one of those, "Ooh, let's apply AI to industry X pop-up companies." They were already in business when AI landed on the scene, and then they chose to take advantage of it much sooner than most. That's important, both because the before AI and after AI of the whole thing makes for a more relatable story for where most companies find themselves today in that before AI state. And because Tuio's story is much more tangible than those we're going to apply AI to industry X companies.

(00:02:11): Those stories for those companies tend to be breathless and shallow and fall into the, we're not having them on the show category. But that's not Tuio. They're an existing company with an existing business plan that decided to take advantage of AI along the way. A couple more things that stood out for me about this conversation. One, we're still hearing about the 95% failure rate for AI projects. That number came up again last week in the news from the Davos Summit. And if we believe that 95% failure rate is still true, it's increasingly important to hear from the other 5%, the ones who are succeeding so that we can learn how to emulate their approaches. And two, we had never met Juan before and never once spoken with him before we sat down to record. And yet, it was like we were reading from each other's playbooks.

(00:02:59): Tuio's approach to AI matches our approach to AI. And I think that's just what happens over time. A couple of years from now, the AI playbooks we're following at P3 and Tuio will just be considered the best practices everywhere. They're not going to be called the P3 method or the Tuio method. It's just going to be called, the method. We're not there yet in the world, but it's coming. In the meantime, all over the world, there's a small minority of companies independently cracking the riddle. And I don't think there's a lot of different ways to crack it. I think it's closer to one way. Sure, there'll be all differences in details, but a handful of specific themes are going to be found repeating themselves in every success story. Those are the themes we talked about on the show today. Those are the themes I'm capturing in the book that I'm writing.

(00:03:45): And rather than feeling threatened by Juan and his company having figured out the same sorts of things that we have, I felt all kinds of validation and kinship. We could have talked to Juan all day, and I think we're going to have to come up with a reason to bring him back. Already then, let's get into it.

(00:04:04): Welcome to the show, Juan Garcia. How are you today and how is beautiful, beautiful Spain?

Juan Garcia (00:04:10): Well, I have to give you a bit of a heads-up. I am not from the part of Spain that's really sunny all the time. I'm actually from the north. So we have this beautiful mountain range that grabs all the clouds and it rains. I wouldn't say all the time, but it rains quite happily. So you could think about us where I'm from. It's like the Pacific Northwest.

Rob Collie (00:04:30): The Seattle of Spain. Yeah.

Juan Garcia (00:04:31): Yeah, yeah, yeah.

Rob Collie (00:04:32): Okay. So Juan, we're really excited to talk to you today. Tell the audience really quickly what your job title is, what your company is, and what you're up to.

Juan Garcia (00:04:39): My name is Juan Garcia. I am one of the three co-founders of Tuio. I run mostly growth product and partnerships here at Tuio. I have two co-founders. One is Josemaría. He runs all finance strategy and operations, and the third is Asís, runs technology and data. We are an insurance company. We leverage technology and now AI obviously, is the password of the age. We are focused on customers from 25 to 55. We realize that this is a very underserved market, at least in the Spain. And it's underserved because of two reasons. One is a customer segment. This segment is used to like, "Oh, we can consume content from Netflix. We can listen to podcasts online and Spotify, and then we can purchase stuff on Amazon and we can even buy our groceries online," right? And we're used to these sub-surface digital first interactions with the companies that we work with.

(00:05:31): Insurance, there's nothing, or at least when we started, there wasn't anything like that, at least in Spain. And looking at from the industry side, and this is a big problem because the industry, when you look at the profitability of these customers from 25 to 55, they are non-profitable at all for the companies, and they're hugely subsidized 55 and plus and 60 and plus. And this is because these customers, they tend to leave more online. They compare a lot, they browse and they're very interested on what's going on with the products, and they look a lot into price comparison websites. And if you're comparing on prices, I mean, that puts pressure on your prices, so that's a top line problem. But also this generation is more financially literate. So as they're comparing, they learn a lot of the products that they're buying and they are more aware of what they are buying.

(00:06:17): And since you're more aware of what you're buying insurance, you tend to use the product more, which companies don't really like. Let's be frank about that. So you have also a bottom line problem since your profitability is worse, your cost of claims is higher than the different generations. So you have a top line problem and a bottom line problem means that basically you have a nonprofitable segment. And why would you invest in a nonprofitable segment if you just want the very hugely profitable 60 plus year olds of that... I don't know, 65% of your margin in cases of some insurance companies. We saw that problem and we thought like, "Oh, there's something that we can do here." Because when you look at insurance as a general, what you see is that the companies, they just put into market like a very universal product and it's like, "Oh, this is the product that you have. You buy if you want, and if you don't, you don't."

(00:07:03): But then they don't look at segments in a way of, "Oh, how can we particularize our product, our value proposition to this segment?" So if we can make these customers go for a self-service digital first platform and then we deploy all these cool technologies that are coming into market, then maybe you can make it more of a product that they like more and also a profitable product. And yeah, that's how we started. We are focused on personal lines, we started with homeowners insurance and then we moved into Term Life, which is not really personal line, but yeah, it's very close to homeowners because of the mortgages and everything. And then we released Pet Health and now we're about to release Auto, which is the other large line. And then by mid-year we'll have travel as well.

Rob Collie (00:07:54): We really love stuff like this. The idea of going into a corner of the market that the traditional players haven't succeeded with and making it more efficient, making it a better product for the customer base, we're vibing with that. We feel that here at P3, even from the AI and data consulting perspective. So when did you found Tuio?

Juan Garcia (00:08:15): So we started at 1st of January 2021, funny enough.

Rob Collie (00:08:19): So pre-ChatGPT, right?

Juan Garcia (00:08:21): Pre-ChatGPT. We weren't born as a native AI company. We developed into a native AI company.

Rob Collie (00:08:26): I love it. The business need and the ambition to be a better product and to be more efficient pre-existed AI. You're not one of these like, "Ooh, let's go slap AI on insurance." You're not just like this wash, rinse, repeat formulas. You had a strategy to begin with, and then AI fell into your lap that you could take advantage of, right? So I think that's really, really cool.

Juan Garcia (00:08:44): Yeah. We started thinking about how we can leverage existing technologies to just do better for this customer segment. And then gen AI came and [inaudible 00:08:52] our strategy. And even before that, we were thinking about AI as a technology. Asís used to have this joke that he would say that what's the difference between machine learning and AI is that machine learning lives in Python and AI lives in PowerPoint. It is what it is, right? But yeah, we started from the very beginning thinking that this type of technologies would help us, but there wasn't anything like gen AI at that point. And that definitely is making our lives easier.

Rob Collie (00:09:18): So when ChatGPT dropped in late 2022, how long did it take before you all were going, "Oh, there might be something we could do here." It takes a while to set in. It might've been the first day, but for most people it isn't. How long until you started your first even whiteboard session of, let's see what AI can do for us operationally or as part of our product offering? How long of a lag was there?

Juan Garcia (00:09:47): I would say we were 2023, we're a smaller company. For us, this is not a matter of streamlining operations and reducing costs. This is like how we can scale without cost exploding? Because if I can control my cost base, then obviously I can grow faster without having to get more people for operations and call center agents and all these claim adjusters and all that. So we started pretty early by Spanish standards, I would say. Probably half '23. It wasn't even 4. It was 3.5, I think, the first that we started building with. And since I said, we pushed pretty massively for details of service. We had a lot of interactions through WhatsApp. So we felt like, "Oh, this is clearly the case that this thing called ChatGPT is built for." I mean, if he's speaking with me through text, why he cannot speak with customers through text?

(00:10:38): But if you remember, I'm sure you do, and most of your listener will as well, 3.5 had a distinct problem with hallucinations. He could say whatever he wanted at any time and with complete disregard for the truth. So we built this machine, this agent, we call it Lea. You will see it through the program that we have away with names. We named every agent that we built and they're all funny names.

Rob Collie (00:11:02): As you should. We salute.

Justin Mannhardt (00:11:04): We could have a whole tangent about naming agents, Juan.

Juan Garcia (00:11:08): So that first one was Lea, which is kind of like Lea, but the Princess Lea, because tech teams are always geeks, and if they're not, then you should fire them. So we call them Lea because IA, as opposed to AI, IA in Spanish is Inteligencia Artificial, so it's Lea. It was a chatbot, but it was built on top of ChatGPT at that point, and it had a huge issue that hallucinated nonstop. So we had this thing and we didn't really built one agent. We built a multitude of agents, a little experts, that they knew one thing and one thing only. Each one of those agents, we can control what he said and what he could not say. We are now at Lea 3.0, so we've re-engineered that a few times already. The models advance so fast that the things that you had to do this way in two years ago, they're just like... I mean, you're going to get rid of it.

(00:12:00): So I remember it took us, I think two months and a half to build that Lea 1.0. Maybe I'm over exaggerating, but we probably had a tree of about 75 to a hundred agents because each one of them, one knew about water coverage, the other one knew about the fire coverage, the other one knew civil liabilities because we have to build it very, very in a way stupid. So if it only knew about one thing and that thing only, because if not, we couldn't control hallucinations at that point.

Rob Collie (00:12:29): Let's zoom in on that for a moment. Even though the models have gotten better, this problem hasn't gone away. It's just become, you don't hit the problem as quickly, but you can still hit the problem, right? You were saying you've had like 75 plus subagents in Lea 1.0.

Juan Garcia (00:12:48): For one product.

Rob Collie (00:12:49): It wasn't 75 different chatbots. The user didn't have to go to a different-

Juan Garcia (00:12:52): No.

Rob Collie (00:12:53): ... agent to ask like, "Oh, I need to go ask the agent that knows the answer to this question."

Juan Garcia (00:12:57): Yeah, we had this orchestrator that was actually the one that got the input from the customer. Then did one send it to the actual agent that knew about that. And then we reviewed that question. There was another agent that reviewed that answer. And if there were many questions, there was another agent that would actually compose one single text message and then the orchestrator would go back to the customer. It was a pretty complex process, I mean, it helped us a lot to understand the technology and how important it could be. So two months and a half to build that first iteration, when we built the second iteration like September last year, it took us three weeks. The models are so much, it's impotent. I mean, you have vectoral databases and you have all these new technologies that came with it, and you have memory now. And at some point, Lea 1.0 was about prompt engineering.

(00:13:48): Then Lea 2.0 was context engineering. And now we're at 3.0 and it's a completely different... We didn't even have a 3.0 anymore. It's just like the speed, the technology advances, it's just mind blown.

Rob Collie (00:13:58): So would you say that Lea 3.0 is, from an engineering standpoint, is significantly simpler than Lea 1.0?

Juan Garcia (00:14:04): Yeah, yeah, yeah. Even 2.0 was way simpler. And now [inaudible 00:14:09] it's like, it does so much more stuff now with a way simpler architecture. Yeah. And it's two years, it's not a lifetime away.

Justin Mannhardt (00:14:17): As a leader in your business, when you look at some of the challenges with AI we've seen over the last few years, hallucinations being one of them, there's tremendous incentive for these problems to get solved. And we see them getting solved, like you were saying, better models, better tools, better extensibility, but it gets better so fast. So as a product leader, when you see the advancements, how do you think about managing your customer experience when things like this keeping up activity that's always ongoing, how have you guys navigated that? I'd be curious.

Juan Garcia (00:14:52): There is this thing that technology teams, even product teams that you need to be aware of for the sake of building, it doesn't really improve customer experience. I mean, you need to be very aware of why you're building, what you're building. And sometimes it's just not customer experience, it's just like easier way of maintaining it. For example, 3.0 is not about customer experience in our case, which... I mean, there are improvements in customer experience. It's just that, it's going to be so much easier to maintain at 2.0. And 2.0 was already way easier to maintain at 1.0. Because 1.0 for the product teams, it was a nightmare because every time you touched whatever in the coverage in your policy, then you needed to go to Lea and just change it manually in every agent that was talking about that. So you need to avoid the excess of building because it's possible now and you just need to be very careful with why you're building what you're building.

(00:15:41): And in our case, the constraints are also the size of the team. I mean, if we had limited resources, then we would build whatever probably. But since we are a startup and we have limited resources, we need to be very thoughtful of what're we building and why're we building. And since we've been releasing new products, then sometimes you just need to... These development resources are go to new products, and sometimes just go to something different.

Rob Collie (00:16:02): So where was Lea 1.0 used? Where would a customer have encountered Lea 1.0? I think it's really wise that you are judicious about where you expose these sorts of technologies to the user. If you're a company that said, "Ooh, look, ChatGPT, we should go do something for, I don't know, pick an industry, throw a dart at the map. Let's go do AI for insurance." That sort of tech thinking company would try to apply it everywhere and just absolutely face-plant. The customer needs to think it's better. They need to like it more than they liked whatever the other process would've been. They have to like it more to engage with it. I love that you've been applying that discipline. I didn't quite catch though in the early going where Lea would've been deployed in a successful way.

Juan Garcia (00:16:49): At the time we deployed Lea 1.0, we realized, and this is only because you are operating in the market, that 70 to 80% of the questions the customer had were about coverages and do you cover this? Do you cover that? Where are the limits? This is very repetitive, very informational type of questions that they had. I mean, that's what ChatGPT 3.5 was done for. I mean, you can ask him things and they would dispute whatever is on its mind. So yeah, at the very beginning, Lea, what we figured is that if we can build this to be aware of our products, then most of these questions, they just go away. Our human agents just help people with purchasing a policy of maybe you need to do this or maybe you need to do that. And it's what I always say with automation technologies, we automate the low hanging fruit and then people can dedicate themselves to just more value add tasks.

(00:17:40): And in our case, Lea 1.0 was exactly that. We automated coverage related and process related and app related questions, and everything else would go to our agents.

Justin Mannhardt (00:17:51): The lens of empathy, I think, is a really good idea for people regardless of the industry building these type of solutions because that's such a hard thing to control in an agentic system, whether through system instructions or a knowledge base. And especially in insurance, I've had to make claims because of an auto accident or burglary, and you want someone that can give you the human, it's going to be okay, not I'm going to pass your information along or misquote what can be done. And I've had experiences with other chatbots where it's just so clear there's no empathy occurring and you just don't want to use it.

Rob Collie (00:18:35): It's wild, isn't it? When you're talking to an employee of a company in a call center, you already know as a customer that the person on the other end of the line probably does this all day long and can't afford to really care that much about you. The human being on the other end of the line, how well you feel taken care of and how well you feel empathized with is just due to their personality and/or how well they embrace the acting component of their job, right? It's such a fine line between when is a human required. When do we absolutely need to talk to a human being versus like one of my own personal experiences with AI that just I did not anticipate was this couple's coach that I built for my wife and I to help improve our relationship.

(00:19:20): And wow, Sonnet 4.5 is actually warm. It checks all the boxes. You can talk to a professional therapist, couple's therapist or whatever, but they're being paid to do it. They're doing this all day long. Of course, they're empathetic to an extent, but there have been times where we're just almost like in tears feeling seen and not in a way that was sucking up to us, which is one of the other problems with AI, right? It also checks me when I'm trying to get away with something, it's like, "Rob, I think you're cheating a little bit there. You need to hold yourself responsible, but you got to get that right."

Juan Garcia (00:19:56): Yeah. And there's sort of a few ways of looking at it, and we've looked at it from different perspectives. And obviously, at the beginning we looked at it one way and we now look at it a bit different. So the first thing, and just to close on the Lea 1.0 chapter, one of the most surprising things that we had with Lea 1.0, and obviously consistency in the answer was one, like you had absolute consistency on the answers that you were getting once we got rid of hallucinations, obviously. 100% consistency is one of the main advantages of AI in general. And we'll talk about that also with Watson, which is another of the agents that we've deployed. Second one is that I think it's just way easier to deliver an empathetic experience on text and voice, and that's how you started through Watson. The NPS of the chats that were full on machine versus the NPS of the chats that were a person was 15 points higher-

Justin Mannhardt (00:20:48): Wow.

Juan Garcia (00:20:49): ... than the NPS that we had on human agents. And obviously there's a caveat there, a consistently higher NPS, around 15 to 20 points. But now it was true also that whenever it had to switch to a human agent, there were usually trickier problems. So that's partially explained by the trickier problem. But when you looked at the first month versus the last month, you could see an improvement in NPS straight away just because of consistency and because of the way the machine was talking. And also, let's face it, sometimes just being timely is just everything. Doesn't really matter what the answer is. It just matters that you answered in 10 seconds as supposed to be waiting for like 10 minutes for an answer. And that was part of it as well. So consistency and timeliness was something that we learned, helped us a lot in Lea 1.0.

(00:21:35): And empathy is a tough one. Like I said before, it's just way easier to deliver an empathetic experience over text and over voice. We started text so Lea in 2023, as I said, half 2023. We only started voice half 2025 because the technology wasn't there. It was way, way trickier. Even when we started, if I look at Sonya, Sonya is the name of the voice agent, which is a Spanish name, but also ends up by IA, which is one of the main motivators of that main name.

Rob Collie (00:22:04): We're moving to Spain. The namings of things are just wide open over there. You're not as camped in URLs as the English space is.

Juan Garcia (00:22:13): We have this investor on our company that says that if the first version of whatever you release doesn't embarrass you in six months time, then you just release too late. And that's happened with Sonia. I look at it now and it's pretty embarrassing now, but the technology was what it was. I mean, the voice LLM wasn't that good. The LLM that actually did the reasoning, it wasn't that as fast. So it was, I wouldn't say a bad experience, but it wasn't as good as a text experience, but just because of the trickiness of the technology and the medium. But if you look at LiveKit now, it works amazingly well. If you look at the new agent platform that... ElevenLabs.

Justin Mannhardt (00:22:50): ElevenLabs?

Juan Garcia (00:22:51): Yeah. The agent platform of ElevenLabs now is pretty amazing as well. We are gearing towards deploying LiveKit now because it's just way more flexible. We think about LiveKit as the [inaudible 00:23:07] of voice. So it's basically it's open source. It's this framework that you can do all sort of cool things. You can plug the recent LLM, you can plug all the voice LLMs. So for our needs, it's probably going to be better. And being open source and being able to deploy in our own infrastructure always helps. But we're using right now, ElevenLabs and we started using them, but the technology wasn't there six months ago. But now if you look at the agent platform that they've released, it's just amazing. We just happen to have a different use case which gears out towards LiveKit.

Justin Mannhardt (00:23:37): This creates the opportunity for product leaders to just rethink how they go about doing what they do. Just to put a period on it, like you said, focusing on the value, the experience for the customer, the business outcome you're trying to affect is way more important than keeping up with... What we would say here in the States, keeping up with the Joneses, right?

Juan Garcia (00:23:57): It puts pressure on you though, because you have to be very aware of whatever is happening in the market and what's the new releases and how they can impact you, and if they impact you. And if you want to develop on that impact, then nothing is as static as it used to be anymore in a way.

Rob Collie (00:24:16): Circling back. So think about that question, how much more effective it is today anyway, delivering an empathetic experience for the user using AI-generated text than AI-generated voice? One of my theories about that, I hadn't thought about that until you said it, is that in the text case, you can imagine the other person's voice. It's a way that we interact with real human beings that is already artificial. A text interface talking to another human being is already somewhat weird and artificial. It's not face-to-face, it's not voice, it's not... Right? But when you bring voice into it, now you're inventing a whole living thing that we know doesn't exist. I mean, it's silly because in both cases, in reality, whatever you're talking to, it's not a human being. In terms of suspended disbelief, I'm willing as a human being to buy in subconsciously to the idea that I'm having an authentic interaction over text and the agent's got a name and I'm like, "Oh, hi. Hi, Griff. Nice to see you."

(00:25:15): I still say please to Claude Code. I just never bothered to take that out of my approach. But yeah, when you start to hear this AI generated voice, it really forces you to confront. This thing's really trying to act like a human. Now it's feeling, yuck. I don't know if we're ever going to get over that.

Juan Garcia (00:25:35): But some of the voices that now you hear, especially in English... Spanish, we're a little bit behind, but especially in English, some of the voices are just amazing. Let's say, that the face-to-face interaction is the core of the human experience in a way, the human experience communication. Then if you call someone through the phone and you get rid of all the visual cues and everything else, but still there's something in the voice that's very human. And if you filter that further and you go through text, you have the text, the communication, but you just remove everything else. So mimicking the human experience is what I think was what's difficult, not the communication part. So that's why I think-

Rob Collie (00:26:08): I agree.

Juan Garcia (00:26:09): I think mimicking that is the difficult part. And technology, only recently seriously technology in voice as near as the human experience as text was, obviously, because... I mean, there's further filtering in text than invoice. And one of the things that we were worried about that when you looked at our trust pilot, insurance space and trust, and trust is in the digital world is based on reviews and we worked very thoroughly on our reviews. And when you start filtering for Lea, you see a lot of people thanking Lea for itself. They haven't even realized that it wasn't a human. And that's probably the best thing that you can say about Lea.

Rob Collie (00:26:46): One of my wife's user complaints of our couples coach is that when you start to talk to the coach, it just is going to keep asking you questions. You ask a question, you talk and you go back and forth, back and forth, and it's constantly "Interested in what's next." So it keeps asking questions. She's like, "I just want to be able to walk away, but it feels rude. Like someone just asked me a question. It'd be rude to just walk away from this thing." It is really, really neat watching human psychology play out. She literally continues the conversation five or 10 minutes longer than she wants to. She's been done, right? It's like the person who's in your office and won't leave.

Juan Garcia (00:27:24): Yeah, human psychology, and you could see that play as well. I think nowadays less in the new models, but in 3.5, I remember very distantly that one of the ways we managed to reduce hallucinations, if you keep your reply to whatever we've given you as context, then we give you $100 and that one worked magically. I don't remember the percentage point, but I remember it was something like half. And then with further context, then we managed to get to the point that we were very happy with the way it was answering. But just that probably means that whenever Skynet rises, we are all going to be dead because we all use that trick. But yeah, that's basically our cognitive bias plugged into the model.

Rob Collie (00:28:10): I'm here for my $100.

Juan Garcia (00:28:12): Right, right.

Justin Mannhardt (00:28:14): There was even a study in that era, like the '22 era of the difference between bribing ChatGPT with $20 versus $20,000 is actually more successful just to offer it like $20.

Juan Garcia (00:28:27): Yeah, I remember that. I remember that. Yeah.

Rob Collie (00:28:30): $20,000, now you're just bullshitting me. We're bullshitting you no matter what, but you're never going to pay up.

Justin Mannhardt (00:28:36): Juan, I'm curious, it sounds like you guys have worked on a lot of things, and you mentioned earlier in the conversation the distinction between looking at AI as a way to control cost and then looking at AI as a way to create a better experience or more incremental value for the customer. How were you thinking about those things today? Maybe what are some of the other things you guys are doing in the business?

Juan Garcia (00:29:00): That was the big shift that happened 2024. Mostly one of my co-founders of Josemaria was thinking around our strategy and what we were doing at that time. And probably being in insurance helped us with this insight, because in insurance just cost to serve. It's just about 10% of your cost base. It's just that, just 10%. And when you look at marketing and you look at the actual claim costs, it's about 85% of your cost base. So if you're working in something that even if you manage to improve by 50%, which probably you want, because I mean, insurance operations are pretty streamlined as it is. Even if you get that 50%, you're only going to get five more percentage points on profit margin. But if you work on 85% of your cost base, if you only get 10%, which is easily attainable with this technology, you're getting 8.5 and you're probably looking at more like 20, 25% improvement, and then that's 15 percentage points or more.

(00:29:54): And to him, who was the one that came with this realization to us. I mean, I don't know if we're going to get anyone where with this. To me, it makes sense that we put our minds onto this 85% rather than this 10% because it's just a lot of numbers. That's the way it is. So we switched it a little bit and then we started looking at marketing and cost of clearance as the biggest needle movers that we needed to figure out with AI. And we figured that we had three levers that we will be investing in, which is growing efficiently, this is marketing. Underwriting is smarter, which is the way you price. But it's not only pricing, it's also all the information that you have to discern risk of a customer. I think we are pretty more granular than competitors. And then managing claims more effectively.

(00:30:39): Focusing on those three things are what led us towards probably saying no to all the different things that we could do with AI. We are just focusing on these three things because these three things are the most impactful on our business. It wasn't that we were more intelligent than anybody else. It was just like the industry that we were in was focusing us on those three things. I would say, if I may, that we were pretty successful. And I'm here to tell you today.

Rob Collie (00:31:02): Number of the things that you mentioned sort of really bring back another key theme, which is there's a very deliberate blurring of the lines between data-driven success and AI success. You were just talking about the granularity of pricing based on risk and all that kind of stuff. I would imagine that some reasonable percentage of that is executed with sort of "Just good data, sort of traditional non-AI data-driven stuff." Maybe there's an occasional LLM usage somewhere in there. It's not a sign of being primitive. If you're solving certain problems without using AI, that doesn't mean that you're missing out on the modern thing, right? It doesn't mean that you're primitive. You're just choosing the right tool at the right time, but the line between the two blurs, right? You mentioned earlier context engineering and that becomes all about data and everything. So where is generative AI currently being used?

(00:31:57): I know behind the scenes you can use it to help write code and all that stuff, but in terms of the runtime operation of your business, is generative AI being used in places that go beyond customer facing chatbots?

Juan Garcia (00:32:09): For sure. So of those three examples, both the one in marketing and the one in claims are still using gen AI as the core of the operation. In a way, we haven't built on AI into an existing insurance operation because we think that only gets you so far. So one of the things we're doing is, and this probably will be bought by me and my partner, Josemaría.

Justin Mannhardt (00:32:33): He'll correct you, yeah.

Juan Garcia (00:32:34): He'll correct me, yeah. But there's this one slide that when I saw it and he showed it to me, it's like that's a great slide. It's just a framework on how we see the insurance operation. And at the very bottom, the unified data layer. Then on top of that, you build your processes. And then on top of that, you build workflows and APIs and screens as apps. And then on top of that, you have your human agents and your AI agents. So to us, it's just like a completely unified framework. It doesn't really matter who's operating on that data, on those processes, on those workflows and how we access them. Can be a gen AI agent or can be a human agent. We have to deploy that technology to ensure that whoever is accessing that data and that process can do their work correctly.

(00:33:21): And that's the way we switched it and we redesign our company. And because we were obviously, at one point, we started before gen AI and we were doing things as they were supposed to be done, right? But once we figure out that gen AI could do way more things than we thought it could be possible at the time, we had to redefine the way we think about our business. I don't think studies that say that, "Oh, only 5% of companies see successful pilots from AI." And it's just like, yeah, because piloting AI to existing processes and data and workflows, it doesn't really work. And there's a lot of human resistance to deploying AI, obviously, because we all fear that they're going to never get rid of us. Unless you rethink the way you do your business, you won't see the results that you should be seeing. Because what we see after deploying everything, we are running our business...

(00:34:10): And I think this is neat. We've moved away from operational KPIs improvement from these end projects. We've re-engineered the whole thing and now we see... We run our business, and homeowners is the best one that I can give you an example of because we've been doing it for the longer. The average in Spain is 5% profit margin. We are running the business at 15% profit margin, and we hope by the end of the year we'll be running it at 18%. So we have proved that if you re-engineer your company with AI at the core, that's how you became native AI, you're not born native AI, and that's how you had to do it. Then you can have outstanding results and way over the average.

Justin Mannhardt (00:34:51): This is a theme I think is worthy of a bold button that a lot of leaders, when they're imagining the future of their company with AI, they imagine what is happening today with AI sprinkled around and people are using AI. And the truth is the organizational chart will look different, the systems and the processes will look different. And you may be benefited from being founded in '21. And then there's a lot of inertia, I think, against some of these transformations in places. And so maybe it's easier just to say, "Oh, well, let's figure out how to get Juan using a chat in his work or let's figure out these different things."

Juan Garcia (00:35:32): Yeah, there's a lot of inertia and there's also a lot of personal risk. If you are whatever director of a publicly traded company and you just go to the CEO and to the market and to your board and say, "Look, we have to re-engineer all our process." We have to invest all this amount of money in AI. And you have no proof because at this point, there's not a lot of data points of success implementation of what we're doing in the market. And us, because it's our business and we have to do it because unless we present a cool story, then we're not going to be invested in. But as I say, you have a cushy job in a publicly traded company, just running the course, you have a brand, you have an agent network at least insurance, and you're selling this amount of policies. Why would you take that risk? I mean, it's just counterintuitive to just keep earning your salary, right?

Rob Collie (00:36:21): Those directors, the board of directors, if you think about it, they have two big buttons they can push. One of them is the one you just said. Yeah, you need to retool everything and blah, blah, blah, blah, blah, right? And that's terrifying. It's terrifying to the director as well. No one wants anything to do with that cryptonite button. But then the other button, which is really easy to press is, "Hey, you all at that company, you need to be doing AI and you need to be doing it now. Come on, figure it out." And that's the button that's just being... I mean, it's worn out. The label isn't even visible on that button anymore. It's been pressed so many times. That one is not helpful either. It just creates a lot of pressure without a lot of answers. Now, as you say, eventually the world's going to start figuring out answers and the business world is a copycat universe.

(00:37:13): It's waiting to latch onto success stories and there just aren't enough of them that people can trust yet. Very, very, very early in this conversation when you were talking about Lea 1.0 and the 75 subagent version of Lea 1.0 that you got working, I wrote down the 95% number on my notes here to bring back up so that you just brought back up. I even encountered that number. The 95% of gen AI prototypes fail factoid in an article yesterday in response to the Nvidia CEO's comments and remarks at Davos. Let's say we believe that. Let's take it at face value. Let's say that 95% number is still what's happening today and 19 out of 20 projects are just face planting. And stack that up against your story of ChatGPT 3.5, Lea 1.0, getting something working in the face of the technology like trying really hard to not be ready.

(00:38:16): You go on LinkedIn and you're only going to encounter one of two messages. You're going to encounter the, "I can't believe how far behind you all are, bro. You're not cool like me. I've already transcended humanity and I'm so far ahead of you, and I feel bad for you and you all should click through to my content and click like and subscribe," right? So that's one message. The other message is this is all a fad, it's a farce, it's a sham, it's a bubble, it's all of that. But the difference between being in the 95% or the 5% is a matter of making good decisions. And those decisions can be made, and they're not rocket science. I mean, we're bright people sitting here talking, right? But we're not Einstein, we don't need to be. The most important things that we're talking about in this conversation are in very plain language.

(00:39:06): There are things that sound obvious when you hear them. You hear them and you go, "Well, of course." But you're like, "Okay, not of course." That turns out to be the thing. You've got to do this, this, and this. There is a formula, there's a recipe for this stuff. We're in this really wild and exciting era of the world doesn't have anything to copy yet. There's no simple sort of "Dumb way" to implement this stuff. You've got to be thoughtful. And that's why for us, it's a really exciting time because we're pretty thoughtful about this stuff. And we're not the ones on LinkedIn breathlessly saying, "I can't believe you're so far behind." And we're also not the ones out there saying sham. Guess what the algorithm likes? The social algorithm loves the controversial takes, the ones that make you afraid or make you feel smug that you haven't done anything.

(00:39:59): Those are the buttons to press on professional social media in particular, and neither one of them serve the species. They're not helping us. So it's a long way of saying congratulations, right? We're still seeing this 95% number and you all were puzzling your way through it in a disciplined fashion and getting somewhere two plus years ago. Hat's off.

Justin Mannhardt (00:40:22): Maybe a corner turn here. You've said similar things explaining your story. You look at any business on the planet, doesn't matter what they do. That business is going to have some way of marketing itself, some way of selling, some way of delivery and servicing people. You look at all these work streams and there's absolutely places that AI can help in every single one of them, but you can't just do everything even though you can be very ambitious with AI. And even more so what's really important is, well, what's the likelihood that the thing we build with AI actually gets used and actually makes a difference in our... Like you were saying, in our profitability or whatever. So you've had a great experience over the last few years. If you were to try and simplify your mindset for other leaders, how would you describe that?

Juan Garcia (00:41:11): I think gen AI can help you at anything. I think it can help you at any point in your business. One of the things I still see, and getting the segue of a 95% that's still failing, I think there's two reasons why 95% is still failing. One is because the main reason why they are building pilots and building projects on gen AI is because of that pattern that the board is pushing like you do anything. It's basically tell Accenture, "Come over and do us and do something." It doesn't matter. But one of the things that I think gen AI can help you with, I have a particular view of outsourcing and technology consulting. I think it's going to be very tough for these companies because to me, gen AI is something very... When you look at it horizontally as big LLM, they can do a lot of things, but if you want something to work for your business, you have to particularize it very well for your business.

(00:41:59): So I think the team that does it has to be your team. The business people that work with them has to be your business people. And also, if the new technology companies, you have product and technology working together to deliver, I don't know, an app or whatever, then in gen AI, it's even more necessary for those teams to be mixed to have the technology people that are working with them and you have the business people that actually are figuring what you can do with that technology. Because if not, I think the days of you just saying, "Do this and bring me the results," are just surely done. And that won't work for gen AI at all.

Rob Collie (00:42:33): Yeah. We 100% agree with the spirit of what you're saying. There's one modification that I'd like to make. The same thing you just said about gen AI is really true of any project, of any IT type project that you would implement, no matter what it was in the past. And yet we still had a very robust consulting and outsourcing culture. And when Power BI came along, when I originally founded this company, I originally thought that we were going to be really just a training company. I thought, "Hey, I, Rob Collie, learned how to use Power BI and got really good at it." That means everyone will. It was sort of like this self-deprecating view of the world. And some point in our history that Kellen, our COO, had to take me inside and say, "Hey, Rob, you know that more than half our revenue is coming from consulting and implementation now and not just training?" And I'm like, "No."

(00:43:32): I had to be updated on it because it turned out that what people really want is the problem solved. They don't really want to learn how to build it. Some people do, and those people are great. Those are the ones that we were educating, but what really matters is close to the business, like really, really close to the business. And so our approach to data and BI projects has always been far, far, far closer to the business than any of our competitors can ever get to. And it's a hard business model.

(00:44:02): Like in the same way that you talked about that demographic of customers that it's hard to make a profit on. It is hard to make a profit using our business model of being really close to the business and moving as fast as we can because you can't milk the client, you can't milk the projects like consulting organizations do. And one of the ways that they milk it is by being slow, by being inefficient, by getting it wrong. It's almost like deliberately built into the business model. That's not a bug, that's a feature in their business model. And I have just put the words down in the book that is still only like 20% written where I said that, I think this principle has always been true. The closer to the business, the better, but it's even more true for gen AI projects.

Juan Garcia (00:44:48): That's true, that's true. It's probably better phrase.

Rob Collie (00:44:49): It's just that we can't expect us to be super, super common. Us sitting here having this conversation and you've got a small team that you work with that's involved in all of this, right? You can't expect that exceptional understanding to exist everywhere. If I did expect that, I'd be exiting this business today. We would be like, "Okay, that's it. It's over. Last one out, turn off the lights." But from my past experience, and especially for something as groundbreaking and is completely unknown and there's like no template to follow as AI, I think there's a very, very, very rich future numbered at least in years.

(00:45:27): I like our chances, like our company, but I do agree with you. We see it, right? The huge big four accounting firm projects, those were a disaster in BI and they're even bigger disaster in AI. It's amazing how much money's being lit on fire, but at least they can go back to the board and say, "I'm doing something." If I wasn't doing something, then I've just completely disobeyed a direct order and I'm going to be fired. But if I go light a couple million dollars on fire with the big four, I'm just like everybody else.

Juan Garcia (00:45:58): Yeah, it's probably a better way of phrasing it. Yeah, what I wanted to say is the typical Accentures of the world is probably going to change massively. Because I don't think that, as you say, being far away from the business, it's just one work in gen AI. Probably what I definitely see happening, something that are more boutique companies that don't have the same overhead these companies and they can stay longer doing things, not just putting the entry level employee, just customer facing.

Justin Mannhardt (00:46:26): Right, the pyramid.

Juan Garcia (00:46:27): I think the model of the Palantirs of the world, or even OpenAI, they're starting to do that, get implants within the customer and start working with them very closely and just basically dedicated person for that customer. The thing with that, as you just said, this is a very tough, but it doesn't scale as VCs would want it, which is a complete different segue of conversation, but it's definitely not a VC model. I see some of the questioning of the current bubble, if I may, that's actually relevant. We think that we're going to see the massive deployment of the ChatGPTs of the world for B2B, which is what is going to really justify the valuations that we're starting to see. I mean, just the consumer productization of ChatGPT is not going to justify the valuation that these companies have. Only the deployment in the B2B world will justify it.

(00:47:15): And if that, what's the model to deploy it? And I think it's more companies going very close to the customer rather than the Accenture of the world. But what scales very well is the Accentures of the world, because being the smaller and having senior people with the customer, I mean, that costs money and sometimes money doesn't scale as well.

Rob Collie (00:47:33): I've watched the last, I don't know, five years, this tremendous wave of consolidation in the IT professional services industry all around the idea of offshoring. I talked to a founder and CEO of a consulting firm in Indianapolis where I used to live, a bigger company than ours, and completely wired around the traditional model that we want nothing to do with. And he was merging with another bigger company explicitly so that he could get access to offshore resources. So the industry still has this tremendous inertia towards getting farther from the customer. Drive down hourly cost, probably don't pass that on to the customer, just juice margins with it, of course. But drive down the cost, but get farther from the customer. Industries just don't turn on a dime. It seems like that's still continuing at breakneck pace, even as we speak while we're sitting around people like the three of us going like, "Wait, what? Like what? You're driving in the opposite direction. You're going towards the volcano. Don't do that."

Juan Garcia (00:48:40): I agree, I agree. I've always disliked that model. When I started in consulting after my years in technology, there was the huge movement for outsourcing call centers and breaking down value chains. I don't know, but I had this hypothesis that that didn't work. And then you saw the pendulum comes back and then there's this insuring trend as well a few years ago. I guess, it depends on the years whenever you came to age in a way in the business world. But I always just like that giving services to a customer from very far away. I tend to think that it didn't work as well for the customer, the one that's paying really. So yeah.

Rob Collie (00:49:21): I had two random questions. One of them is that slide that you mentioned earlier about the different layers and how to approach your business, would you consider that slide proprietary or is that something that you'd be willing to share either with us just for our own edification or would you be willing to share it with our listeners?

Juan Garcia (00:49:38): I don't know why we wouldn't.

Rob Collie (00:49:40): And the other question I had was, because my ears really perked up at this. You mentioned that gen AI has been something that you're using in marketing. Can you expand on that a little bit? I'm very interested in this.

Juan Garcia (00:49:49): Marketing came later. So we learned first about claims and then we learned about how to deploy that in marketing. I can start by claims because it explains a little bit how we think at gen AI and what we think gen AI is really good at. A claim doesn't really follow a straight line. What previous automation technologies were really good is just building trees and just if A, then B, then C. But claims just don't follow that path. They branch depending on the different coverage as different trees, the severity of the claim. Obviously you have a fire, but it's done. It's severe, but it's not as urgent. But if you lost your keys, it's very urgent, but it's not as severe. Then obviously if you have fraud signals that we work a lot on, and then if you work with third parties or not third parties, and what's your expert?

(00:50:38): Because a given claim, the same claim, look through the lens of two different claim adjusters, probably going to have two completely different outcomes. So what gen AI does very well is actually picks up a bunch of information, treats it, and gives conclusions. With this insight, we felt like, oh, so maybe we don't need to automate the whole tree. We just need to automate the decision making points. And that's why I said, we stopped chasing the absolute cost to serve efficiency path and then we start chasing the making better decisions path. Because for us now, we apply this insight to marketing. What we think gen AI is really good at is building NBA machines, is building next best action suggestions. This is what I said about rephrasing and reframing your company. Before how a claim would work? Do we get all the information from the customer? And then a claim adjuster would look at it and then, oh, I know what I'm doing because I have this expertise, and I would do this and I would plug maybe a further expert because you need to do the appraisal of the claim because you're not sure.

(00:51:42): And then maybe just send a repairman because in Spain, at least we do a lot of repairs. We don't reimburse for the cost, we repair it. We send the plumber and we send the painter and everything. So it was a very artisan type of process. And what we built is that in our case, a claim starts digitally with video and images and a declaration from the customer. And Watson, that's the NBA machine we use for claims. It enriches and brings that data, but also brings data from the customer history, the policy itself, then manuals. We built a lot of manuals, and this is where context engineering comes to. Working with our more senior adjusters, we build a manual on how you would, at every step of the claim, you would treat it. So the robot has that manual as well. And then it also brings similar claims because you would think that every claim is different, but it's not really.

(00:52:31): And then you also bring external signals like the weather partners and everything, because if you have a water damage, then obviously knowing the amount of rain that it bring, it helps as well. So, it takes everything and then runs this triple analysis. It's severity, urgency, and application. And then with all that, well, it gets all these suggested actions and the confidence level for every action. And then our claim adjuster comes in and then revise it and then it's okay or not okay.

Rob Collie (00:52:57): Human in the loop.

Juan Garcia (00:52:59): Human in the loop. Exactly. That's the way we work. And then-

Rob Collie (00:53:01): Love it.

Juan Garcia (00:53:02): ... you would never think of working that way if you just plug your AI to give superpowers to your adjuster. Now, the reality is the adjuster is another tool of the NBA machine of Watson, right? So some of the actions that it can suggest like [inaudible 00:53:17] reserves. Reserves are really something from insurance is that when you have a claim and there's this table that just like, "Oh, this is a fire claim and it's X amount of square meters. So this is going to cost us, I don't know, $50,000 whatever." But it hasn't cost that yet because you need to finish the analysis. This is the reserve. It's a prediction of what's going to cost you, right? So to do that, which doesn't have impact on the customer, it's not as critical. So that we have almost fully automated. If it has a confidence on that suggestion above 70, then it's fully automated.

(00:53:51): We don't need a human to just review that because it's just a reserve in a way, or scaling repairs. Once it is a stage that has looked into everything and has gone into the appraisal. And we more or less know what it has to do and you have to scale an external repairman, then it can do it for the adjuster if it has more than 70% confidence in that suggestion. So there are things that helps that the human doesn't have to do anymore because it would take time for him to do all that and it just does it automatically. There are things that we don't want it to do ever. So for example, payouts or rejections, and this is where empathy comes from. If you're going to reject someone in a moment of need that, I don't know, I had my house on fire and it's completely destroyed, but for whatever reason is not really covered in your policy. I mean, that's a very vulnerable moment for the customer.

(00:54:41): So even if you have to reject it because it's not covering the policy, I mean, you don't want a machine to do it. Even if you can deliver an empathetic experience, it's something that you really want a human to do and that's something you design on your business, right? And payouts and issues with payments is also as well, because money is a very delicate issue, so we don't want the machine to do anything about it automatically. Even if it's better for the customer, we want always a human to review it. So we have a human in the loop always if the decision doesn't hit a certain threshold, or there are several decisions that are designed just as human only type of decision. So it used to be data adjusters could treat 10 claims a day, now they can do 50. Just because all the gathering of the information is just done automatically for them, they can just review all the different decisions.

(00:55:28): Some of the decisions, some of the actions are taken by the machines. So it's just like a speedier process, but also a flexible process because you're not fixing the tree, you're just helping them make decisions at every stage of the process. So this was something that's very particular to insurance, the claims process, right? But once you get that generative AI is really good as an NBA machine, then... I mean, any process that's based on decisions and actions, it can be automated the same way. So that's where we took it to marketing, because in the end marketing, what is digital marketing really? You just look at your campaigns and the different ad platforms, you look at how they are performing, and then you take decisions. You can increase your expenditure or remove several campaigns that are underperforming, remove keywords that are informational keywords and they have less click-through ratio or they don't sell policies on that keyword, right?

(00:56:19): So it's just the same exact process apply to marketing as opposed to apply to claims. So once you get the insight of, "Oh, generative AI is really good at taking information, processing it, and making decisions with different levels of confidence," obviously, which used to be a tough cookie to automate with previous technologies, now it becomes very clear that you can do it.

Rob Collie (00:56:41): How close to marketing are you specifically?

Juan Garcia (00:56:44): I'm pretty close to marketing.

Rob Collie (00:56:50): Yeah, I thought so you talked about it. Yeah, I see the scars. Justin and I share them.

Juan Garcia (00:56:53): But it's true, it's true. If you look at it used to take you a lot of time. And one of the things that Google made, for example, and I'm sure your listener that are marketing inclined and you guys, you have the exact match for the keyword, you have broad match. Unless you were pretty sure that something and you usually used it to negate keywords, you only used exact match to negate keywords. I mean, it is a very tedious, very time-consuming job, so you use broad matching. But now with this machine, you can use only exact matching. And that's one of the things that we figured that we can have this absurd campaign tree and it works because Atom, which is our growth AI agent, [inaudible 00:57:37] for us.

Justin Mannhardt (00:57:37): I think the thought process here on Human in the Loop is really important for people to think about because just the idea you've thought about, okay, at what level of confidence, honestly, relative to risk of a mistake are we comfortable with? And then also thinking about when do we just not want the AI to do anything? A trap I've seen some people in when they've started trying to build some of these workflows is when the AI arrives at a decision point or a recommendation, it's always human in the loop. It's, I could do A or B, Juan, tell me which one. That happens 100% of the time. And the problem is what you're doing there is you've now created another inbox of work that you already didn't have the capacity to deal with. We've done some examples here of some of the agents we've built.

(00:58:30): It's like, okay, what percentage of the time are we comfortable at not doing it exactly how we would want it to do it? Or where do we need to invest more? And even in your Lea 1.0 example, a lot of people just gave up at the hallucination. I was like, "This thing can't do anything." But you pressed and you said, "Well, let's figure out how to solve this problem."

Juan Garcia (00:58:50): Probably working on hallucinations gave us the insight for the confidence level.

Rob Collie (00:58:55): Oh, yeah. I think that level of training of you having to deal with 3.5 at that point in your business has benefited you tremendously. It's like you can imagine sort of the grumpy old man attitude here like, "Nope, nope. You come to work here. No one gets to use Sonnet 4.5 out of the gate. You all got to go back to ChatGPT 3.5." It's like the Wall Street firms when you join as an analyst or a trader, they sit you down with a computer and they don't give you a mouse. They explicitly don't give you a mouse because you have to get good at using the keyboard to operate Excel. Touching the mouse lose all kinds of time, too slow. They burn that discipline in.

Justin Mannhardt (00:59:31): That's why I never made it as a Wall Street trader.

Rob Collie (00:59:33): Me either. And they pry keys off of their keyboards too, right? Certain keys that you hit introduces like a seven-second delay like the help system loading or whatever. Oh my God, I want nothing to do with a sweatshop application of Excel like that. You mentioned the original Lea 1.0 and the GPT 3.5. I did have one more thing to circle back there as well. So GPT 3.5, you weren't using it out of the box. GPT 3.5 didn't know anything about your offerings, anything like that. It couldn't answer any of those questions. That wasn't in GPT's pre-trained information. You were still having to, with all these sub-agents that made up Lea, you were having to teach. Like every time GPT 3.5 got called, you were having to give it information about your business. And you mentioned that that first version was about prompt engineering and then subsequent versions were about context engineering.

(01:00:28): Again, I understand I think what you mean there, but I want to make sure that we explain to our listeners what we mean. Can you explain the difference between prompt engineering and context engineering between Lea 1.0 and Lea 2.0?

Juan Garcia (01:00:40): So for example, since the amount of leeway that we would give to any of the subagents in Lea 1.0, so every one of them had everything to know in the prompt?

Rob Collie (01:00:51): In the system prompt.

Juan Garcia (01:00:52): In the system prompt. Let's say, it's an expert on water damage. You're an expert appraisal on the coverage of water damage. This is water damage and you would have the piece of the coverage, there in the prompt and you cannot do this, this, this, and that. You cannot think this, this, and that. So everything of the behavior of the subagent was there in the prompt. You didn't need to concern yourself about where is this subagent going to find all the information so you didn't have to deal with RAG databases or vectoral databases or anything like that because the context window was so limited, you couldn't give it very many information before it started to hallucinate or to forget of what it was saying. So it was just like every sub agent had everything that it needed to know in the prompt. So you just needed to work the prompt to make sure it didn't hallucinate, to make sure it had all the information about the coverage or whatever it needed to do. To know how to behave, for example, how it would give the answer.

(01:01:51): And that's something that we actually started with. But then as I said before, it had one singular subagent that would actually write down what did it say. So we just removed that part from the system prompt. It didn't need to know the way we wanted it to talk because it wasn't going to talk with the customer. It just needed to know and give the correct answer. Just made sure that it didn't hallucinate, it had the information and give it the context like, "Oh, you know about this, whatever."

Rob Collie (01:02:16): Yeah, that's what I expected. That's what I thought the answer was. And it'd be even more clear for our listeners, when you're saying prompt, this wasn't a prompt that the user of the chatbot ever saw, no one's typing this prompt into a text box. That prompt is living in your code behind the scenes. And every time your regular normal CPU, regular software code called ChatGPT's API, it would hand this same prompt every single time so that it knew what it was doing.

Juan Garcia (01:02:45): Exactly.

Rob Collie (01:02:46): And then when you transition into context engineering, you get... First of all, there's an opportunity to give it more than that, but also the opportunity to let it go grab what it needs, which was not an option. You couldn't give the 3.5 release some sort of access to a database so that it could do self-service and find the information that it thought was relevant, which you can do now. And so this gives sort of a lot more flexibility. It's still the same problem in a way, which is making sure that this LLM knows what it's talking about and has all the information it needs. But you just have a lot more degrees of freedom in it than you used to.

Juan Garcia (01:03:26): That's exactly right. So Lea 1.0 for your listeners, each one of these subagents would look like a custom GPT from ChatGPT that you can still do today, where you give it everything that it needs to know and behave on the custom whatever system from. So each one of those were like that. So imagine maintaining that on 675. So whenever you need to change something, you just need to find out what exact one and then change it. And then if there's something about something new that we realized didn't work well and it wasn't from the coverage itself, but something that we had mistakenly done in the prompt, then you have to change it with every one of them. So it's just completely different. And whenever you just have separate databases and your context window is bigger, and then you can mix and match whatever it goes into it. And then you give it tools as well, then it's just way more maintainable.

(01:04:11): It's a different problem. Before, you only need to know about prompting. Now you need to know our relational databases and vectoral databases. And you have to edit, for example, our policies. Then now you have to chunk it and then you have to mark it down. And then you have to do all these different things. And most people think, oh, it's just technical jargon, but it's not. And it's a real thing that you as a business person that also have to learn. Because most of the time the manuals and the policies, they have to be treated by the business people, which are the ones to know how you want your customer to interact with the machine.

Justin Mannhardt (01:04:43): And if you don't do that, you're going to get a vector database that is incredibly efficient, programmed by some technical wizbang, but it just gets everything wrong all the time. That's what you're going to get.

Rob Collie (01:04:57): I mean, technically solid.

Justin Mannhardt (01:04:59): Totally solid, really efficient, token use. All the ratios look great. So good.

Juan Garcia (01:05:06): That's one of those things. I'd never thought I studied telecommunications engineering, but then I moved away to the dark side to strategy. But then the Juan of 10 years ago would have never guessed that he would be speaking at a podcast about chunki, just in particular. And now it's like, oh, chunki. Yeah. And then you have Markdown and you have AML and you have JSONs and it's all fun and dandy. But then it's one of those things, one of these crazy predictions that I maybe sometimes make. I think we will all be, to a certain degree, obviously, technical in the next few years. We will turn technical again.

Rob Collie (01:05:40): I never left the technical realm, it turns out. I didn't go strategy. Maybe I should have. Maybe I should have gone strategy for a little while.

Juan Garcia (01:05:49): I know this in Wall Street and how they couldn't get the most. I was like, "Yeah, it was one of the ones with PowerPoint."

Rob Collie (01:05:54): Oh, don't talk to me about PowerPoint. I got a strong PowerPoint game one.

Justin Mannhardt (01:05:59): It's true.

Rob Collie (01:06:00): I can't actually make it look good, but I can animate the heck out of it, man. This has been awesome, by the way. Like freaking awesome. I love this.

Justin Mannhardt (01:06:09): And Juan, is there anything you wanted to share with the audience that we didn't cover?

Juan Garcia (01:06:13): We can talk about underwriting if you want, because to me, underwriting, the engine here is called Elizabeth. So one of the things with underwriting is that, as you said before, a lot of the technology that existed before is still useful. You don't have to use gen AI to do a pricing engine or to do a prediction engine. One of the things that we've done, and that was definitely assist from the very beginning. At 2021, we didn't know what gen AI was. Nobody knew, but we knew that if we're building all this tech first platform for insurance, one of the things that we will be able to gather is data. So from 2021, one of the first things that we built is this thing that we call the customer DNA. So we capture everything from the very first moment that you start navigating, even in Google and which app you click.

(01:07:01): So we get all these data points across every interaction we gather at onboarding, we navigate on our blog, or even if you're an existing customer, how you behave in your claim. And we enrich that with third party data, I don't know, social media, social login, I'm sure it gives you a lot of information. But yeah, if you compare that with the 20 data points that normal homeowners insurance gather by their brokers and the actuaries, you can provide the risk way better, right? And it's not only because of the price, because some of the things that we've seen with this is just not price based, for example, but you can use it throughout the whole value chain of the insurance. I can give you two examples. Unfortunately, if you're like me, an Apple product user, you're going to have a more expensive homeowner's insurance with us. Because one of the things that we realized that if you're navigating and purchasing your insurance with us from an Apple product, more often than not, your claims are going to be more expensive than claims with an Android user or a Windows user.

(01:08:01): And why is that? Because your devices are more expensive. So it's one of these things that now it makes complete sense and it's really like, oh yeah, of course. But then you need to have this digital first infrastructure that you actually are gathering and then proving, which is the way it works, that if you're purchasing from an Apple product and then you have a claim, you have on average, more expensive claims than other users, right? So me and some of you guys and some of your listeners, that if you ever come to Spain and you buy a homeowner's insurance policy with us, then you're going to have more.

Rob Collie (01:08:34): Bring a laptop, bring a PC.

Justin Mannhardt (01:08:36): Yeah, bring a PC just for getting your insurance, but they'll learn about it later.

Juan Garcia (01:08:40): This is really price-based, but we have another insight that we usually share. Because, I mean, they're not proprietary. And when you think about it, they make sense. And the other one is that if you have someone that's purchasing the policy and they don't read any coverage, but they just stop at one coverage and they read it very thoughtfully, more often than not, they're going to have a claim in the very first week to two weeks at the start of the policy. Then when you look at it as like, oh, maybe we don't do anything about pricing because you're not sure that this is going to happen, but you flag it. And whenever that person has that first week or second week of the outstanding policy claim, then you just realize, oh, this was one of these. Then maybe I asked for more pictures or maybe I ask him what happened with the claim. And you start to try to figure out if this is a real claim.

(01:09:27): More often than not, what happens is that they already had the claim, they just bought the insurance so they could get covered. And the funny thing about this is that you actually prove it very easily. Sometimes the picture that they send you, even in the name of the picture and the metadata are from before they purchased the policy or the video was taken before they purchased the policy. I mean, it's just not very sophisticated, but it's just you just need to look for it. And you only look for it by treating the data and that's where ML comes into place. It's not gen AI, but it's pretty neat as well.

Rob Collie (01:09:57): Now I know how to navigate the whole system.

Justin Mannhardt (01:10:02): You got to really take your fraud game to the next level with these guys.

Rob Collie (01:10:06): Whatever pictures you take, you need to subsequently screenshot them and resave them on the right date.

Juan Garcia (01:10:13): One of the things that we saw, and we just saw it today, that's really funny. We had our, probably not the first one, but the first one that was AI generated but so bad that we actually [inaudible 01:10:27]. And it was a really awful picture. I mean, it's like, look guy, you couldn't even go to Nano Banana. That's not free. It's free and it would be better. I mean, you could definitely see there was AI, and you get it through the filters and it obviously was AI.

Rob Collie (01:10:40): Was it like a picture of a car? What was it?

Juan Garcia (01:10:43): No, it was a glass. It's broken, but it was so awfully built. It's like the table where you put the glass is not the same table of the very next picture. So it wasn't very coherent as a story. You have four pictures, the tables were different and it didn't match with the living room picture that you sent us. And then the actual giveaway was, it did seem that the glass was floating over a table with this sensation with the pictures like, yeah, this is definitely AI generated. And it was funny.

Rob Collie (01:11:16): It wouldn't even occur to me that someone would do that. Of course, they would.

Juan Garcia (01:11:22): We've seen a bit of everything. We've seen some people that they will send us a picture from Google Images, but it wouldn't even be on the 10th page. It would be like the very first answer from Google Images. We've seen pictures watermarked as well.

Rob Collie (01:11:39): It's like Getty Images.

Justin Mannhardt (01:11:40): A Stock photo, yeah.

Juan Garcia (01:11:43): Having to look for the metadata is sophisticated from what we've seen.

Justin Mannhardt (01:11:47): Yeah, digital forensics. Yeah.

Juan Garcia (01:11:49): Yeah, yeah, yeah.

Justin Mannhardt (01:11:50): That's awesome.

Rob Collie (01:11:52): I mean, look, at least there isn't a tremendous reservoir of highly sophisticated adversaries out there, right?

Juan Garcia (01:11:59): Probably they're very sophisticated, you don't even realize that it is fraud.

Rob Collie (01:12:04): Yep. I just leveled up.

Juan Garcia (01:12:07): If someone gets a Nano Banana picture so good that it can actually pass it through us for a real claim, I mean, more power to them. They just deserve it. I just clap with them. I think I just clapped with them and that's it.

Justin Mannhardt (01:12:22): This is like a cool red team business idea, Rob. You could just be the guy that tries to break Juan's AI defenses with Nano Banana.

Rob Collie (01:12:33): I don't think that's the money. I don't think the money's in that. I think the money is in-

Justin Mannhardt (01:12:36): A lot of good laughs though.

Rob Collie (01:12:37): No, no. I think the real money is in sophisticated services for end users who look to defraud insurance companies. I think we just wait to take a cut, "Tell us about your problem. Okay, we got you."

Justin Mannhardt (01:12:52): Does this happen in Spain where contractors will go around neighborhoods trying to convince people they have roofing damage and they'll offer to take over the whole insurance process? That's what you're saying, you want to be the guy that convinces everybody they got hail damage.

Rob Collie (01:13:09): Yeah. What [inaudible 01:13:12] you? An insurance company, I'll get them to pay, even though it happened six months ago.

Justin Mannhardt (01:13:17): There you go, Juan, you came on the show and now Rob's your arch nemesis.

Rob Collie (01:13:20): I've got a new SaaS platform. I'll have Claude Code working on it and it'll be spun up probably somewhere. What's a good country to base this operation in that would be hard to sue? This is where I should have gone to strategy.

Juan Garcia (01:13:36): Right.

Rob Collie (01:13:37): I would've been a natural. Maybe not. We will not be pursuing this business idea. We have an opposite ethos here, it turns out.

Justin Mannhardt (01:13:44): The previous segment was for comedic purposes only.

Rob Collie (01:13:48): Well, Juan, listen, I've really, really, really enjoyed this. I appreciate you reaching out and... I mean, I'm serious about coming up with a reason/excuse to have you back. It's been a real pleasure and a heck of an enlightening conversation, so thank you so much.

Juan Garcia (01:14:02): Sure.