Semantic Models Give You (and Microsoft) the AI Upper Hand – Plus a History Lesson on Why Frontends Matter

Rob Collie (00:00): I told you I know what we're going to talk about and you don't.

Justin Mannhardt (00:03): That's right.

Rob Collie (00:04): How great is that?

Justin Mannhardt (00:05): It's pretty great, considering I haven't even had one second of mental bandwidth to consider a podcast topic this week, so I just had to be here.

Rob Collie (00:15): Yeah, that's right. Just show up. Bring yourself, right?

Justin Mannhardt (00:17): Easy for me.

Rob Collie (00:18): But before we dive into my chosen topic, let me tell you a little vignette about how much snake oil is out there. There's so much snake oil out there that I am now getting a refund for something.

Justin Mannhardt (00:31): Tell me more.

Rob Collie (00:32): In addition to all of the normal hands-on stuff that I've been doing with Copilot, the things we've been talking about with RAG, and offline corpuses, and connecting AI to your business, all that kind of stuff. The real driving force behind learning about what's really going on in this space, we're doing the things. In addition to that, it was like, hey, let's supplement that with a high-level overview of what business leaders are being told about AI. At our house, the colleagues, we're the kind of nerdy couple that'll sit down and say, "What do you want to do tonight? You want to play a board game, watch TV show, maybe take a class on AI leadership from MIT?" These are things that my wife looks at with me, and goes, "That sounds like a good couple's thing to do." And so, we signed up for this class from MIT on AI, business leadership, et cetera.

(01:18): And I'm fully expecting this class to tell me a bunch of things I already know, probably also tell me some things I disagree with, but also open my eyes to things that are being discussed. It was a no-lose proposition to spend 12 weeks of evenings going through this class. We were going to get our money's worth out of it, and it was very well constructed. It's super well put together. Three minutes into the first video and oh, no, it's so clear to me, this super, super smart professor, he's talking about the pandemic as if it was recent.

Justin Mannhardt (01:52): Oh, boy.

Rob Collie (01:53): And it's like, oh, my God, this class was built before ChatGPT dumped the world on its head. This class isn't about GenAI. This class isn't about LLMs and LRMs. And this is about what AI was about before all of that happened. And all that stuff is still relevant, but when we talk about AI today, 95 to 98% of the reason we're talking about it. 95 plus percent of the impact of AI in the world is because in between the time this class was made and today, we invented artificial brains. None of the stuff this class is going to be talking about has anything to do with companies saying, "Hey, we're going to reduce staff." The thing that's totally disrupting our view of the world is not going to be covered. And this is obvious in the first three to five minutes of watching this first video, and I'm like, holy shit.

(02:44): And you go back and you look at the curriculum, and okay, yeah, they never claimed outright that it was about GenAI. It's written in a way that you can absolutely come to the conclusion that, yeah, you're offering a $7,000 course for business leaders on AI. It would be unconscionable to offer such a course and have it not be about the things that actually matter, except there's hundreds of students. They already let it slip in one of the other intros. This is like the seventh or eighth cohort that this one facilitator has helped run. So, how many cohorts of this have they run? And it's several hundred enrollees at seven grand a pop, so this is north of 2 million in revenue and it costs them almost nothing.

Justin Mannhardt (03:27): Yeah. Just the startup cost of doing it the one time?

Rob Collie (03:31): They're paying a live coordinator who have office hours and all that kind of stuff. And she's a PhD. She's really smart. I really like her. They have some guest instructors that are coming in and doing live sessions, but really their cost to serve is near zero. They have every incentive to keep running this class financially. If you came out of this class and weren't upset about it, you've been misled. There's only two outcomes of having taken this class. You've been utterly misled about what's really going on with AI or you're angry. Those are the only two outcomes. Buyer beware.

Justin Mannhardt (04:07): Yeah. Because there are still some really good and valuable educational content out there about the things you were just describing. Neural networks, old school machine learning, if I could use that term, that if you want a deeper understanding of how we got to where we are today, that's cool stuff to know and learn. But if you feel fleeced of like, oh, I didn't mean to learn that though, what are you going to do with your refund, Rob?

Rob Collie (04:35): Yeah. We're going to put it back in the company bank account. It's still a good idea. Taking a class like this is and using my evenings to have an alternate learning channel, it's not a bad investment. But it is a bad investment to spend 12 weeks of consuming content, and doing exercises, and stuff about something that's not the thing that's dumping the world on its head. No, they should pay me to do that.

Justin Mannhardt (05:05): Looks like TV night is back on the menu.

Rob Collie (05:08): Yeah. For now. The thing I want to talk about today is a little bit under that theme of like snake oil, but not really, because this kind of snake oil is the kind that actually wins and wins for a good reason. So, I want to take us back to a point in time called 2010. And the reason I want to take us to 2010 is I think something that played out back in 2010, in that era, there's a lesson to be learned from it that is 100% applicable to what's going to happen at the intersection of AI and BI today. In a way, this is a follow on to our Copilot episode from I think two weeks ago. So, in 2010, Microsoft had Power Pivot and then Tableau had Tableau, two of the big competitors on the market. I mean, there was also ClickView and TIBCO, and there were some other players that have since faded from the marketplace.

(06:02): But let's just talk about Tableau versus Power Pivot. Power Pivot in 2010 had already basically had the guts, the core of SSAS Tabular done. The SSAS Tabular Engine, the VertiPaq Engine, the DAX Engine, all of that that underpins Power BI. I mean, it was missing a lot of things that we really rely on today. But really, in terms of like-

Justin Mannhardt (06:25): The 20% of the things that get you 80% of the way there?

Rob Collie (06:27): Yes. It was there in 2010. And that thing was a world beater. It's a world beater today. In terms of crunching, and relating, and fusing data to paint an overall picture of your business, and to answer sophisticated business questions, it already had no peer then, still has no peer now 15 years later. It's the smartest business data crunching brain ever. It still holds that title today. But it had as its front end, the way that users interacted with it, it had Excel.

Justin Mannhardt (07:01): Which we love, by the way.

Rob Collie (07:03): We love Excel.

Justin Mannhardt (07:05): Still.

Rob Collie (07:06): But it's not a dashboard tool.

Justin Mannhardt (07:07): That's correct.

Rob Collie (07:08): It's a spreadsheet. I was part of that team on Power Pivot. We tried to make it a dashboard. We tried to make the Excel canvas into something dashboardy. We added slicers, which hadn't existed. We literally added slicers to Excel, because Excel needed it to be a BI tool. And we also leveraged things like cube functions, so you could have a little cells here and there and everything. We could get really creative. But in the end it was still Excel. On the other side of the market we had Tableau.

Justin Mannhardt (07:36): Beautiful, gorgeous Tableau.

Rob Collie (07:40): I would also say multi-touch, interactive, filtery Tableau. Every chart was essentially a slicer by default. And so, the entire experience of interacting with the Tableau dashboard was first of all pretty. It was defaulted to widescreen format. It was built for 16-by-nine, whereas Excel was still kind of maybe four-by-three over 16, whatever. Anyway, it had a default web experience that was easy to connect to. In hindsight, years later, I credit Tableau with having moved the market away from accepting reporting. This traditional SQL-based, green bar style, SSRS-style reporting had been like the overwhelming majority of the amount of business data was overwhelmingly conducted via that type of technology. And it sucked.

Justin Mannhardt (08:33): Affirmative.

Rob Collie (08:34): It sucked badly. And the world needed to move to something more like interactive dashboards and Tableau did that. Microsoft didn't move perception to dashboards. Tableau did. And oh, my gosh, are we better off for it. But it turns out that that version of Tableau had no brain, no brain at all, nothing. It had nothing to even really compete with what was under the hood of Power Pivot, like SSAS Tabular. And so, it was just as smart or as dumb as the rectangle of data you could feed it via SQL. Once you had that rectangle, now it was interactive. Getting to the right rectangle became everything. And when you're actually hands-on practicing with something like Power Pivot versus Tableau, the amount of data work you're having to do all the time is just immense with something like Tableau. And the amount of data work you're having to do with something like Power Pivot was not. You built a data model that was ready for almost anything. But guess which one was kicking ass in the marketplace?

Justin Mannhardt (09:36): The pretty one.

Rob Collie (09:37): It was the pretty one. Even just to call it pretty is a little bit dismissive of it. There was absolutely something about that which was superior, fundamentally superior. Who would rather interact with an Excel spreadsheet no matter how well-constructed to be interactive? Who would rather interact with that than what we think of as a dashboard today? No one, not even me really in hindsight. And I slept on this at the time. I was so firmly entrenched in the Power Pivot camp.

Justin Mannhardt (10:09): Our brain's better.

Rob Collie (10:11): Big data brain, better than pretty interactive front end. Okay, that was wrong. I was wrong about that. And the market showed that I was wrong about that. And Tableau dashboards captured people's imagination.

Justin Mannhardt (10:23): Right. It brought them into something they could feel more natural with, I guess.

Rob Collie (10:29): Yeah. And it very crucially captured business leaders' imagination. One of the things that I talked about a lot over the years was there were these three big lies in data. And one of them was that data is going to be easy. And a finished Tableau dashboard being demonstrated to some sort of business leader told a very powerful story. And there was some dishonesty in it, but it was a very powerful story that you don't need to worry about all that data stuff anymore. Now, you just load your data and you get an experience like this, and it's like big fisher price, clicky, clicky. The business leader would fill in the rest of the story for themselves that wasn't even being told, which was, "Oh, I don't need all those data people anymore. Data's going to be easy." In a way confirming their hunches like, "I knew it."

Justin Mannhardt (11:23): "I knew it."

Rob Collie (11:24): All these years, data was easy, but it's these Excel people that have been holding me back. Even though this thing was super, super, super expensive, Tableau was so expensive, people just fell all over themselves to buy it. But what they were buying was this perception that they no longer were going to need all of that data work, that they could now be self-service essentially, the business leader themselves.

Justin Mannhardt (11:50): So true.

Rob Collie (11:52): And of course, the reality never matched that, but at that point it was too late. You've already paid for the licenses. And in fact, now you're even more dependent on the data people than you ever expected, because creating these SQL views what Tableau commits you to endlessly. But here's the thing, we should not think the market made a mistake falling for that. That's the place that today I see with different eyes, is that that interface is where the world needs to be. That dashboard interface is where the world needs to be. And whoever gets there first is going to win market share. That's going to happen every time. Now, Microsoft let Tableau run with that advantage for at least five years.

Justin Mannhardt (12:35): At least.

Rob Collie (12:36): The first early versions of Power BI desktop with its dashboardy layout didn't show up until 2015 at the earliest. Tableau was granted somewhere between a five and 10 year head start. And all it took was Microsoft to come along with something that was 80% as good as Tableau's dashboard experience and a lower price point. Power BI still didn't win over Tableau, because of the super smart brain under the hood. This is really dissatisfying to those of us in the know, right?

Justin Mannhardt (13:05): Yeah. Yeah.

Rob Collie (13:05): It won because it delivered the dashboard experience at a lower price point. That's why it eventually won. Look how many years of head start Tableau was given. Here we are again. So, we talked two weeks ago about the power of this Copilot interface to existing Power BI models, to existing Power BI reports. The same thing is going to play out here. Whichever vendor or vendors craft the most compelling end user experience around interacting with data, they are going to seize market share. That's going to happen irrespective of how good the brain underneath is.

Justin Mannhardt (13:47): Correct.

Rob Collie (13:48): We have a client who is being told by the big four consulting firms that, "Oh, no, you need to get all of your data out of those semantic models, those Power BI semantic models, those proprietary semantic models, and get it back into SQL so that we can build AI front ends over the top of it." This is awful, awful, awful advice, right?

Justin Mannhardt (14:08): Yeah. But if it's fitting the narrative perfectly.

Rob Collie (14:10): Well, yes, it fits the narrative perfectly, but it also fits the narrative of this big four accounting firm or consulting firm being able to charge millions of dollars forever to constantly try to inadequately capture what was in those semantic models, to capture it in SQL, which by the way, we invented semantic models as a species. We invented OLAP as a species to counter the weaknesses of SQL for these purposes. So, this is the wrong direction. Unless you're building massive amounts of money to move people backwards, this is not the direction to go. But there's also a million startup companies building BI tools that are AI-fronted, an AI-fronted BI tool. It starts with the promise of end user LLM-style interaction. It starts from scratch with that assumption.

(15:04): And these tools are going to tell essentially the same "lie" to business leaders that Tableau was able to tell, which is, look, now it's going to be easy. And at the same time, we need to recognize that that sort of interface, an AI LLM-fronted interface for end users to ask their questions and be taken to the right place or being given an answer that wasn't already built into a dashboard somewhere, that is the place the market needs to go. That is better. That's what we talked about on the Copilot episode. But it's going to happen with a lot of products that don't have a brain. So, to give you an example of this. Apparently, Tableau recently debut MCP support. And we've talked about MCP a little bit. It's sort of positioned as USBC for AI. MCP is brand new, but it's also just really nothing different. And once you understand that it's both of those things at the same time, it's like much less mysterious.

(16:04): So, the MCP support for Tableau, it's a big deal. And there's a blog post making the rounds on LinkedIn. We'll link to this blog post. The blog post, if you just kind of blaze through it, read through it really quickly, it's pretty exciting. The blogger sits down with Claude Desktop, an LLM interface that happens to be a desktop version instead of a web version, and then installs this plug-in from Tableau on their computer. And the plug-in is the MCP translation layer, the translation API so that Claude Desktop can talk to this Tableau MCP. They call it a server, MCP server, even though in this case it's sitting on your desktop. You got to differentiate between server and cloud for a moment.

Justin Mannhardt (16:42): Back to basics on the terminology here these days.

Rob Collie (16:45): Nouns are really fun to play with.

Justin Mannhardt (16:47): A server is but a computer, am I right?

Rob Collie (16:50): Yeah. Well, in this case it's a part of a computer. So, now, Claude can talk to this. And that thing turns around and talks to Tableau Cloud. To demonstrate this MCP server and the MCP capability, the blogger grabs a data set from Kaggle, which I go and look it up. It's just a CSV. It's just a CSV of soccer players. It's a big CSV, but it's a CSV. So, it takes the CSV and uploads it to Tableau Cloud. This has nothing to do with AI at the moment. So, now it becomes available as a Tableau data source. I'm sure Tableau's got many, many, many different kinds of data sources, and CSVs are just one of them. The point is this is a CSV and all it is. It's not improved. It's just a raw CSV file. It's now available. It's been saved in a folder on the Tableau Cloud.

(17:40): And then connects to that via Claude Desktop and starts asking questions. Questions are like, what columns are available? Who are the highest rated players? And what countries have produced the highest rated players and things like that? So, it's very much an LLM style interface over the top of data, and so it checks those check boxes. And at some point they even start doing charts in this blog post. What's really funny is that the charts are Claude charts. They're not Tableau charts. There's nothing in this MCP hookup yet that allows you to build Tableau charts, nothing in this thing that allows you to build a Tableau dashboard.

(18:17): So, all of the smarts in the entire article, you come away from it going like, wow, I just got an LLM style interface to answering questions about data. That's pretty awesome. And it's got Tableau, its name stamped all over it. The thing is, there isn't anything from Tableau at all important to any of this in the entire experience end to end.

Justin Mannhardt (18:39): Yeah. We've hosted the CSV.

Rob Collie (18:42): All of the intelligence is either already in the CSV or it's in Claude. Imagine if I, Rob Collie, had come out to great fan fair and said, "Hey, folk, you know what I've done? I have built an MCP server API for CSV files."

Justin Mannhardt (18:59): Whoa, whoa.

Rob Collie (19:02): You can talk to CSVs with Claude Desktop. That already exists. All of the interesting future capabilities that will be added to this MCP interface, over time I'm sure that they'll start letting you build Tableau dashboards with it, but who cares? As long as I get my answer. That's one of the beautiful things about all these experiences is that it gets me to the answer whether or not there is a dashboard for it.

Justin Mannhardt (19:32): That's an interesting user experience question. So, you're describing in a chat interface with Claude, and I'm asking questions, and they'll either just answer my question or at some point it will render a chart for me. I think we'll still always love charts.

Rob Collie (19:48): Yes.

Justin Mannhardt (19:48): But an interesting question I had, because you were like, yeah, maybe it'll let me build Tableau dashboards, but why would I want to? Is the idea that we can clickety clack across different visuals, is it really that important if I could just go back and ask, well, what if we just exclude or include? Or how much is that building of an artifact going to matter as opposed to just being in an immersive interactive experience?

Rob Collie (20:15): Yeah. I think there's still definitely going to be some really important use cases for dashboards as we understand them today.

Justin Mannhardt (20:24): I agree. Yeah.

Rob Collie (20:25): When you were first pitched on the idea of dashboards, those really, really crucial things that you monitor every day, that was the way that dashboards were pitched to us in the very beginning.

Justin Mannhardt (20:35): Correct.

Rob Collie (20:36): When you have one of those, your ability to think at the speed of thought, and manipulate the mouse, and click, and drill, and filter, and all that kind of stuff is so much faster than your ability to type or even talk. You can have a really, really high efficiency, high speed interaction with a dashboard that you know with a mouse, that's way faster than conversation. Way faster than conversation. And so, you and I just sitting here thinking about the dashboards that we use to run the business here at P3, you can just envision them in your head. You're like, yeah, no, I'm not going to be asking English questions to substitute for that really high signal interaction that I get. Really high protein.

Justin Mannhardt (21:18): High fidelity.

Rob Collie (21:19): Yeah. So, those core uses of dashboards I think are going to stick around. It's just that we end up with so many throwaway dashboards, just like we always end up with so many throwaway reports in the old SQL world, these one-offs that were meant to answer certain questions. And we never go back to them. We had to build an artifact in order to get the answer, but now what? Do we delete the artifact and we just let it hang?

Justin Mannhardt (21:42): Yeah. The great example for me is we have our core models and reporting and dashboards we use. I can visualize the thing I do every single day multiple times a day. But it's that workflow where I want to go deeper on a question for myself, or maybe we're working on a problem together. And that's when I'll fire up Power BI Desktop and I'll connect back to the model and I'll start doing new things. I could see that being gone.

Rob Collie (22:09): Yes, yes. So, I think you fast-forward to the future, you'll see a much smaller footprint in terms of number of dashboards floating around because of those one-off uses.

Justin Mannhardt (22:19): Well, there'd be less trash.

Rob Collie (22:20): Yeah, less trash, yes. Yeah, less digital trash. Because those one-off questions, it is much faster to answer those questions via a chat interface than it is to go build from scratch, even a throwaway dashboard. And furthermore, it's not just faster. There's a different class of business user who's willing to interact with the chat interface, who's not willing to start rearranging, and dragging, and dropping to create a dashboard. If I'm using a chat style interface to get answers to questions, that tells you right off the bat, we're kind of off the main road. The main road being the place where we know people's use cases and we built really, really hyper-optimized dashboards to help them. But if using the chat interface and it hasn't taken you to one of those, how important is it that it then builds an artifact? It is important, but it's not the main show.

(23:15): The fact that I'm getting a Claude chart, a chart drawn by Claude as opposed to drawn by Tableau in that article, as far as the user concerned-

Justin Mannhardt (23:24): Doesn't matter.

Rob Collie (23:25): How much does it matter? It matters maybe a little bit because you're expecting to be able to click on it and have it interact in the way that a Tableau chart might, but only if it's given you two visuals and not one. So, I think there's something there. I think there's something real there, that we do want these interfaces to be able to build the same sort of interactive experience, even if it's a throwaway. But it's not the most important thing. If you just zoom back from this Tableau article, you see everything. This whole phenomenon is on display. It is simultaneously a very compelling blog post. Even for those of us in the data world, it's like in the screenshots and everything, you're like, "Yeah, this is cool. It's totally cool."

(24:05): At the same time, there is nothing behind this other than Claude is really the only star. If it retitled the blog post, the most roundabout way to let Claude talk to a CSV file. The fact that Tableau still to this day doesn't have anything resembling a brain under the hood, like Power BI does. Now, I know that they have done a lot in this space. They've done data modeling, and dimensional modeling, and all these sorts of things. They've been playing catch up in a way, but that hasn't gotten any traction. That upgraded brain hasn't gotten much of any traction in the market. It even struggles to get traction internally at Tableau. It came up as a different tool. It came up as the visual tool for over the top of rectangles of data.

(24:48): It is a very subtle distinction. Why a data model is important is a very subtle and actually kind of academic thing to explain, except that in real life it makes tremendous difference once you know the difference and you use it. But it is hard to understand and it is hard to explain. If anyone's listening to this and be like, "I still don't get it, I don't understand," You're in good company. It's taken me like a better part of a career to ingest and internalize this distinction and be able to explain it even conversationally. So, we're going to see a flood of this. We're going to see a flood of tools grabbing market share and being hot for a moment. There's going to be tool du jour, hot tool of the day-

Justin Mannhardt (25:37): Hot tool.

Rob Collie (25:38): ... that is going to say, look, power BI is so passé. They will not have built a semantic model engine. They're just going to be using direct connections to SQL, direct connections to data lakes. Or even funnier, direct connections to CSV files.

Justin Mannhardt (25:55): Hosted in Tableau Cloud.

Rob Collie (25:57): That's right. Yeah. Right. A lot of these startups are going to make some people very, very wealthy. That's going to happen.

Justin Mannhardt (26:06): Correct.

Rob Collie (26:08): Microsoft gave Tableau a better part of a decade head start on dashboards, and still clobbered them. I believe, and I think you share this belief, that these semantic models that underpin Power BI, that underpin Power BI dashboards, are far more powerful, far more important, far more capable than the dashboards themselves.

Justin Mannhardt (26:29): Yes. I do share that opinion wholeheartedly.

Rob Collie (26:32): I also believe that there is no better structured data source on the planet now or in the future, for the foreseeable future anyway, for an LLM front end to talk to than a Power BI semantic model. They had advice from the big four consulting firm. No, no, no, no, no, no, no, no. The semantic model is going to be the best thing, the best front end to talk to, because remember, we don't want the LLMs directly crunching data. They're not good at it. They are good at writing queries for something else to run and crunch the data, so you want the sophisticated capabilities like a Power BI model. This is also really important. The Power BI model that has established rules, let's say a measure or formula that defines what percentage of customer requests did we service-

Justin Mannhardt (27:31): Within an SLA.

Rob Collie (27:32): ... within an SLA? We met the SLA or didn't meet the SLA. The SLA differs by customer, very different contracts. It's just an incredibly rich and nuanced formula. If you were just going to do that in SQL, let's say you're not going to have a semantic model. You're going to take all the data that's fed into the Power BI semantic model, but instead just have it raw sitting there in SQL. So, where are we going to capture the semantics of that formula? You're going to start off with a raw bunch of SQL tables and you can make some cool demos. It's going to get a bunch of things shockingly right, because it's able to write SQL to answer those questions. But as soon as it starts to get the least bit nuanced, the least bit complicated about your business, Claude doesn't know your business. ChatGPT doesn't know your business. SQL sure as hell doesn't know your business, so where is this extra information going to be captured?

(28:25): You're going to have to start inventing. Whether you're an individual organization kind of like doing this yourself, or whether you're a software or vendor trying to build a Power BI competitor, you're going to have to start inventing this semantic store of extra instructions that you give to the AI to help it figure out, okay, by the way, when someone asks about SLA, our SLA hit percentage or satisfaction percentage with SLA, and then blah, blah, blah, blah, blah, here's a bunch of English about how that's calculated. And then you have to trust that this thing understands that and is executing it properly every single time, because remember, there's some non-determinism.

(29:04): There's a little bit of deliberate randomness-

Justin Mannhardt (29:05): Some?

Rob Collie (29:07): ... in how these things operate. They're not going to give exactly the same answer if you ask it twice in a row the same thing. You're going to want something so fundamental to your business to be reliable. You're going to want to know, like the line from Pulp Fiction. We don't want to think. We want to know. We want to know with 100% certainty that every time someone asks about that particular metric, the same correct formula is being followed. We want to be able to test it, debug it, make sure that it's solid, and then have it go with that. So, you're going to end up having to reinvent everything that the semantic model already did. And then you're going to reinvent it in a really, really inefficient and clumsy fashion. And then in the end, one last final mwah, you're going to deal with the incredible performance problem of running SQL or whatever data lake type of queries you've got over this aggregated space that is your business.

(30:00): Because that's the other reason why OLAP databases were invented in the first place, the reason why SQL server analysis services existed, was because raw SQL style queries were too slow for many, many business purposes. The Power BI engine, the semantic model, and the engine that processes it, that answers questions about it, I believe is here to stay. I believe it's the apex predator in its space. And yes, Microsoft is going to need to crack the riddle of having the front end experience, the AI front end experience. It's going to need to be really good. And they are also guaranteed to fall behind on that.

Justin Mannhardt (30:43): 100%.

Rob Collie (30:44): Today, I think me, you, Brian, Julius, we could sit down and come up with, data Goblins' Kurt already has done this, an open source alternative to Power BI Copilot, that is probably better than where Power BI Copilot is today. But that's not going to last.

Justin Mannhardt (31:01): Correct. I agree. Yep.

Rob Collie (31:04): I described this the other day as this is a baseball game, each batter is getting up. So, the new startup goes to the plate. They get three strikes just like everybody else. They get three strikes, they're out. And all these people come to the plate one at a time. They get three strikes, some of them hit, some of them get on base, some of them don't. And then eventually, Microsoft comes to the plate and they look at the rules. It's like, oh, yeah, Microsoft gets 25 strikes. They have the incredible field position. The most valuable data source to be interrogated by an LLM is a Microsoft semantic model. And everyone's looking to them for it to be built in. They just have to get there.

Justin Mannhardt (31:46): It's such a good point of reflection when you were talking about how there hasn't been a true competitor to the tabular database. And the only way that this doesn't play out the way you're describing it is if something like that shows up from somewhere. And I don't know what it would be. I was like, okay, how could I devil's advocate this? And you could imagine, okay, sure, maybe there's an AI that is also able to maintain and add to the rectangle data store underneath it. But the compute, that's where it all falls apart. If it can't efficiently compute the answers to questions and serve the user, what are we doing?

Rob Collie (32:35): These GPU based LLMs are kind of like us. That's the scary thing. They're getting better, and better, and better at thinking and acting like us. But just like human beings are terrible at crunching data into an aggregate answer, these GPUs, it's not quite a perfect metaphor, but they are certainly less efficient than optimized CPU based code, which is what a Power BI model does. That's just like a law of physics. We are going to need those CPU driven systems still, and CPU driven systems that are capable of ingesting and following semantic rules about your business, that capture how your business actually operates. Whatever comes along that truly threatens Microsoft's hold on all of this is going to need to check those check boxes.

(33:27): And it's even reached the point where I think talking about who's going to win the BI tool race in AI is kind of weird, because BI tools have always been just a means to an end. In the end, I need to be informed enough, most importantly so that I can take action to improve things. And the race isn't to become the new Tableau or Power BI with an AI front end. That isn't the race. I think the race is to have an experience for structured data that addresses the most need. To me, as far as I can tell today, that looks like Power BI semantic models with a much upgraded AI powered front end.

Justin Mannhardt (34:18): The cool stuff with the changes to Copilot are encouraging, but I don't think it's quite there yet. It's still got a lot to be desired, like the sexy demo where you just show up on a chat interface and it's popping new charts at you and that kind of stuff. Whoever delivers on that is going to gain a ton of attention, a ton of market share. But I think it's needed. Whether it's Microsoft or somebody else, that's going to be the signal where we all kind of rally and say, yes, we want that to exist. And then the big players will start catching up to that, and acquire each other and all that.

Rob Collie (34:57): Whatever that end user experience ends up looking like. When you say, because they're going to acquire a ton of market share, like 3% market share. And that's going to be huge. 3% of the market is a lot of money, and so that's enough to make the founder maybe even to a low end billionaire. They're going to be a media darling for a little while. The things that are going to come up are going to be the things that are easy to build in a way. And it is going to be sort of like the design of their UX. That's kind of the real winner. And that's so easy to copy. That's going to be easier to copy probably than it was to copy Tableau's dashboarding experience. It's going to take less time. It's a less durable lead.

(35:37): We will always remain vigilant, but part of vigilance is sorting through the noise and the hype to say that this isn't it. When you're being told this is the threat to XYZ, you've got to be open to the idea that it might be. But then when you go and you look at it and you look at it with critical eye and determine, no, this is not, and it's just hype. And you've got to be able to do that, too. This Tableau MCP server as it currently stands, this is silly.

Justin Mannhardt (36:06): Kind of a joke. It sounds like a joke. It would be humorous in certain circles.

Rob Collie (36:12): Yes. [inaudible 00:36:14] very tightly, properly constructed joke here that would make a certain class of data people just absolutely gut-bust in laughter.

Justin Mannhardt (36:21): [inaudible 00:36:23].

Rob Collie (36:28): So, then I said...

Justin Mannhardt (36:34): Apparently we're that class of people. All right, Rob, did I tell you about my Markdown fiasco?

Rob Collie (36:41): No.

Justin Mannhardt (36:42): M365, the Copilot app, they have this feature in there, I think it's called Notebooks. I like keeping organized, and so I've been doing this thing with ChatGPT canvases, where I go through certain routines every week, so I'm going to give this a whirl in Copilot. And so, I was looking for a template. And so, it rendered a template for me in our chat in Markdown notation. And if you're not familiar with Markdown, is you use certain characters indicate this is a heading or a sub-heading or a bulleted list or a to-do list. So, I take that response and I copy it and I paste it over in this notebook thing that I can supposedly also interact with. And it doesn't format it at all. It was like four back and forths of like, "Hey, you idiot. You gave me this thing in Markdown and it doesn't work in your tool. Please stop." It was just woefully insufficient compared to ChatGPT.

Rob Collie (37:36): Well, so I noticed this with Claude. Not Claude. I think it was Gemini. Gemini wouldn't give me a Word doc. It would refuse to. It would only give me Markdown. This was my first experience with Markdown, was Gemini refusing to give me a Word doc. It's trying to give me instructions on how I can convert the Markdown into a Word doc, Word formatting and headers and stuff.

Justin Mannhardt (37:57): It's just been glaringly obvious. And it's not just Microsoft. Maybe pull back the aperture. The big four tech players, so if Amazon, Apple, Microsoft, and Google, you could throw a Meta in there, I guess, but Google and Meta are the only two that are like, "We have our own models." And Apple has been totally face planting with AI. Amazon's really not doing much of anything, other than like the AWS, you can deploy some infra stuff, and then you got Clippy Copilot over at Microsoft. There'll be a fast follower move, I'm sure. It's just a matter of who's going to do that play versus they'll figure it out on their own.

Rob Collie (38:40): Yeah. I mean, Apple being a consumer brand and an individual brand much more than a... They're not a business tech brand. They're largely just going to sit most of this AI stuff out.

Justin Mannhardt (38:53): Just keep building cool iPhones, and watches, and glasses, and whatever else.

Rob Collie (38:57): Whatever stupid Apple intelligence is, like, no. If Apple had a hyper, hyper, hyper upgraded Siri, truly like my personal assistant, I could imagine them getting that right, maybe.

Justin Mannhardt (39:08): Oh, but they haven't. Have you read any of this?

Rob Collie (39:08): No. No. No. No.

Justin Mannhardt (39:12): Oh, it's been bad. It's been a bloodbath.

Rob Collie (39:14): Okay. There's no reason to bet on them over some third party.

Justin Mannhardt (39:20): No.

Rob Collie (39:20): But of course, then Apple's going to find ways to bake it into their iOS in ways that external people can't. Some of this is just going to play out via the old rules. If you think about software companies as real estate companies, they own certain real estate. Semantic models are a very, very, very valuable piece of real estate that Microsoft owns. They built it, so it's not like the normal real estate game where you just come along and buy land that was already there. They did build it, so props to them. But would you rather be moving faster with AI tools? Which by the way, the people who are building these things, they don't own the tools either. If someone's building a BI tool right now based on AI, most of their code is actually just OpenAI or Claude or Gemini. That real estate is owned elsewhere too. It's going to be interesting. Every AI driven feature at Microsoft is going to have to shed whatever this anchor is that they're dragging. You can feel it. They're dragging it kind of everywhere.

Justin Mannhardt (40:26): Well, I will be grabbing my bucket of popcorn.

Rob Collie (40:29): To sum up, the AI driven end user interface is going to be the place the market's going to go. That's the place the market needs to go. That's the place where these tools need to go. And there's going to be people who are putting together very, very, very compelling demos in the meantime and developing companies with lofty valuations carried on the PR of that. But when you really get down to it, the two most important components of any such solution are going to be the LLM itself, the big AI brain itself, like from OpenAI or one of those other players. It's going to be that plus whatever the engine and storage is for storing the semantics of your business, what the formulas should be. What does this thing mean? What does that thing mean?

(41:17): And processing the query, because the LLM isn't going to do either of those two things. So, Microsoft has a very, very, very strong solution for that second half. I don't think anyone else really does. From a longer-term prediction market, I still think this is Microsoft's game to lose. I don't think they're nearly the trouble that some people think that they are. You got to be clear about what's actually important in this space. I've been really grateful to have the opportunity to be refining my picture of all of this over the most recent couple of months here.

Justin Mannhardt (41:54): It seems like we figured it out.

Rob Collie (41:55): Totally. Nailed it.