It’s Time to Start Looking Into Microsoft IQ

Announcer (00:04): Welcome to Raw Data with Rob Collie. Real talk about AI and data for business impact. And now, CEO and founder of P3 Adaptive, your host, Rob Collie.

Rob Collie (00:20): Hello, friends. If you've been listening to the show lately, you know that I am writing a book about AI. As expected, it has been a grueling four-month process, a labor of love at the same time, but also grueling. I really do feel like I had to write it, both for business reasons and for human reasons. I spent the last 18 months going down the rabbit hole of what does it mean for a data firm, or even a software and data professional like myself, to survive and/or thrive in the age of AI?

(00:51): Now, I didn't go down into that process thinking I was going to write another book because the Power BI books I wrote in the 2010s, as successful as they were, those things damn near killed me. Writing this one is definitely part of the reason why I have been sick for two-plus months. It didn't get easier 11 years later. Trust me, when I say I feel like I had to write this book, I mean it. I wouldn't do it if it felt the least bit optional. It's not optional because the things I've seen on that journey, I need to share them. I'd need to write this book, even if it were just our 50 employees reading it, which they have been in draft form as I go.

(01:30): But I see a much broader need in the market. The big tech firms are racing so hard and so fast against one another that they don't have time to stop and explain to the average professional what this all means and where it's headed. Eighteen months ago, even me, a longtime tech professional, very well-connected, didn't know what AI really looked like in a business sense, so how can we expect the broader business world to know? Now that I have a much, much clearer picture of it, I feel a sense of relief and optimism that I definitely didn't have before.

(02:01): Yes, the book will benefit me professionally. It will benefit our company, too. But doing something this difficult, just because it's smart, I've never been that guy. I need an emotion to drive it. The emotion for me has been this feeling that I can't just hoard the picture that I now have. That is what is driving me to finish it.

(02:22): But I'm actually not here to talk about the book today. I told you all of that because I'm writing and recording this podcast instead of putting the book to rest. Right now, I could be doing the most amazing thing, which is finishing the final chapter, chapter 16, of that book. I seriously could be finishing it today, putting it to bed, putting it behind me, which will feel amazing. Plus, I was due to be finished on Friday, two days ago. I'm past deadline. I am writing and recording this on Sunday instead of putting the book emotionally and symbolically behind me. Instead of kicking off the process of getting it out there, I'm putting that on hold to tell you that you need to be learning about Microsoft IQ.

(03:06): A few weeks ago, I had a meeting with someone very senior at Microsoft, someone whose time I really have no right to getting, but who was gracious enough to give it to me. In this meeting, he suggested that we, P3, start getting into Microsoft IQ. Of course, I heard that, but we were talking about a lot of things in that meeting, and honestly, I don't absorb new stuff all that quickly. I usually require the long form explanation of something that might take a couple of hours before I get it.

(03:32): Anyway, someone at Microsoft who's important enough that I even feel a little bit guilty about him meeting with me, told me that I should be looking into Microsoft IQ, and I didn't. Again, so many topics covered in that meeting, so much change already in progress at P3 as we pivot into AI, I'm writing a book, and I don't understand the relevance of things when they come at me in compact form like that.

(03:54): Ah, but friends, then I had to write chapter 14. Chapter 14 took me two full weeks to write, a single chapter, and it's the chapter about data strategy for AI. Now, pause for a moment. In a book about AI, the single longest chapter, by far, is the one about data, and the chapter's title is Data is Nearly Everything. That should be exciting and reassuring for anyone listening to this. The chapter is about how your AI strategy depends on data, on structured database and data lake kind of data, and also on unstructured data, the everyday content living, document libraries, and folders, and also the handbooks and backgrounders which shape the behavior of AI agents. Also, documents.

(04:44): Now, going in, I thought this was going to be a pretty simple chapter, like a layup. When I'm writing a chapter, I'm really telling you a story. I'm taking you on a trip that shows you something important and paints you a picture. Whenever I'm about to go on the trip of writing that story, writing a chapter, I tend to have a map in my head of what that trip is going to look like. Sometimes, the story plays out just like the map in my head. A number of chapters in the book have played out just like that, but not chapter 14. Chapter 14's story took me where it needed to take me, and it took me to Microsoft IQ.

(05:18): I did know going into chapter 14 that I was going to talk about more than just structured and unstructured data. I knew that I was also going to talk about metadata, things like Power BI semantic models. They're decoder rings which help the structured data make sense in a business context. If you're listening to this and you've been up to your eyeballs in Power BI for the past N years, that's what you've been building, semantic models that serve as decoder rings. Yeah, you've also been building dashboards, but if you've been doing it right, you've been powering those dashboards with well-built semantic model decoder rings.

(05:53): Those semantic model decoder rings are crucial for any organization's AI strategy. I mean, seriously, that's the truth. There isn't going to be some bright agentic AI future for any organization until they get their semantic models in order. Even all of the data and BI software vendors who were out on semantic models during the pure BI era are now in on semantic models.

(06:20): For years on this podcast, we made fun of Tableau not having a semantic model. When they finally got one, we then made fun of how they neglected it and how none of their customers adopted it. Power BI remained the only first-tier BI platform built around and committed to semantic models. The other software vendors in the BI era naturally downplayed the importance of semantic models. "Nah, you don't need that nerdy shit," they would say. "Just sit down, start writing SQL queries, and building dashboards." Yeah, yeah, yeah. Every new dashboard does require you to reinvent the wheel with multiple pages of brand new SQL queries, just to paint a slightly different picture with the same data, but so what? It worked, right?

(07:01): Now for this next part, I'm going to continue to use Tableau as the example since they're firmly the second-place BI vendor, but this sequence of events is basically what went down at every non-Microsoft data and BI software company in the last 18 months. Reinventing the SQL query wheel for every new question did suck in the era of BI, but it also still worked as long as you had a SQL-wielding Tableau developer to change their desks and make them bang that shit out for years on end. It was slow, yeah. But if you didn't know there was a better way, and if you were a Tableau customer, how could you know there was a better way? You just lived with it.

(07:40): Well, it's one thing when a human being has a new question and that question goes to the Tableau developer's backlog, but it's another thing entirely when an AI agent has a question. The whole point of AI agents is automation. An automation that waits days or weeks on a new SQL query to be written isn't automation. Tableau and other vendors saw that, and I don't have any insider information here at all, but I'm still absolutely positive this is what happened next.

(08:09): First, they were like, "Oh shit, that means none of our customers are ready for AI. When they start building AI agents, they're going to realize that we, Tableau, are no help." That's a bad place to be. Now, I'm sure Tableau leadership's next question to their engineering teams was, "Oh, okay, okay, okay, folks. Now tell me, the AI agent can just go ahead and write the SQL queries to answer its own questions. It can step in and fill the role of the Tableau developer, right? We're okay after all?" Nope. Friends, that doesn't work either for many reasons.

(08:42): Without certified definitions of metrics, the kinds of things that live in a Power BI semantic model, if you leave the AI to freelance its own definition, its own formulas, not only is it going to get it wrong, it's not even going to come up with the same definition each time you run it, so not just wrong answers, but different wrong answers every time. Oh, and it's also going to take it a long time to come back with those wrong answers because even writing incorrect SQL takes a long time, and it's going to burn a lot of AI tokens in the process. Without a semantic model, your AI agents are going to be slow, expensive, inaccurate, and inconsistent. What's not to love about that?

(09:23): What happens next at those vendors? Major change of heart from basically every data and BI vendor. Semantic models are now super important. In September, Tableau's official blog featured an article saying that the agentic future demands a semantic layer. Now, of course, the article specifically was titled The Agentic Future Demands an Open Semantic Layer, because when you've realized the Power BI folks were right all along and the install base of deployed semantic models in the world is like 99% Power BI models, you can't just invent your own competitive semantic model format. You have to band together with a bunch of other companies and develop an open standard because the promise of interchangeability is really the only carrot you can dangle at the world when you're that far behind.

(10:10): To be clear, it's a good carrot. There are dozens of data and BI companies signed on for this new standard, some big names in there beyond Tableau and Salesforce, like Snowflake, Databricks, AWS. That is going to make this new standard, called OSI, a viable competitor, but it still doesn't make millions of semantic models just start magically appearing at their customers. That's work that the customers have to do.

(10:35): As anyone from the Power BI ecosystem knows, you have to learn to build these things. You have to understand why they're important, and then you need to spend a lot of time building, iterating, fine-tuning, et cetera. Ignore what the software companies are doing for a moment. The customers who have been in on adopting Power BI have a big headstart on AI, which is good news for most of the people listening to this.

(10:58): Folks, that was going to be the end of the chapter as far as my original map in my head went. But no, I knew that Microsoft has been expanding outward and upward from semantic models with this thing called ontologies. Now, if you think "semantic model" is a nerdy phrase, ontologies is here to say, "Hold my beer." This was a word that I learned originally in college in my philosophy double major in a class called metaphysics. Now we're all going to be saying this ridiculously nerdy word for the next several years minimum. That's delicious.

(11:33): Anyway, now Microsoft has ontologies. I mentioned they grow outward from the Power BI semantic models you already have, so think of them as semantic models plus plus. Folks, you are going to love this shit. If you've been doing Power BI the right way, the part you've enjoyed is building the semantic model. No one loves building and formatting dashboards. Okay, that's probably too strong a statement, and some of you probably do love that too. I don't. It's fun for the first 10 minutes, but then all the detailed formatting kicks in and yuck.

(12:05): By the way, as an aside here, this week at P3, we're also unveiling our P3 AI framework. You'll be seeing it on LinkedIn. As I've mentioned previously on the show, we've hired full-stack developers for the first time because we've been building reusable hardcore software platforms for the first time ever. P3 AI is 100% built around Power BI, and lets you do amazing things from a chat client like, for instance, build and modify Power BI reports, the things that I hate building, even the slickly formatted custom visual Deneb kind.

(12:35): You just tell it in chat what you want, and it both builds and deploys straight to the Power BI service the thing that you're asking for. No files to upload, no Power BI desktop required. You get a screenshot of the report in the chat client for quick preview so you can quickly fine-tune without even leaving the chat. You can be in Teams or Slack or even WhatsApp on your phone and creating new Power BI reports or modifying existing ones, or not even bothering. Just asking questions, and getting answers and charts rendered in the chat window, 100% laser-focused on your question and not deployed anywhere, all backed by our semantic model.

(13:12): We're not selling this as an off-the-shelf SaaS product. It's something we can deploy for our clients, and then customize to their needs so it goes beyond just answering questions. It gets into things like creating alerts and even taking corrective action in other line of business systems by notifying other people, whatever you need without ever leaving the AI chat experience. There's a web version, there's a built into the Power BI reports version, and there's support for basically every chat application under the sun.

(13:38): I wasn't kidding about WhatsApp, by the way. We've factually got a P3 AI project running right now where, I'm not making this up, the top two or three executives at a major financial firm are using WhatsApp on their phones to answer data questions sourced by their Power BI semantic models. For more on P3AI, see my LinkedIn feed. There'll be more details on there and a demo video, link to a webpage, all that kind of stuff.

(14:01): But now back to ontologies and back to what it means for our listeners. Semantic models are fun to build. It was fun learning how to build them. Now, Microsoft is giving us a brand new playground like that, a reason to revisit and extend your semantic models in ways that define alert thresholds and actions to take. Bringing verbs to your models, which have long all have been about nouns. Holy hell, folks. This is like kid in a candy store, Br'er rabbit being thrown into the briar patch territory. Oh, and it also starts to bridge the gap for your careers and your companies between BI and AI, a bridge to the future, a lifeline, the thing we all need.

(14:44): Now, ontologies and semantic models are the star of something they are now calling Fabric IQ. Fabric IQ is the layer that AI agents go to when they need to know about structured data, what it means, what it's telling us about our business, and now what actions to take when it tells us certain stories.

(15:02): But since structured data isn't everything, Fabric IQ has two friends, which agents go to when they want to know about unstructured data and other things. Those two friends are called Foundry IQ and Work IQ. Foundry IQ is about all of the everyday content, documents laying around your organization. When an agent needs to find relevant information for a task, like previous statements of work or case studies, Foundry is kind of like the registry and search layer that the agent can go to rather than carrying all of that indexing and search code in its own tool set. But there are more smarts in Foundry IQ than just registry and search. It's more like a research librarian, able to comb through things, make judgements as to what's relevant, and even iterate until it has the right material to give back to the agent.

(15:48): The second friend, Work IQ, is actually about a mix of unstructured and structured data, but through the personal lens. If an agent is more like a direct personal assistant to you and you say to it, "Hey, please notify the working group that we're going to miss the deadline by a week," it needs to have access to your individual calendar, emails, and Teams messages to even determine what working group you're talking about, who is in it, and maybe even people who aren't strictly in the working group, but do care to be notified. Three different kinds of data, structured, that's Fabric IQ. Everyday document, content spread over your company's shared folders, SharePoint sites, wherever, that's Foundry IQ. Access to the logged-on user's specific world, that's Work IQ.

(16:34): Now, my theory is that they needed consistent naming across these three categories, but they needed it after they'd announced ontologies, which is why Fabric IQ seems to "just" be semantic models and ontologies. I don't know if that's the truth. I can't really tell yet, but it seems likely. It helps me understand the broader Microsoft IQ thing, so I'm sticking with that understanding until I learn otherwise. Fabric IQ equals semantic models and ontologies.

(17:00): Now, most of this Microsoft IQ stuff is either already released or in some form of preview today. I always struggle to stay on top of that sort of thing, so I'm the wrong person to ask about specifically where things are in that pipeline. You absolutely can start reading about all this and you can start playing around with ontologies, which is the natural place for you to start anyway, but don't sleep on the other two components of Microsoft IQ. I want you to view the domains of Foundry IQ and Work IQ as "just" new frontiers in our data professional careers.

(17:32): We used to be fenced in, structured data only. Nope, now we're free-range. What counts as data today? Anything that ever gets stored to disk. The version of me at the end of writing chapter 14 was a different person than the version of me who started writing it. Not only are these IQ things super relevant to data people making the transition to AI, but they're also a huge help to the people programming the AI agents themselves. Before I wrote chapter 14, I had been subconsciously leaning toward something like, "In the age of AI power development, you can just build everything from scratch."

(18:09): Now that I've seen the whole strategy, I'm starting to reconsider that vibe. Even with AI power development, no one really wants to write their own search-the-entire-Microsoft-ecosystem code any more than someone who makes a video game wants to write their own video card drivers. There is so much to AI agent building that feels like you can just freelance it in the beginning, and you can, but as you climb the capabilities curve and want to build more and more into your agents and then increasingly need to monitor, govern, and administer your agents, yeah, I'm afraid I'm going to have to admit that we're probably going to want to at least consider reigning in a bit of that freelancing and start leaning on some prebuilt platforms.

(18:46): Microsoft IQ is, of course, just one such platform, and the other vendors are in various stages of unveiling their equivalents, but I think all of us who spent the last decade plus in the Microsoft ecosystem can now see their vision for keeping us all at their table for AI. That starts with, but does not end with, our existing investments in Power BI semantic models. It's early days, of course, and time will tell. Even here at P3, Microsoft IQ is new enough that we're still ramping up, but we're ramping up fast and we're ramping up faster than we were before because chapter 14 took me back to the place that a Microsoft executive had previously told me we needed to go.

(19:25): Thanks for listening, and catch you all next week.