Rethinking the ROI of Dashboards

Rob Collie (00:00): Hello there again, Justin, this is not our first time talking today, but as far as the podcast audience is concerned, that's the only time we talk all week.

Justin Mannhardt (00:07): We could keep that illusion alive if we wanted to.

Rob Collie (00:09): In fact, we're actually frozen in temporal stasis most of the week. They just,-

Justin Mannhardt (00:15): They let us out.

Rob Collie (00:15): Defrost us to do a podcast. It's like Severance, like this version of us only does podcasts and we're never allowed to even see the outside world. Is there a sky? We don't know.

Justin Mannhardt (00:25): I've not watched Severance, but I've been given the context recently and I was like, oh, I want to watch this show.

Rob Collie (00:31): It's also like the plot of the Animaniacs, the old cartoon. They constantly escaping from the water tower, but yeah, I highly recommend Severance. It's probably the only show on right now that I'm actively craving. I can't believe that I have to wait another week for an episode of it. I think the same became true of Silo as well. Those are both solid plays, but that's not what we're going to talk about today. I stumbled across something on Reddit that I thought was really, really thought-provoking and it touches on a number of themes that are near and dear to us. It's not like a brand new question, but the way that it was formulated caught my eye. It was in the business intelligence subreddit. The formulation of the question was, has anyone here moved away from dashboards as their main source of insights?

(01:12): So it's kind of like another version of that are dashboards dead question. This question isn't being asked in a way that's meant to lead people to adopting a particular software platform. This was a genuinely thoughtful discussion. The gist of it was that dashboards are too static. Even though they're interactive, dashboards are too static to really scratch the itch of someone who's truly analytical. At the same time, the average business user, the average business consumer doesn't interact with them enough, and so the utopian view of dashboards is that you look at them, you see things, and then you can point and click and get your answers to all your follow-up questions and all of that, and isn't that wonderful? And certainly that is how it does work some of the time, but I think we've all experienced this phenomenon of people aren't using the dashboards the way that we thought that they would.

(02:06): While at the same time, if you or I are sitting there and we're the ones with the data model, we've got Power BI Desktop open or whatever, a lot of times the questions that we have, we go, oh man, I haven't built a dashboard for that yet. I haven't built a report for that. So I create some random throwaway tab in the Power BI report. Sometimes I copy an existing tab of the pibix file and use that as a jumping off point. Between those two, essentially this Reddit thread was questioning whether or not the ROI of dashboards is all it's cracked up to be, and I love the fact that we've even reached the point in the world where we can have such a conversation. We need to recognize the fact that you have to get to a certain point before you can criticize it. Even if these people in this thread were rabid dashboard haters, which they weren't, they were very reasonable people. Right. Even if they were rabid dashboard haters, we would be able to say to them, listen, previous generation of dashboard builders, they died to make your hatred possible.

Justin Mannhardt (03:01): Show some respect.

Rob Collie (03:02): Yeah, that's right. Do you know what it was like before dashboards?

Justin Mannhardt (03:06): Have you ever seen Cognos?

Rob Collie (03:10): If you reach the point where you're starting to question whether or not your dashboards are adequate ROI, you've actually created yourself a good problem. You've gotten to a good place where you can think thoughtfully about getting better. No matter what we would conclude from such a conversation, we have to start with the acknowledgement that most people still haven't reached this point.

Justin Mannhardt (03:27): It's true.

Rob Collie (03:28): They don't have sufficient dashboard level clarity into their business for them to start getting hungry for more. You've got to recognize the progress that's been made to even get you to this point of criticism.

Justin Mannhardt (03:41): This is probably the most authentic framing of this question that I've seen. A lot of what I've seen in this category of discussion is intentionally provocative, push an alternative product or methodology, but this is someone who does data for a living. Is this the best way to keep doing it? And I think there's a couple themes that really rise out of this conversation for me. One is the idea of maturity within a company or a group of people with data and tools. Then the other idea is the correct application of tools to problems, which is another interesting thread, and so I think there's different ways I could look at the context from which this question might be being asked. So for years and years and years, the business community was in a data drought. The earliest level of maturity in a data journey is just the fact that people can get it.

Rob Collie (04:45): I love the use of the word maturity there. That would've been a better way to crystallize what I was trying to say before. You have to reach a certain point of maturity to even have the right to ask these questions. I wouldn't want people listening who haven't reached that level of dashboard maturity go, well shit, we shouldn't even try.

Justin Mannhardt (05:00): Right.

Rob Collie (05:00): No, no, you should try. You need to get there. There's a lot of ROI. Leaving behind the world of 500 static reports and a bunch of Excel spreadsheets is a game changer, and then you arrive at this next place and go, hmm, what's next? So yeah, maturity is a really good word and in last week's podcast I even hit on the same thing. If you think about it, how many hundreds of years of business has the world had? Centuries of business. It's really generous to say that it's only been in the past 15 years that you could actually be data-driven. Honestly, for most companies it's more recent than that, that's even been possible, so let's not miss the bigger picture here.

Justin Mannhardt (05:38): The maturity and the application of the tools to problems. There's a cross section of this that might be interesting to get into. When I read that initial post and some of the threads, and I've seen this with companies too, there's one level which is, okay, we finally have dashboards. People in our company can see our data. They can understand what's going on. They don't have to wait a month for a report. There's a really positive impact that happens there.

Rob Collie (06:06): Yes.

Justin Mannhardt (06:07): And then as people start using, you see a couple things that are common. One is, let's take the idea of let's say an executive team. They're using dashboards, but they realize they can't click and point their way to every answer to every question, and so then they're going back to the BI team and maybe they got a good thing going and they can keep supporting each other, but maybe there's some friction there. I want this tool to do something that it's not doing.

(06:33): When someone who's very analytical and very curious about the data starts to get very, I'll use the term advanced, I don't think that's quite the right term, like with their metrics and their analysis and the things they're presenting, sometimes that can obfuscate a lot of what an end user wants to make sure they also understand. It's like they start communicating in a language that is foreign in a way. Well, how did you get there? Where's the underlying data? And so there is this interesting dilemma that I think people struggle with and that's where I think it really has to do with, are we applying the tools in the best way to the problem that we're trying to deal with?

Rob Collie (07:10): You mentioned the executive that comes back and says, Hey, this thing doesn't answer what I need it to say. It's a coin flip whether they just haven't found the thing.

Justin Mannhardt (07:18): Right.

Rob Collie (07:19): The things I've recounted from the hockey dashboards, it's been kind of neat having that stakeholder audience because the stakes are so low. No one's career is being judged when they ask a question like, Hey, it would be great if the hockey dashboards could do XYZ. It does. It's on page four, but you got to click over here. Right. And so mapping your questions, things you want to know, mapping those into this geographical space, that is the list of reports you have access to and all of the tabs in those reports and the names of the visuals, the titles that were given to the visuals in each of those reports and the titles that were given to each tab, does that line up with the way you think about your question? And so often the answer is no.

Justin Mannhardt (08:01): Right.

Rob Collie (08:01): There's even a finding the answer that's there and knowing how to operate the dashboard got to introduce you to control click so that you can multi-select those two bars on the slicer because you want to see the two of them combined, right? Do you know how to do that? We should be very, very clear that all of this is definitely more difficult than we want it to be, but again, don't get the mistaken impression that dashboards are worse than the things that come before it or that dashboards are worse than not having dashboards because they are infinitely better than not having them. The only thing I don't like about how the question's formulated, is anyone else moving away from dashboards source of insights is because it right off the bat, sets it up as a binary thing. Moving away as opposed to being really nuanced and precise about where dashboards are not getting it done for you and then asking on an individual basis, what should we do about each of those individual problems?

Justin Mannhardt (08:56): 100%.

Rob Collie (08:57): It isn't going to be, well throw this whole thing out. No, I talked about this in other podcasts as well. We are visual creatures like it or not, and the concept of having a bunch of visuals all together on one page with a relatively natural navigation experience, not great. This is a very powerful concept that is going to survive a lot of attempts to declare it obsolete.

Justin Mannhardt (09:21): There's also, you just mentioned the, has anyone moved away part of the question. I kind of also am fixated on dashboards as main source of insights and that last word insights. One of the first comments is insights is such a cursed word these days.

Rob Collie (09:38): That's in the Reddit thread, right?

Justin Mannhardt (09:39): Yeah, it's in the Reddit thread and because insights, it's a word that can take on so many different possible meanings, everything from insight into the business, I know what's happening in the business to some sort of new idea or eureka moment of what's going on. I think sometimes there's almost this expectation that we're going to have these constant magical discoveries with every single dashboard. That happens, but to say that dashboards only purpose is that, it's sort of setting expectations up to lead to disappointment.

Rob Collie (10:18): Every time I open a dashboard, I'm going to have this eureka moment running around the hallways, right?

Justin Mannhardt (10:22): Yeah. Ooh, insight.

Rob Collie (10:23): Did you know?

Justin Mannhardt (10:24): And I think that's that application thing where if what you're really trying to do is deeply analyze and understand a problem, searching for a critical insight, that can be kind of a breakthrough moment. That's somebody doing the work that's not just, Hey, here's a dashboard. Go find it. Right. There might be dashboards involved and iterations of dashboard.

Rob Collie (10:46): That's the make a copy of the sheet tab jumping off point that's being performed by the person who built or someone who's capable of building things like that, right, like not the average business user completely. It just occurred to me that as a jingo. Right. If we wanted to be the provocative dramatic influencers on social media, Justin, we could start saying something like, insights are dead.

Justin Mannhardt (11:09): Insights are yesterday's news.

Rob Collie (11:11): You know what it's about now? It's about out sites. I'm just going to take the opposite of in, squash it together with sites, and we're going to come up with something that means whatever we want it to mean, and it will get clicks, Justin, it will get all the clicks. Anyway, a number of different dynamics here, right? Just the fact that dashboard's being kind of a natural interface. They're not that natural. There are parts of it that aren't, and some of this is the people who build these things have a different take on it and a different type of enthusiasm than the people who are going to be using them. It's so easy to sit down and build a bunch of dashboards going data forward. I'm looking at my data model and going, Ooh, what can it tell me? And I start creating dashboards from that.

Justin Mannhardt (11:53): Pages and pages.

Rob Collie (11:54): Yep. As opposed to stopping and thinking, what are the things people out there need to know? What are their jobs like? What are their workflows? And working backwards from that. This whole notion of verb oriented as opposed to noun oriented. No matter how many times I teach myself this lesson or teach it to others, I still am vulnerable to it. I'm still vulnerable to making this mistake. It's so hard to force yourself to get into the stakeholder's head. As a parallel remember, if we think about dashboards as software, come back to my days at Microsoft, the first design we would ever come up with for any new feature in Excel, we would take it to usability testing. We had the benefit of a laboratory with two-way mirrors and cameras and all kinds of, time stamping and just all kinds of equipment. It felt like we were in a recording studio with Aerosmith.

(12:43): And inevitably our first design that we were so proud of would just absolutely face plant. Users couldn't understand how to make it work, always things that would seem so obvious to us going in like, well, that's the button you click right there. We're sitting behind the mostly soundproof glass going, "Come on, click it." Right. Like it's right there. We're all can see it. We're just like, ah. If our first designs for features like that in Excel were invariably DOA in V1, why do we think that dashboards aren't going to be like that? They should not survive first contact with your user base. But here's the thing, whereas in the Microsoft case, we had all the resources in the world. We could recruit. We had a whole recruiting arm that would go and find us people that were in the right demographic and incentivize them to come in, and we had teams that would help us build the prototypes and everything. And so in an average company environment where you're building dashboards, you don't have even the cultural expectation that that's going to happen. It should and it wouldn't be that hard.

(13:49): What's the prototype? The prototype is just the dashboard. You built it. You don't need a development team for that. You are the development team. And here's the thing. They're going to think of it unfortunately as a litmus test on their intelligence. Whereas these people who are coming in to our lab at Microsoft, they kind of knew when it didn't work, that it was our fault because we're big, bad Microsoft. Their failures with our software weren't going to be watched on video by their colleagues back at their real job. Right. They're just going to be watched by us. So that level of feedback as to whether or not people are using it or succeeding with it the way that you want them to, it's kind of one of the reasons why we were so kind of jazzed by the BI Pixie conversation with Gil a few episodes back.

Justin Mannhardt (14:35): It's a good reminder of a few practices that have always been helpful for us. The projects where things just really, really went well. As much as the idea of talking to your stakeholders and gathering their questions and that requirements-esque exercise. I still think even though things move faster, we're not in the same conversation about the dashboard as much as we need to be, and that's that idea, we call it co-development. Yeah, there's times where I go away and build and come back, but when you can sit there and have a starting port of some data and be in the same room or on the same call, be there live with you, sort of iterating through this, it always helped me get a much better sense of where the dashboard would go because I'm responding to your questions, not anticipating your questions on my own.

Rob Collie (15:29): And responding to the misunderstandings. Sometimes the stakeholder asks a question and you go, oh, right? You and I, dear stakeholder have been walking in parallel universes for the past half an hour without realizing it, and now they merge.

Justin Mannhardt (15:46): Yeah.

Rob Collie (15:46): And now everything we've talked about before, I see it in a different light, and that only happens with two-way communication.

Justin Mannhardt (15:53): That's right. That's like an idea where I think dashboards really suffer because there's just a lot of volleyball communication.

Rob Collie (16:02): You mentioned before, the default expectation is they provide insights. Right. It sets the bar sort of too high. I don't say this to be critical of the people in the Reddit thread because again, we all make this mistake. There's also sort of a lazy expectation that the dashboard I build is so inherently obvious, what it does, how it operates, right, and that is not the truth. It is not optimized. In a way when you build a dashboard, you think it's kind of the only way it could have been built. That's not the case. As soon as you start interacting with your stakeholder, you go, oh, there were 400 ways to build this thing, and the one that I thought was the way is just one thing, and it's probably honestly the shallowest thing, the shallowest understanding of the space, but there's like this hubris, again, it creeps into all of us, and so then when they don't get it, it leads to a cynicism and a fatalistic view.

(16:55): They'll never live up to it, and you see that running through this Reddit thread as well. Now, there's a wide spectrum of stakeholders and users out there. Their levels of enthusiasm, their levels of interest. There are people you can't reach, but if we use the rule of threes again, there are some people who are never going to be able to reach, some people who are already onside so hard to begin with it doesn't matter, but that middle group of the ones that will be in with a properly tuned and optimized dashboard or set of dashboards for their experience, and it'll be out if it's not like that, that's the bulk of people. And you do not want to paint with a broad brush and say, the business users can never live up to it.

(17:33): It's kind of like the Mike Tomlin thing, right? When he's talking about other coaches talking about a player saying, "Oh, this guy's never going to get it," and Mike Tomlin says, "We see what you're really saying is your teaching is failing." The internal locus of control view of it is you have a responsibility here. If your business users aren't getting it, it's incredibly likely that you're at least partly at fault.

Justin Mannhardt (17:54): Yeah. Even with IT teams, for example, and I don't mean to poo, poo on it teams, but they'll say, "We'll make the data available. Then the business, they can do whatever they want to do with it." Some of that's in this thread, sort of the abdication of responsibility for the dashboard. It's like, give me my data warehouse or my semantic layer or whatever and then give the, just ah. Let them have their own ad hoc tool. That's like going backwards from where we are today with dashboards.

Rob Collie (18:21): The caricatured view of what's being pitched in this thread at times is let analysts be analysts. Even I don't want to do that. I want to start with an existing report and duplicate it and lay out all those goddamn tables again, et cetera, because usually I just want to riff on something that's already there. So give analysts the analyst experience and give everyone else slide decks.

Justin Mannhardt (18:43): Yeah.

Rob Collie (18:44): And guess what? I think there's absolutely a place for slide decks.

Justin Mannhardt (18:48): You're a man where PowerPoint is near and dear to your heart.

Rob Collie (18:52): I do weave, I create tapestries. PowerPoint is my medium at times. By the way, one of the things that annoys me the most about making PowerPoint decks is that if I've got a great Power BI visual, the way that storytelling works best is if you can animate it, click by click, right? How the hell do I take my Power BI visual and animate it? I want to build a click next in PowerPoint and have the trend line show up then and paint left to right. I want every last one of these things to be an object in PowerPoint that I have control over it. So often what'll happen is I take a picture, screenshot of my visual or visuals, and then I go and I freaking trace it in PowerPoint. I make the bars in PowerPoint. I draw the dotted line of the trend line in PowerPoint.

Justin Mannhardt (19:37): Rob, you'll go to great lengths to tell a good story, so I'm not surprised.

Rob Collie (19:41): We don't want to get caught in this world of thinking that it's like analyst tools and then slide decks. The thing in between, dashboards still really, really important. And a lot of the complaints, rather than casting them as failures of the person who built them or failures of the users, we should come up with a more neutral phrase like poor fit.

Justin Mannhardt (20:01): It's like a product market fit thing.

Rob Collie (20:03): Yeah. Do we have the time and energy as an organization to tune that? Do we have the political will to do it? Imagine as a dashboard developer, you show up at a stakeholder's office because you want to watch them. You want to say, "Hey, I want to learn more about how you use it and what your struggles are." It's a very well-intentioned and authentic goal. You come in there, the stakeholder might not want to do it.

Justin Mannhardt (20:26): Yeah, I don't got time for that, Rob.

Rob Collie (20:27): Get out of here nerd.

Justin Mannhardt (20:30): Yeah. Can't you figure this out?

Rob Collie (20:33): Imagine if the executive leadership was the one saying we had to do it.

Justin Mannhardt (20:38): Right.

Rob Collie (20:39): The dashboard builder in that situation shouldn't be authority, right?

Justin Mannhardt (20:46): That's right.

Rob Collie (20:47): And so ugh, we've got work to do as a species.

Justin Mannhardt (20:50): Yeah. I think the product market fit idea works really well in this conversation and some of the comments that one of them's like, "Well, a lot of dashboards just end up devaluing it. End up being these maintenance metrics that you monitor." I'm like, "That's actually really amazing. That's a great thing."

Rob Collie (21:09): We really needed to mature to this point to have the luxury to make fun of this.

Justin Mannhardt (21:13): The insights word I think maybe contributes here. I am noticing like, oh, I've got a problem in inventory or a problem with a customer or a problem in the shop. That's really important, that you have those tripwire type systems. The idea that insights come from other places, well, you might very well be right, that some of the real Eureka moment level insights come with someone wrestling with the data in those copied pages or doing new calculations or even wrestling around with it in another tool. That still happens. Even when that insight is figured out, most roads lead back to the dashboard.

(21:52): And so I had this idea that's always helped me of understanding the type of dashboard I'm trying to create and the maintenance metrics that's in a type of dashboard. I use it all the time every day. My team, we look at it helps us. Sometimes it tells us everything's fine, and that's great. That's all we need to know. Then there's this idea of a project specific dashboard. We were very curious about a specific problem or a specific opportunity, and we needed to analyze it and understand it. And there might be a dashboard that lives for just this moment in time to help us understand something, but I think the third type of dashboard, which is the thing I don't think really fits anywhere, which I think is the source of a lot of the criticism in this thread are the dashboards that try to be everything to everyone, all the questions like the Swiss Army knife, all the answers, all the pages, all the charts.

Rob Collie (22:43): The monster now, right?

Justin Mannhardt (22:45): Yeah.

Rob Collie (22:46): It's your opponent, not your friend.

Justin Mannhardt (22:49): Right.

Rob Collie (22:50): Circling back, I think the next evolution in this stuff, we've talked about this a little bit, is the truly natural interface to a data model. It does involve dashboards. The answers that you get are going to look like dashboards. Maybe it's a pre-built dashboard, maybe it's not. Maybe it's one that's been generated on the fly. I was relatively computer illiterate in the late 80s, early 90s. I'd stayed away from PCs. I had an Apple IIc that I'd had forever and stuck with that. The IBM PC format I didn't know much about.

(23:26): I remember when I'd go over to my friend's house and we'd want to play a game like Civilization on his PC. I was always struck by the fact that we'd sit down at the screen, the DOS prompt, and he would type CD space, name of the game, and then after you press enter, then he'd type name of the game. Change directory into the civilization directory and then type civilization because that Civilization.exe and that launches the game, but I didn't know anything about it. I'm just sitting there going like, why the hell does this thing make you type CD space game name, and then type game name? I'm like, this is bizarre. I just want game.

Justin Mannhardt (24:03): Right. Play game.

Rob Collie (24:04): But you had to translate your intent. Play a game into the computer's systems framework of I've got a log. I changed into that folder. We didn't even call them folders back then. They were directories. We didn't have fancy windows with this graphical icons yet. But anyway, there's something going on here with that too. Why can't I just walk up and say, I want to know this, and the system goes right you are. Here it is. Here's the dashboard that gives you that, and by the way, wow, this would be really amazing. I've pre-selected certain things so that you see exactly what you're looking for, teach you to fish in a way, right, in the same way that macro recorder in Excel did a lot to teach me how to write VBA in Excel.

(24:43): If there isn't a dashboard for it, building a visual on the fly in response to your request, as long as the way that your questions worded can be appropriately translated into the structure of the data model. I look forward to the day, not that far off, where us authors of data models and reports are putting time into making sure we're capturing all those semantic equivalences, the 18 ways people say the same thing. And also building constraints into it so that the computer knows how to generate sensible responses based on the data model. Right. The metadata on the metadata, and then I think large parts of this debate just evaporate.

Justin Mannhardt (25:23): Yeah. That experience, which a lot of people pushing towards delivering that experience with what's happening with AI and everything. That is an example of a theme in this thread, which is there is a true widespread need for acute ad hoc analysis and question answering. In my opinion, there shouldn't be a dashboard already existing for every, because you and I don't know what we'll face in our business next month in our own company. We're spinning up new dashboards all the time for different issues, right? And so I do agree, yes, there's actually a really important need for that agility in that ad hoc space, and there's a missing solve for that to make it actually work.

Rob Collie (26:15): Here's a funny example that I think you'll 100% appreciate. It doesn't have to be deep stuff either. It doesn't have to be deep analysis. It just has to be the thing that wasn't put on any dashboard already. So I have a report, a dashboard page in the hockey dashboards for our league. I'm still calling it our league, even though I've moved away and I don't play anymore, and it's really a bummer, but that has the season leaders in every category. It adds some sense of interest and drama. If your team's been eliminated or whatever, you could still get the scoring title or whatever, but it never occurred to me to put a season leader visual on that page for penalty minutes. Who wants to lead the league in penalties? That seems just like, it's a shame stat, but in this week's iteration of the dashboards, I'm still part of the text thread every Wednesday night when they're talking about the superlatives, the extra things that they want to put in, the hand entered stats essentially. Right.

(27:08): And there's one guy who he basically leads the league in penalties every season. His name is Mike, and that's how this came up recently. I was like, I wonder if Mike is leading the league in penalties like he probably is, right? Yep, sure enough. But there's no report that tells people that. So last night that came through, like one of the hand entered stats is only evidence that Mike played in last night's game, 2.5 penalty minutes. He didn't appear in any of the scoring categories. Right. Mike's actually also super, super, super nice. It's so funny that he's,-

Justin Mannhardt (27:40): Just clumsy. Always tripping people.

Rob Collie (27:42): No, I just think he's got a little bit of trouble with impulse control.

Justin Mannhardt (27:50): Okay. Yeah. Okay.

Rob Collie (27:53): But anyway, so I've got a guy that left behind with a laptop and he's running the refresh every week and everything.

Justin Mannhardt (27:59): Yeah.

Rob Collie (28:00): And he's pretty technical. I still haven't had time to connect with him to show him how to put penalty minutes on this season's tab, right? And nothing about that's hard. It's just, oh, now we got to rearrange the damn thing. The biggest problem is I used every pixel of real estate and like, oh, when are we going to, ugh. Now there's all these things to talk about and everything. I'm like, I don't have time for that, man.

Justin Mannhardt (28:21): Well, even some of the features in Power BI. For example, there's a feature that allows an end user to swap out the metrics that are on a chart or to change the type of chart. There's nothing obvious about the dashboard design of the user that says like, Hey, you could do that. Then if they do know they can do that, like now I've got the field list. Oh, no.

Rob Collie (28:43): Run away.

Justin Mannhardt (28:47): Dim fact. What? No, I'm out of here.

Rob Collie (28:51): The art of building software, whether it be building Power BI, the Power BI framework at Microsoft or building the individual dashboards, it is very difficult, both technical challenge, but making it mesh with the humans is an engineering discipline. It is not the thing to look down on. In fact, I'd say it's even probably more of an engineering discipline because you got to deal with the most difficult things on the planet, other people. I had a manager at Microsoft that said, "Deciding how to make the software work is ordering a single pizza for 100 million different people." What do you want on your pizza?

Justin Mannhardt (29:31): I don't like mushrooms.

Rob Collie (29:34): You end up with an empty cardboard box.

Justin Mannhardt (29:36): Right. Yeah. You can't,-

Rob Collie (29:39): I'm gluten-free and,-

Justin Mannhardt (29:42): Lactose intolerant. We're allergic to tomatoes.

Rob Collie (29:45): Vegetarian. Someone's allergic to every single vegetable, and some people don't eat meat. You don't even get through the first 100 people. There is no pizza.

Justin Mannhardt (29:56): Because the ad hoc analysis and that more natural interface, the utopia is everybody can have a pizza.

Rob Collie (30:04): Yes. Yeah, what pizza would you like today?

Justin Mannhardt (30:06): But it's hard. I mean, organizations large and small on their maturity curve here, they end up in a state, if they've been doing it long enough, more than once at this point, we kind of realize we have a lot of dashboards. Which of them are good and which of them are not? It's never done. BI is not a project that you finish. It's always improving, iterating, changing, morphing the needs you had. The way we used to look at something two years ago, it's not the way to look at it anymore. It's not good or bad. It's just the contexts are different.

Rob Collie (30:41): Our business has evolved and changed, and outside conditions have evolved and changed, and also our understanding of things has advanced. So of course our dashboards, if we don't modify them, they're going to lag behind. Just the other day and we don't need to get into what the metric is. Just the other day, me, you and Kellen, were looking at a metric and realized, why have we been looking at this metric for so long? This metric is dumb. There can be good business stories and bad business stories that run counter to it. And by the way, Justin, did we go and remove those lines from our, no, we haven't. They're still in there. We need to just follow through. The courage of our convictions. Bye-bye. Get those out of that report. I don't even want to see them anymore.

Justin Mannhardt (31:26): Yeah, delete. And the focus on dashboards is, to come back to this thread, I would argue the dashboards usually aren't the problem. It's that product market fit, understanding the stakeholders. Is the software meeting the needs of the user?

Rob Collie (31:42): So you said the dashboards. I think they are the problem, but I think what you mean to say is dashboards in general aren't the problem. It's not the art form that's dashboards. It's the specific dashboards. They would need to change.

Justin Mannhardt (31:55): Yes, absolutely.

Rob Collie (31:56): And also there's frankly a little bit of education or enforcement. There's cultural change. Sometimes people just have to be told they have to use it.

Justin Mannhardt (32:04): Yeah.

Rob Collie (32:05): Right.

Justin Mannhardt (32:05): Yeah.

Rob Collie (32:06): It's the people, it's the dashboards, but it's not generally speaking, even though we've spent a long time here categorizing and listing all the ways in which dashboards as a form are limited, that they're not optimal.

Justin Mannhardt (32:20): Right.

Rob Collie (32:20): It's not the form, it's not the art form,-

Justin Mannhardt (32:22): No.

Rob Collie (32:22): That's the problem most of the time. Yeah.

Justin Mannhardt (32:24): When you were talking about sometimes people just need to be told to use it, I can't remember who it was, but there was a CEO. He was just all in on the Power BI initiative, and he realized the best thing I can do is essentially categorically reject any information that is given to me that didn't come from Power BI because he said, I will be the forcing mechanism, and people won't get the excuse of, oh, Power BI is not right yet, or we don't trust it, and that's kind of the thing that forced the issue to get it right, to get it working.

Rob Collie (32:59): There's a genius to that, right? Yeah. Because when the reports, the dashboards aren't able to answer the question that needs to be communicated to him, forces the conversation to get it fixed.

Justin Mannhardt (33:12): And it close the back door to like, oh, we're still using the old thing.

Rob Collie (33:12): Yeah. Every now and then you run across someone with that level of vision and willingness to follow through. That's just like, wow.

Justin Mannhardt (33:19): Yeah.

Rob Collie (33:20): Really impressive.

Justin Mannhardt (33:21): Yeah.

Rob Collie (33:22): Did that work out well in the end for them?

Justin Mannhardt (33:24): It did.

Rob Collie (33:24): You could also imagine that not working out, right?

Justin Mannhardt (33:26): Yeah.

Rob Collie (33:26): Because again, if you're just a jerk about it as an executive, it's not going to work.

Justin Mannhardt (33:31): Right.

Rob Collie (33:31): But you're holding the line firmly but gently and the rest of the orgs is able to live up to it. Just an absolute masterclass.

Justin Mannhardt (33:38): It worked out well because I mean even, you can apply this same idea to even ERP implementations. There's this messy period where you're building something new and it's not like the old thing, and people want to keep going back to the old thing. Kind of need some leadership to say, no, we're going to the new thing. We're going to embrace it for all its good qualities and bad qualities, and we're going to deal with it along the way, and you can really accelerate some progress when you get everybody on the same page with that.

Rob Collie (34:08): So now all we need is some daring visionary business leader to step up and tell their organization, you know what? We're going back to static reports and slide decks, and sure, we'll have a handful of people armed with blank pages and field lists.

Justin Mannhardt (34:22): It's funny, it took us this long a conversation to get to something like that. I'd love to see the company who like, you don't just get rid of dashboards and oh, now we're better. No, it's not [inaudible 00:34:35].

Rob Collie (34:35): Kind of reminds me of my old dare that I ran on the blog for years, which was all these companies that were advertising their BI tools as being a replacement for Excel, and I would challenge all of them. Okay, remove your export to Excel button. You're so much better, and you just replace the need, like just take it out.

Justin Mannhardt (34:53): Yeah.

Rob Collie (34:54): Let's see how quickly your company dies.

Justin Mannhardt (34:56): Yeah. Well, and even that, like that's such a first line question. How do we stop people from exporting to Excel? Wrong question. Why are you exporting to Excel?

Rob Collie (35:08): I actually find that most of the things I build, I do not export to Excel because I'm building them for my own purposes, and so I build it directly to answer those purposes, but when I build them for someone else's purposes, the chance that I'm missing something goes way up.

Justin Mannhardt (35:23): Yeah. There are times where I want to be in Excel with a short stack of data looking through the details. It's the ultimate flexible canvas for getting in the weeds on something.

Rob Collie (35:34): Never again, am I building a chart in Excel.

Justin Mannhardt (35:35): No.

Rob Collie (35:36): I am just not. I was talking to someone, let's just say, very influential from Microsoft and told him about what an absolute shit show it is trying to connect Power BI to Excel online.

Justin Mannhardt (35:48): It's awful.

Rob Collie (35:49): Just like, how? He couldn't believe it. He was just almost like stammering, right? He's like, "You've got to be kidding." I'm like, "Nope. I wish you were in a position to fix this." He isn't.

Justin Mannhardt (35:59): [inaudible 00:36:00].

Rob Collie (36:02): No. They didn't bother. The description was enough. So frustrating. That's the biggest obstacle. Right. The better charts are now 100% in Power BI, but I got this table of data in Excel that I just hand typed in or whatever. I'm just like, oh, no. How do I do this in a way that doesn't suck? It's just going to suck. Oh, here's another small rant. If you want to export to Excel to work in Power BI, you have to leave all of the floaty on object thingies turned on to pop up, right?

Justin Mannhardt (36:30): Right. Yeah.

Rob Collie (36:31): So as your mouse is flying over the dashboard, this rippling noise of visual, and I don't want that, so I discovered you could blanket turn that off. Or I'm like, hell yeah, I can turned that off before I publish everything, right?

Justin Mannhardt (36:43): I've always been so frustrated that that top bar of icons on visuals is only visible on hover because nobody leaves enough real estate on the dashboard in between visuals for it to come up and not be behind something or on top of. It's, oh my gosh, fix that.

Rob Collie (37:07): And inevitably when you're trying to click on it, as you move your mouse to the button to click on it, you've stopped hovering over the visual and it goes away.

Justin Mannhardt (37:15): Yeah.

Rob Collie (37:15): Just like your blood pressure rises.

Justin Mannhardt (37:17): Yeah.

Rob Collie (37:18): There's just nothing about those things that I like.

Justin Mannhardt (37:21): I've had to train myself because there's a visual on one of our reports that I look at a lot, but I always blow it up into focus mode. You have to click in the visual and then hover and then click.

Rob Collie (37:35): Rob built dashboards do not have this capability. I was just like,-

Justin Mannhardt (37:38): Just take it away.

Rob Collie (37:40): Nope. Nope.

Justin Mannhardt (37:40): What are these buttons for?

Rob Collie (37:42): Yeah, it makes you think that you don't know what you're doing right off the bat. You see these things and you don't know what to do with them, so your confidence in everything else just plummets. Whatever it gives you, it's not worth it.

Justin Mannhardt (37:53): Right.

Rob Collie (37:54): Turn it off. There's a hot take.

Justin Mannhardt (37:56): Turn it off.

Rob Collie (37:57): At the source. Turn it off at the source Microsoft. Make that the default.

Justin Mannhardt (38:01): All right. See here we've got a single down arrow, two down arrows, and then this one looks like a decision tree or maybe like a pitchfork. What do these do?

Rob Collie (38:11): Yeah. And they're so large and friendly. They're just big enough to cover things that I want to see. Just small enough to be like, I can't hit them with the mouse. Yeah. Kill it with fire.

Justin Mannhardt (38:26): Try again.

Rob Collie (38:28): We should jump in on this Reddit thread and say, look, what if we turned off all of those things?

Justin Mannhardt (38:31): Yeah. [inaudible 00:38:32].

Rob Collie (38:34): Okay. Dashboards are back on the menu.

Justin Mannhardt (38:37): But a couple concessions.

Rob Collie (38:40): Yeah, this was a fun one. Always down for talking about essentially what nuanced human factors and how important they are, and also it's not just that the users aren't capable.

Justin Mannhardt (38:53): Right.

Rob Collie (38:53): What can you do?

Justin Mannhardt (38:53): That's right.

Rob Collie (38:54): Again, with the caveat of how much support do you have? All right. Well, it's a pleasure.

Justin Mannhardt (38:58): As always. See you next time.