Max Performance in Power Query – Ep. 422
Mike, Tommy, and special guest Alex Powers (the engineer who literally writes the Power Query documentation at Microsoft) unpack how to get the most out of Power Query across all its hosts—Excel, Power BI Desktop, and Dataflows Gen 2 in Fabric. The conversation spans staging defaults, V-Order optimization trade-offs, query folding strategies, and why treating Gen 2 like Gen 1 is the fastest way to blow up your capacity bill.
Beat from the Street: Keeping Up with Fabric Updates
Before diving into the main topic, Mike and Tommy discuss their strategies for staying on top of the relentless pace of Fabric updates—April’s release alone was 65 printed pages. Both use personal projects as testing grounds for new features, prioritizing what removes friction from real client work over shiny nice-to-haves.
Key strategies:
- Build internal projects around each new feature to get hands-on runtime before recommending to clients
- Prioritize friction-removers — Features like materialized views and copy job improvements that fix existing pain points get priority over nice-to-haves like real-time analytics
- Think event-driven, not real-time — Most businesses run on-demand; data pipelines should be event-driven rather than constantly streaming
- The “Conductor’s Era” — Tommy shares an article arguing we’re shifting from building to delegating, with AI agents and multiple models orchestrated together rather than one tool doing everything
Main Discussion: Max Performance in Power Query
Guest: Alex Powers
Alex Powers joins the show—he authored the Microsoft documentation on Dataflows Gen 1 to Gen 2 migration and the scenarios docs. He’s deeply embedded in the Power Query engine team and came to, in his words, “brainwash everyone” about rethinking their Power Query habits.
Power Query Lives Everywhere
Alex reminds everyone that Power Query isn’t just in Power BI Desktop. It’s hosted in Excel, Azure Analysis Services, Power Apps, Microsoft Teams, and even when you paste tabular data into the service. With 30 million monthly active Power BI users, patterns that worked locally in Excel or Desktop don’t automatically translate to cloud efficiency.
The Gen 1 vs. Gen 2 Paradigm Shift
The critical insight: Dataflows Gen 2 is not Gen 1 with a new coat of paint. The UI looks identical, but the execution engine is fundamentally different:
- Gen 1 = ETL tool that outputs CSV files, can attach to storage accounts, became a consumable item (and people wrongly treated it as a database)
- Gen 2 = ELT tool that writes V-Ordered Parquet files to Lakehouses/Warehouses with a completely different architecture
Alex’s plea: stop copy-pasting Gen 1 M code into Gen 2 and expecting identical performance. The generations are not the same under the hood.
Staging: On by Default, But Should It Be?
Staging is enabled by default in Dataflows Gen 2, which lands your data in a behind-the-scenes Lakehouse before transformations occur. Key nuances:
- Small files — Staging auto-disables with a yellow banner when file volume is too low
- Warehouse destinations — Staging is required (no choice) because the engine needs bulk operations, not single-row inserts
- The CU cost — Every additional step costs capacity units; users should evaluate whether staging is necessary for their scenario
- Alex’s aspiration — Microsoft should auto-optimize these settings after a few runs based on observed patterns, rather than burdening low-code users
V-Order Optimization: The Hidden Cost
Every Dataflows Gen 2 output applies Vertipaq V-Order compression to Parquet files—the same algorithm used in Power BI semantic models. The problem: you can’t disable it in Dataflows yet (unlike notebooks where you control this per table).
Mike argues that if a table has no lineage connection to a semantic model, V-Order should automatically turn off to save CUs. Alex agrees but raises the tension: at what point do you add so many configuration knobs that a low-code tool stops being low-code?
Query Folding: The Star Wars of Data Flows
Alex describes the ELT pattern in Gen 2: data lands in the staging Lakehouse, then transformations fold back to the SQL analytics endpoint. The catch—every point-and-click transformation either maintains or breaks the fold:
- If it folds → SQL pushdown, blazing fast execution
- If it breaks → Falls back to the Mashup engine, dramatically slower
Tommy’s summary: “The path to the most efficient way is very narrow.” Alex pushes back—it’s not narrow, you will get to the end. It just might cost you a lot more CUs if you’re not careful.
The Right Tool for the Right Job
Alex lays out a practical decision framework:
- Foldable sources (Azure SQL, Snowflake, Oracle, Teradata) → Fast Copy in Dataflows or Copy Job can move data at scale
- Non-foldable sources (SharePoint files, Excel) → Power Query is still the easiest tool by far for ingestion
- Separate by purpose — Don’t put all 10 tables in one Dataflow. Split by update method: drop-and-reload tables in one Dataflow, append/incremental tables in another
- 50 query limit in Gen 2 → Spread across multiple Dataflows based on source, update pattern, and fold compatibility
- Copy Job + Dataflows → Use Copy Job for bulk moves, then Dataflows for the final transformations (merges, appends)
The “Deception of Being Too Easy”
Tommy lands on a core tension: Power Query’s UI makes everything look equally easy, but in Gen 2, not all operations are equally efficient. Unlike a notebook where you’d naturally research the most efficient approach, the Power Query UI presents all options without restriction—and patterns that ran fine in Desktop or Gen 1 can become CU-expensive hindrances in Gen 2.
What About Excel Users?
Alex advocates hard for the Excel audience (700-800 million monthly active users vs. Power BI’s 30 million). Most Excel Power Query users just want a cloud scheduler—copy-paste their M code and have it run on a schedule. Dataflows Gen 2 with cloud refresh schedules serves this need, and the latest Excel builds now have “Get Data from Fabric” built in.
Mike pushes for tighter Excel-Fabric integration: tables in Excel should push directly to Lakehouses, and a full Excel experience should iframe into the Fabric workspace. The opportunity to bridge these audiences is massive but underutilized.
Gen 1 Sunset Signals
Mike discovered a hidden admin setting (appearing around April 5th with zero fanfare) that lets admins disable Dataflows Gen 1 creation tenant-wide. Alex confirms Gen 2 has migration tooling—“Save As Gen 2” in the ellipsis menu, plus Semantic Link Labs functions for bulk upgrades. Gen 2 advantages include CI/CD support, VNET data gateways, and proper security through Warehouse destinations.
Transcript
Click to expand full transcript
0:16 Heat. Heat. Good morning and welcome back to the Explicit Measures podcast with Tommy and Mike. Good morning everyone. Welcome back. Good morning Mike. How you doing? I’m doing well. Just clipping along. Things
0:46 Have been ramping up for us. I think it’s been a slow part of the beginning of the year here for us just in general consulting, traveling. It’s it’s been just very busy already, but yeah, I’m liking where things are going and we’ve had some really interesting conversations around some stuff. I’m excited. next, this is actually this is the week of sorry, this is a recorded episode, so I’m trying to get my my weeks configured here. This is the week of Build. So, I’m sure there’ll be some announcements made at Build for Fabric and PowerBI. , we are pre-recording this
1:16 Episode because Michael is traveling. So, that’s my fault here. But stay tuned. We will probably digest more of the Build activities throughout this week. , and we will have some more episodes in the future talking about things that are happening at Build. Wasn’t Build the conference that fabric was actually That’s a great question, Tommy. It was because we did that live reaction. I’m pretty sure it was in May in the summer. You might be right about that. Build was very build is supposed to be more like developer
1:46 Centric and very like technicalheavy. u and one thing that was so I went to build last year and some of the presentations just felt a little bit too fluffy for me. It was too too salesy. I wanted more like show me how to do it, open up code, get, , write some stuff. Let’s show me like what’s what are the new technologies that I should be interested about. , I just have a creeping suspicion, Tommy, with all of the , effort Microsoft is providing around data agents inside the Microsoft fabric
2:16 E ecosystem. , you look at what Microsoft is doing, dude, there’s co-piloty things everywhere. Everything’s talking to AI anymore. And I I think , before we get to our main topic, I’ll just have one thought here and we’ll we’ll do the news here in a second. I do think there’s a level of whenever I’m I’m using I’m getting to the point right now where any program I use, any app that’s on the web, I’m almost expecting AI to be part of it in some way. And my understanding of what AI can do has really
2:46 Changed my mindset honestly, Tommy. Like I’m like the fact that I can generate images, it can rewrite words, it can rewrite text and things. Anytime I’m inputting text to various parts of programs, I’m just expecting the AI to be there to like rewrite it for me. Like I’ll put down my thoughts. Oh, but it’s it’s an expectation now. And I think maybe this was what Microsoft saw earlier on by throwing cop at everything cuz every other program out there is going to do the same thing. And now here we are. If you have a product that doesn’t have
3:16 An AI in it, you’re now behind. Let me do you one better here. I think not only do we expect that, but Mike, I think we expect it to have some level of talent or some level of output that chat GBT has. I’ll give you an example. Yeah, every tool has it. That’s an expectation. But I use Confluence build a lot of spaces for clients and they have an AI agent tool. Of course you do. Y I I I I don’t use it. I use chatp in my other tooling and then I’ll put into confluence even though it has
3:46 A tool because just because you have an AI tool doesn’t mean it’s going to be used thing. So I I I’m 100% with you and honestly I’m not going to be impressed in Microsoft build unless Aoon’s a data agent. Everything that’s said is the data agent Aoon thing. So that’s going to be where where I’m going to be the most impressed. Yeah, I we’ll see what Aroon comes out here. So I I I’m not sure what his analysis will be. I don’t really know, but I I I just feel like we’re we’re at a tipping point right now where
4:16 There is so many rich tools. So I I feel like also in my mind the idea of a single agent or a single AI to do everything is not what maybe what I thought that was going to do initially. I’m thinking now probably not probably there’s going to be multiple agents that are doing multiple different things and I think it’s going to be more important for us to get access to a large library of multiple agents holistically as opposed to only focusing on well I’m just going to use just the open AI one right I think I think
4:46 Each agent or each model out there is going to have its own specialties and certain ones will do better than others in certain areas and we got to be able to customize them we have to be able to weight them and I think that’s such a huge huge part now where that’s a true especially in in a for a service as an organiz for an organization or software as a service. You can’t just give me a model or an agent and say that’s what the organization is going to use. It has to be custom regardless if it’s the data or the instructions. I think we’re just getting to that point now where give me
5:16 The default but it’s got to be it’s got to be clumped. So anyways, I love that. Let’s get into our main topic for today. I’ll just give you the topic here and then we’ll hit the news real quick. So, our main topic is introducing advanced analytics to organizations. And already I disagree with the title here a little bit and I think we’re going to have a good conversation today, but advanced analytics and how that fits inside your organization. Now, I would maybe argue it’s not advanced analytics. We’re talking about data literacy maybe that’s what really the language I’m trying to use around this. A bit softer of a term because
5:46 I really feel like there’s people in your organization who know how to use analytics and there are others who don’t quite understand it yet. Oh, we’re going to have fun with So, I think I I think there’s a there’s an educational , as a company, as you’re being a data driven company, right, if that’s what you are or you’re trying to strive to be, I think there are specific actions, trainings, conversations that need to be held to start bringing your team forward to be more analytical. You’re not just going to hire people and know they’re analytical. they
6:16 They’ve got to be u educated or trained in various ways to then help their minds think about the basic stuff, understand it and then move on to more advanced analytics or other things. So that’s the topic. I’ve already got my boxing gloves on for that. So but Mike, I realizing in my older age now this the this is a scenario for you. I’m realizing first off I’m reading books now that you could not pay me 10 years ago to read where I I had no interest in them. I would find it despicable. You read books now?
6:46 Yeah. Like that’s I just I just watch internet. I just watch internet. I watch I watch YouTube and that’s like that is like my form of Well, you’re funny, Tommy, because you you say that, but like a lot of the books that people have written like usually there’s like a podcast or some video content around parts of those books and I’m actually consuming the video side of those content pieces, not actually the book itself. Well, , but it’s and it’s entertaining. Now, I’ll give you example. I read a book about the 10 years ago. I would like not even pick it up or walk
7:16 Past. It was a history of oil. Page turner now 37 where I’m like, “No way. General Patton said this.” Like, so just weird. I’m reading a book about Jamie Diamond. He is the CEO of JP Morgan Chase. Yes. And this scenario, there’s just a part of the story I’m like, I need to bring up this up to Mike and a data culture. So one of the things that he implemented was every month he insisted from each of his department heads he got 50 different profit and loss statements but they were all printed and
7:46 He would dive through them as a CEO and I’m like thinking about this well it’s all data and it could be a pageionated report but the scenario I have for you Jamie D you work for Jamie Diamond and he wants all these printed data these reports on a monthly basis. Do you push to say if we have PowerBI, hey, all this is available anytime you want in PowerBI, just take a look on your iPad or take a look on your laptop or do we adhere to the printed side
8:16 Of things? Because the theory with PowerBI is rolled a no like we don’t do printed because we have the data in PowerBI. But this goes back into that, but what actually happens in the real world? So that’s the scenario for you. You’re the data analyst for J JP Morgan. Everything’s printed. Do you push for this to get actually pushed to PowerBI and focus there? This is interesting, Tommy. So, I’m I have I have a couple reactions to this one around this one. So,
8:46 My first my first inclination is, right, it’s a CEO. They’re going to they’re going to want to do things a certain way. Jamie Diamond is also more seasoned in his years. So, because of that, he’s just used to seeing things a certain way, right? He wants that physical paper touch. She wants that that book of P&L statements, right? , but when you look at someone who’s in the financial industry, you live and die by the profit and loss of what’s going on, right? What products are selling well? What products are not selling
9:16 Well? And at the end of the day, the bottom line is are you making more money than you’re than than you’re losing or you’re spending to to do the business, right? So, I think I think this scenario is a special case because if you look at profit and loss statements, they’re very not regulated, but they’re very consistent and they have the very same information. Money coming in, money going out, money at the end of the day, like what’s what does all that look like? And so, I think when you are that level of CEO at
9:46 That top of the top of the echelon, you’re not you should not be worried about the minutia of all the little details. you should really be focusing on, okay, what are our goals as a company? And again, at his level, he’s interested in making sure shareholder value is continue to go up. So, whatever whatever the teams beneath him are investing in, he needs profit and losses on those items. And so, he’s looking at where we at right now and are we marching closer to what our year goals look like. So, I think this is a special
10:16 Case because of profit and loss. He’s not the one going to be digging into those details. And if one of those profit and loss statements is really off and not doing well, I guarantee you he’s not going into, oh, let me drill through to the next page and see the answers. He’s going to the individual who owns that area and says, “Your profit and loss is not where we need to be. This is not going to do what’s right. Go tell me what’s going on. You need to give me more information.” So, I think he’s really trying to figure out across the health of those 50 departments or 50 business units, which department
10:46 Is the one that he needs to focus his attention on. So all that to say, , what do we do for Jamie Diamond in this situation? I I would also argue here because the profit and loss and the financial side of things are very known quantities around what is expected. printed being a no no like a we don’t do that anymore with PowerBI. We just build the reports. I’m of the opinion if I was a if I worked for him and I had to produce some of these reports, not all of them, but a portion of them, I would for sure go out
11:16 And build a semantic model, I for sure would have what I would need for the next level of detail down on that profit profit and loss statement, maybe build reports, maybe build some other things, but I would for sure build that profit loss in a page report and I would just print it out for him. I’d have that whole sucker automated and just ready to go. I would not be ming around in Excel. I would have it all ready to go because that is every month or every week, right? As fast as he needed it, I would make sure I have that information done and ready to go. , I have worked in a company that
11:46 Takes the same approach. They would print everything out on Greenbar and had everything printed like all all of their stuff like all their sales information would go up to upper leadership and they’d have these long printed reports that they would hand think 150 150 pages of reporting and the team beneath that person had to like really figure out how to produce this stuff and there’s a lot of waste and checking and making sure things were right at the lower levels because a lot of it was being
12:16 Done like ad hoc in Excel. all these other things. So, we came in and actually modernized a lot by just adding models, semantic models to to aggregate and bring that data up to a higher level. , we still gave them printed reports. We still printed out the the information, but we were able to then automate more of that. And so, to your point, Tommy, right, depends on how Jamie Demi is getting these reports. There’s something to say for being efficient here and being able to produce that same report every single day of the week, right? or or
12:46 You’re producing that report after a week’s work of time for someone to produce the report that you can then go give to Jamie Diamond. So where I think the the value here for this is right give them what they want right I think PowerBI is flexible enough to give users pageionate reports or pretty interactive reports depending on your use case this seems like it’s a very specific use case around what they need but I would focus my attention on automating all of that now the idea here why I
13:16 Want to automate I think I want to automate because then I can explore other areas I could say let’s understand let’s start doing some waterfall statements. Where in this department is are we making more money? What was the trend of things? Are we seeing something change in the interest rates area that’s really impacting our bottom line? Like I would be doing a lot more like, , after I give him the P&L statement, I would do a lot more work to figure out what’s the reason why I’m not making my marks or why am I doing really well and trying to clearly communicate what that
13:46 Would be. I think as an employee, that’s where the real value comes from is like you understand your area really well. You have to be able to articulate the why behind that P&L Yeah. No, I I love that. And I think perhaps part of this too is unique for Jamie Diamond. First off, if I ever get hired somewhere where my CEO has a biography written about him, I’m not starting until I’ve read that book. That’s probably a good idea. That’s a probably a good idea. But he was a number cruncher and his big
14:16 Thing was going through and cost cutting, slashing. So I think to your I really agree with you because maybe 10 years ago I would have said no we got to push PowerBI because that’s the data culture but the thing is if there is especially when you’re dealing with the sea level there is something you have to adhere to and say fine the big thing here the big win is the automation you can have the conversation with him or with the CEO go look correct I know you want this every month we can print it out but guess what we
14:46 Can do this every week we have this automated if you give whatever the resource and budget because you’re obviously looking at this once a month, but do you need it more more frequent? Do you need this on a frequent basis? Would you like to search this rather than putting your finger and running through with a highlighter? These are all things that we can work on for you. Have it in the same format you want. Because the one thing that you don’t want to do, I think as a data analyst going in anywhere is trying to in
15:16 A sense be like a bull in a china shop like we’re going to change the data culture and you come in guns blazing because people have an expected format that they’ve looked at and goes in our topic too today where there’s a way people know how to look at data and if you change that drastically that’s going to do more harm than good. So to answer the question here, yeah, you’re keeping the format. Anything you do would be gradual. Anything that you go has to be an expectation that happens on a gradual basis. I’m
15:46 Not sure the profit and loss statement is really something where you put a lot of advanced analytics around things. Anyways, I think that’s just , , that’s also a very historical look on things as well. It’s it’s very, , here’s tabler. Here’s today and here’s here’s back in time, right? Like that that’s the historical look at things. Everyone knows how to look at a profit and loss to your point. Well, it’s also very regulated. So, this is also where I think regulation comes into place here because if if Jaime Diamond starts lying about his numbers for whatever reason, his costs are not right or his profit isn’t correct and someone’s
16:16 Not like they’re they’re inappropriately out his it’s his neck on the line. It’s his like his job. He could go to jail for doing wrong things at that level of the company. So, to that end, right, I think this is one of the reasons why you see a lot of the profit and loss thing. It’s it’s a known quantity. It’s a communicated standard and everyone in that profit and loss space understands and this is going to lead into our topic today. There is a certain language you’re com you’re communicating with the data and everyone knows how
16:46 To talk the same language of that data. So with this I think we should really transition into our main topic today which is how do you introduce and I’m going to call it analytics culture. Tommy’s going to call it advanced analytics into your And I I think this is going to be a really good topic. So Tommy, let me stop talking here for a bit. Go ahead. Give us some more concept here on what do you mean by introducing advanced analytics to your organization? What does that look like for you? Yeah, for context, me and Mike were already arguing about this before the podcast started. So I introduced
17:16 This to Mike and I said, “Hey, how about this first topic? How does a BI team get past simple bar charts and line charts to more advanced analytics?” actually part of this inspiration Marco Russo wrote an article about prao charts and actually introducing more advanced analytics using DAX. It’s like well my thought was how many people are wanting that? How many people are wanting the more advanced analytics cumulative gain six months rolling and or just custom visuals too that usually require what I would consider the advanced analytics
17:46 And how do we get others to adopt this approach? Mike, you already had an opinion here. here it’s like well I don’t call that advanced analytics that’s just a data literacy thing. So I think there’s two areas here that we dive into is let’s talk about data literacy and then let’s talk about that moving past because to me a lot of organizations man they are either living in Excel or they expect and have that same assumption just like what a profit and loss statement looks like what a
18:16 Report should look like and usually those are not what I would or what you would consider the advanced analytics I’m not even talking key influencers I’m just talking, , more than your simple volumes, more than simple rates. I’ll pause there. I’ll let you talk here and just let me get your first take on this. Well, , it’s going to be a good episode when Tommy and I start arguing about the episode before we even get on air. And this is going to this going to be a spark, so hopefully we’ll fly. So, this will probably be good. Well, let me stand back and say,
18:46 So I’m going to give you a bit of my past experience of like how I was trained to think analytically about some things. And I think this is a skill that is probably required by more people in your organization. I think the challenge is how do you get this skill into more people’s heads? That’s the that’s the issue that we’re dealing with here, right? So, there’s a couple things that I think companies don’t emphasize enough on and they and some that do emphasize this well encourage their employees to to really create value from their data. Okay.
19:16 One of them is I like this idea of being able to have everyone thinking about analytics in a consistent way. One of this one of the things I like looking at is the IBCS standard. International Business Communication Standard. IBCS is the name of the standard and it’s like a it’s like a nonprofit organization I think that has set up this idea of like here’s how we should think about good. So people have studied this. I think a lot of times we build visuals, we build reports, we think you have to come up with everything from scratch. Actually the answer is no. There’s
19:46 People way smarter than me, way smarter than Tommy who’ve already thought about what’s the best way to represent year-over-year change. What’s the best way to represent comparison? What is the best way to represent a profit and loss statement graphically? Right? We don’t have to creatively come up with everything from scratch. So, one thing is where do your people go to learn this stuff? like what does your company have in place to if someone desires to go a little bit further above this where where do they go? Do
20:16 You have a class? Is there something you point people to can you run people through IBS certification? I I I think there’s a lot of times we’re just given Excel in a company and say here you go and then you learn on the fly as you go through things like what is it valuable where’s analytics come from but instead I think pushing people more through these analytical spaces actually helps people think more critically around their analytics so one I’ll just point out is where are you getting your education from people have made built and studied
20:46 This IBCS is one place you can do this another area that I think businesses are very weak and I this plays well into PowerBI is the level of automation of things right so even though we’re talking about communicating through analytics advanced analytics organizations if you’re spending all your time just pushing data between Excel files when are you having time to spend any effort around learning advanced analytics I would argue you’re going to have less time regardless whatever it means right you’re either going to spend more time outside
21:16 Of work to learn this stuff you’re going to spend more time here and I like to think of this as a communic communication standard across two different people. Like if I know how to build this really cool IB IBCS chart and I give it to you, Tommy, if you can’t read the information in there and understand what I’ve written, then I’ve I’ve failed. I’m talking a different language to you and you don’t even understand the language I’m speaking. So to me, there’s there’s that exchange there. And then automation is something that we need to put in place here to help people have space to go learn and educate
21:46 Themselves on these things. Those are my two initial points. So, I’m gonna I’m not even trying to I know when I’m ready to argue when I’m twirling my Surface Pen in my hand, and that is happening at a rapid rate right now. So, right off the bat, Mike, I this is something Yeah. So, this is something that I realized very early on, and I don’t remember if someone told me this, but it was very apparent to me. As much as I love analytics, and I think both you and I have a little a warped view because Mike, you
22:16 Took the What was the black belt course you took? I I don’t remember the name of it but it a phenomenal course yeah on that and we’ve obviously both had our education we both have our way of looking at data and also understand the impact of data well here’s the here’s the reality of the situation the sales manager the marketing manager their job is not to know advanced analytics or to that’s not their education their job is focused on their sales reps to meet their corda or
22:46 To have these campaigns be successful ful data might be a part of that but their responsibility is not to understand that data literacy or have that advanced analytics by default. You can argue that most organizations should but my counter to that would be that’s such a small percentage of most organizations that have such a healthy data culture. Like how many organizations do you walk into that have such a high and mature data culture there? To me it is
23:16 Very very low. more so in organizations again my job as a sales manager or operations manager is my is the team I manage the quota I’m supposed to do reports may be a part of that but advanced analytics is not so with that in mind I’m just going to say this advanced analytics thing and again I’m going to define it here and we can talk about that first to me this definition of an advanced analytics this whole concept here to me this is a
23:46 A growth of the normal metrics that one would look at. If I look at my sales quarter and I look at this in a bar chart, advance analytics is simply looking at maybe a predicted outcome. It’s something that’s not just the visual itself, which is the cognitive load, but it’s a way of looking at that same number. If I look at call rate, if I look at my open rate, whatever that may be, it’s something attached to that that again is going to take more effort and it’s not part of that standard lexicon of
24:16 The user. So that is going to be my definition of what we call advanced analytics. It’s not just the visual I use, but it’s also how we’re looking at that same metric. I’m looking at it from a different side of the box. For a lot of teams, you are now breaking their assumption of what a report’s supposed to do. Most organizations, and I’m gonna putting a bucket here with most, most organizations expect a report to show me my number, where I’m at, where I’m going, and
24:46 That’s the default of a report. Anything past that is out of the norm. It’s past the expectation. Most people are not going to go, I’m gonna figure this out. I’m going to discover this. Most people are going to say, “No, I want to see this, know this, move on.” Advanced analytics, it takes a little more effort. So, that being said, Mike, this whole idea with advanced analytics, and maybe you may disagree on my definition, the idea here that we’re taking your simple numbers and
25:16 We’re looking at them a different way or that’s going to require a little more effort. How do you see that in organizations or in a sense people willing to do that? So here’s here’s I’m I’m going to go back to your example here. I’m going to re unpack this again. So your example is I’ve got a sales manager. They’re worried about the sales performance of their team. and they’re they’re trying to figure out where is the best opportunity for them to spend their time. Right? So I think of this as a problem of optimization in this is in this situation, right? We
25:46 Have x number of sales representatives selling x number of products. There’s two ways you can sell more things. You either take the existing sales team you have and have them sell more or you hire more salespeople and have them sell more. There’s only two solutions to this one. I don’t think there’s any another solution beyond what you have. You you have a limited amount of resources on your team. Now my my thinking here around so using that scenario I’m thinking about okay you define advanced analytics as like something that’s maybe more
26:16 Predictive or something that’s not necessarily a bar chart or a line chart. It’s something else more complex in nature. And I would argue your statement around that analytics leader or that sales leader of that team. He says you you made the note of they don’t need to know advanced analytics. They just they need to identify the outcomes and how they get the job done. Right there I think is the miss. I think that’s the missing data point here for me is because even if I am if I can communicate the
26:46 Most elaborate predictive cool interesting report whatever that may be let’s just imagine just on theoretically if this report is so advanced that it says it it basically tells you what to do next if I can’t communicate that information to the person if they can’t read the visuals that are there I’ve missed the mark and so even back to your point here So, , you’re you’re saying the word advanced analytics. This is where I think I sorely disagree with your point on this one. It’s not advanced analytics. This is data
27:16 Literacy in your organization and is us as the team developing and making the information that could be in multiple forms. but even if it is or isn’t in that form factor, I’m looking at it going, , it’s not it’s not predictive. like it it if even if you have just standard KPIs that are being built in front of people like again if they can’t understand and interpret what we’re trying to give them you you’re basically it’s missing
27:46 It’s all gone like it’s not it’s the point isn’t even being made at that point so that there’s a thing that so I just feel like I’m trying to communicate numbers analytical things and if I don’t have that same understanding with that leader then we’ve missed the mark and that’s not advanced analytics that’s just data literacy and that’s what we need to Your point blew me so away that my camera broke. That’s how good your point was. Like I’m done. Tommy like my camera who thing can’t handle it. That information couldn’t even process my computer. No.
28:16 So this data literacy thing and I this is where I want to dive into because I really think you’re you’re to me hitting home here. A long time ago in a galaxy far away, we actually had this conversation about data literacy. Yeah. Do you remember the three things? I don’t I wish I remember the author who talked about this, but there are three things that define a good data literacy in an organization. It’s the ability for a team to listen to data or read data to talk about data. Do you remember the last
28:46 One? I remember argue with data. So a this author the study basically looked at teams and said if we’re going to define a healthy data literacy at an organization they need those three elements to read data to listen to data to argue with data. So to your and this is where I actually do agree with you regardless if it’s advanced analytics or your normal numbers if you don’t have a team who can look at their numbers or look at regardless if
29:16 It’s a part a pie chart or a spider chart I don’t like whichever one it is if they don’t have that ability to be able to ret talk with their team about it or again argue with it it’s a non-starter it’s a non-starter on whatever you try to do now but also Let me let me just pause you right there real quick. I want I want to I want to drop down on that point you’re making. So break my camera again. No, no, I’m trying not to break your camera again. but I but I agree with you Tommy and I think also to your point here in this right when we’re
29:46 Talking about data literacy around I can send you information but even having the same mutual understanding of what is a customer, where do they come from? what do we what do we define our products? there’s some simple understanding of just data literacy elements that if you don’t put those in place, the organization just can’t figure out like you’re not going to get anywhere. And so we’re going to what happens is there is there is so much data coming in anymore from all the different tools. We have so much
30:16 Data. The challenge has now become how do you discern which action should I take that is going to drive the right outcomes? there’s so much data and so this is where I like your idea of like arguing with data. A lot of times we show up to a data pile of information and say I have assumptions of what this may mean and you really do have to approach us with like where I find this works well for me is approaching data projects like a hypothesis a test and then come back with the results. What does that mean? And
30:46 We were doing a lot of hard things in in my corporate job a while ago where we were really looking at sales of things and the market would shift. The game would change underneath our feet. Someone would offer a discount. Someone would change a product. Someone would change the price on something. So it was constantly a game where we’re trying to figure out what everyone’s doing in the marketplace and figuring out where we fit and where our customers needed pos to position their products so that everyone gets maximum profit and we get like we sell a lot of these things, If
31:16 You come in with those assumptions, you have to come in, this is my hypothesis. I’m going to test this and does the data support it. You have to be okay to argue with the data. Yeah, it may be wrong. You may have to adjust your mindset around the data. And if your if your company doesn’t have the culture or the the rigor around being able to do this, then it falls apart. I’m getting goosebumps. This is so good. So, I’m going to turn it I’m going to turn this on a Ted Mike because we’re talking about the team not having the data literacy. And
31:46 I think that really misses the mark here because here’s the problem with that. We think because we have the semantic model and we have the metrics that we have the data literacy. I understand the sales. I understand their member count because I’m the one creating that. I think that’s so wrong. that is so far off the mark because that does has no impact on the decision making truly let me give an example and I’ll expand on this disagree wholeheartedly disagree but give me your example and maybe you can convince me
32:16 Organizations on what actually impacts them and this is the what I call the pain point or the pain thresholds one of the workshops that I do with organizations when we are working on new reports or building their metrics is I ask them and this was a sales team of a certain region and I said if you we were building their normal sales report and it was basic the quota and I asked them okay for the next quarter if you had a million dollars and a thousand data
32:46 Analysts working for the resources were unlimited what would your report look like what would you actually see on a screen that their answer was so far off anything that’s been created or anything that they’ve ever asked for and it was simply like, well, , a big thing for us is actually we want to know who’s going to no longer be a gold member. I was like, oh, okay. So, well, that’s never come up before and a thousand of your, , tickets that have come in. And this whole idea
33:16 Here about something that was actually very easy to create and this exercise I’ve done many times. , another example is I always go these pain points on what would keep you up at night. And there’s a whole workshop around that. Yep. It’s the same numbers, Mike. And this is we’re dealing with the same numbers, but I’m understanding the data literacy in their world, not just in, hey, we’re meeting 80% of goal. That’s fine, but that’s not
33:46 The data. That’s not me understanding as the data analyst what’s going to impact them and how they actually are interpreting the numbers because that’s the big difference here. So, that’s where I’m turning this on his head. But I’d argue that conversation you’re having, all you’re doing is teasing out additional metrics that they don’t even know they needed. Right? So the in the first one you’re talking about like, , look, , how many of our members are going to downgrade from gold to silver? How many members are
34:16 Going to stop subscribing to whatever we’re doing? How many members are we going to gain each month based on whatever XYZ marketing activities, right? That’s all churn analysis. This is all known stuff. And again, a lot of these things are like that keep people up. Okay? But again, I would go back to the fact that they’re asking for just simple reports around what actually happened and what’s an expected goal is not actually addressing the real problem. It goes back to my point earlier which is that’s a data literacy issue. Like we’re not able to actually articulate the
34:46 Problem correctly so that we get the right metrics so we can actually look at it. So what should be happening and can going to keep expanding on your scenario and why you tease this out with them is the churn analysis is a very well-known thing and that’s a very great thing you could throw towards advanced analytics. to get to the churn analysis. I would argue that’s more of an advanced analytical thing, right? That that is where the advanced analytics show up. You’re doing heavy amounts of statistics. You’re doing basket analysis. You’re
35:16 Doing who is like what are the characteristics of people that have left or not left. There’s a whole bunch of features you need to supply to the statistics model to get that out. And a lot of people will call this AI. People be like, “Oh, the AI will help like that’s that’s a machine learning project.” Well, yeah, it is, but it’s just heavy statistics done at scale, I think. And a lot of times, all you need is a a really good stat statistitian to figure out the probability of someone leaving or not leaving. That’s really what’s happening here. We’ve just fancied it up with all these words. So,
35:46 For that chart analysis, they know they need it, but they’re unable to articulate why like the report and how it needs to be had handled. So my vision of this would be is if you’re talking about that with a company, let’s put a score on every single individual you have across your whole company. Let’s score them all as likelihood of they’ll churn and then produce a list of those items and then action on that. So yeah, you have your standard like goals and other things, but you need something
36:16 In your hands that say this is the short list of people that we think will churn. It’s of our opinion to go directly to them and give them the, , the the the bonus, the the extra thing, the free month, the, , interact with them more because we know if we interact with them more and they open more of our newsletters, if they go to the website more, that’s means they’re becoming more engaged. So, I I think you’re you’re saying the right thing, Tommy, but I would still argue it’s a data literacy gap that you’re
36:46 Addressing. But again, this goes back to my original point, though. It’s not their job to come to me say we need a churn analysis. That’s a culture. No, no, it is their job. That is that’s smart as a sales manager. It’s our sales managers. But but that but that’s the point though. They’re not going to do that’s the point. They’re not the analytics person. If they can’t if they don’t even know what a turn analysis is, then we’ve got a problem. We’ve got a different problem because now we’re solving. So now the business is actually this is again back to data literacy
37:16 Issues. Like if you got if you have a sales team that’s not even thinking about these things and these aren’t even the questions that are keeping them up at night, they’re missing the mark. That’s not the right person for that position in that role. And what is going to happen, I’m going to guarantee you because I’ve seen it a number of times, they’re going to focus on all the wrong KPIs. They’re going to start hiring more people that produce less and not actually get out real value from their team. So what’s going to happen is we’re going to put false things in place because we’re not actually looking at what’s going to solve the
37:46 Problem. If anyone’s listening and you’re a sales manager ask analysis, stay there. You’re in a good place. And I is you’re in a very healthy place thing. But I I think my the point though is it was through our conversation that we grew to that. And I think again the data literacy I didn’t know that the turn analysis was important to them. It came from a conversation. Now I have that knowledge on what impacts them and I think that’s part of the data analyst. So I think a
38:16 Lot what I think the point I’m saying is a lot of times we blame the business for not having data literacy because we’re the one building the semantic models but it’s just numbers to us like I don’t know what like that churn analysis I don’t know when a number is at 60% that someone is vomiting or like sick to their stomach because the number is at 60%. That to me is data literacy. It’s not just I know how this is calculated. That 60% makes someone sick to their stomach and I need to know. That’s
38:46 My point though. That’s the point. This company doesn’t have the metrics they need to be able to do their job effectively. That’s my point. That’s the data literacy part. Like even if you communicated the churn analysis for a certain number or you communicated these in different different ways if if you can’t if that business leader can’t communicate like again I I think the argument here is a lot of business people understand what they need but they’re not able to articulate like after until they see it. Such a good point. Stick on that. So they know they know what they want but they they’re not sure
39:16 Like how to like get the report or the results or like get the information out. So to your point, right, man, if we could just make sure we have more gold people in the level or we had we had more like if we could only take 5% of our silver and bump them up to gold and give them a value ad that makes them want to purchase more. Who is that person? What is their persona like? How do we engage them? Like those are the questions we should be asking. And again, I’d go back to like this is all like that. That’s that data literacy. it’s been able to tie what actions are
39:46 Like what actions can we do right now to drive actual change in those numbers. So let me go back one just quick note here real quick. if you go back to what I got formally trained on which started a lot of my analytics career. I went to this company called Delta Associates and it wasn’t me. It was somebody else who said everyone on the sales and marketing team will go through this one. And so this is where I form a lot of my opinion around this. Right. You need to come in and bring education to your
40:16 Team. And to your point, Tommy, these are sales people that are directly talking to customers. We’re doing analysis. And , the sales team may not actually have the analytics background, but there’s someone behind them that an analytics person that’s building, grooming, curating the data for the sales team. So, we had that front-facing person that’s handling the relationship, which would then take needs requests from the customer and then bring that back to the analytics team, which is what I was on. But between the analytics team and the sales
40:46 Team, there had to be a really tight coupling of like understanding of where the data came from. And when something happened in the market and something was different and sales were down, the numbers were given back to us at the analytics team and we had to make the the assumption of like, here’s why your numbers were up or down. We had to we had to have this really tight understanding. So Delta Associates has this thing called business insights. It’s called black belt level one certification. It wasn’t called that initially when I started it, but essentially it’s black belt level one
41:16 Certification on business insights. And it was category management understanding how your company performs against other companies. And it was it opened my eyes to oh my gosh, there’s people who are doing really good jobs winning in the category management space. And there are metrics and KPIs that make sense that I didn’t even understand existed. And so this is this is my point though. My point is like there are systems out there in place that
41:46 Is you need to have your team all needs to be able to communicate on that same st standard that same wavelength. There needs to be something out there that everyone understands, hey we’re having a problem with churn analysis. This is important to the business. There has to be a full education thing either internally or from an external company where you’re saying churn is important. Here’s the why. Here’s why churn is important and here’s what we’re going to do about it. And so by having that common understanding across all people on the team now you can set up a process that
42:16 Actually you can measure and then you can stand back and say okay after doing this process is this thing helping or hurting and I really think businesses are in this never- ending game scenario where you just need to try something. If you think something’s not working come up with a good reason as to why you think it’s not working. Build something new. Implement it. Roll with it for a number of months. See how it goes. come back, evaluate, did that work? Do we keep doing it or do we pivot to something else? And I think honestly, just keep trying more and
42:46 More things until you get to something that is either working and the market will change out from underneath you. What works today won’t necessarily work 6 months from now. But going back to your but going back to your Jamie Diamond problem earlier, right, I’m going to go back to this one. Right. There’s there’s some things that are so universal for years, 50 years, hundreds of years, I don’t know. Everyone’s been looking at this profit and loss thing. It’s it’s so universal. It’s so fundamental to how businesses are working. Like
43:16 It just has to be there. So there’s some things that again we we need to have that basic level understanding of knowledge. We have to start with the very simple printed out profit and loss statement and then from there we can build the churn analysis. Then from there we can build where’s our pain thresholds what’s keeping us up at night what are these other things that we need to then manage and we actually have to build out new business we have to work that business intelligence muscle right we have to get to a point yeah we have to
43:46 Make the delivery become a part of our culture and then we can make the better KPIs and then we can make the better reports to make sure you move the needle I’m surprised that didn’t break my camera I know we’re at time so I’m going to give a quick closing thought here and for me Mike I think for the data analyst or our role. It’s not enough just to understand the language. Like if I moved to another country, if I moved to Italy and I took Rosetta Stone, it’s not enough just to know the words. I I think our job is the interpretation of the slang as well. And I think a lot of organizations, they have ways
44:16 They interpret things. To your point, Mike, there’s a lot of things that we need to try. We need to ask different questions. It’s not enough just to build a reports and understand the data. I need to understand the person. This goes back the old adage about empathy. anything we’re going to try if it goes to advanced analytics or it’s just a building off the current reports making the simple report impactful if we don’t have the literacy of the team or the business it’s going to be a non-starter so that is my closing thought love the conversation today I
44:46 Think when I when I focus on the the communication of data between people and the team I think I would call that data literacy right that’s what I would talk here when I think we talk about advanced analytics advanced analytics to me are things that are not necessarily visual based. They’re more of tools, solutions, Python libraries, things that are like more behind the scenes where I can say I can take in ingest lots of data, do advanced things with it, things that people just can’t do normally. And then again, it
45:16 Dovetales back into once the analysis of that is complete or that report is completed, then we have to step back into the data literacy conversation and say, look, these are the results and information. Can we all understand the results and talk about them and have a mutual understanding of hey churn is down 3%. That’s a good sign. Like do we even understand that churn is up 10%. We should have nervous and be sweating bullets about why this is happening. Let’s understand did something let’s understand the root cause as to why that is occurring. So I
45:46 Think this is a really good conversation. I like the debate here between advanced analytics and data literacy. I think that’s where we’re going with this one. we’d we’d love your comments and see where you think about data literacy and advanced analytics. Where do those fit in your organization? Do you have it? Do you need more of it? Let us know in the comments down below. Tommy and I will react to the comments as well. With that being said, we appreciate your listenership and we would love for you to share the podcast. Let other people know that you found this conversation valuable and interesting around what you’re doing and how you’re thinking about
46:16 PowerBI. Tommy, where else can you find the podcast? You can find us in Apple, Spotify, wherever you get your podcast. Make sure to subscribe and leave a rating. It helps us out a ton. And please share with a friend since we do this for free. Do you have a question, idea, or topic that you want us to talk about in a future episode? Head over to powerbi.tips/mpodcast. Leave your name and a great question. And finally, join us live every Tuesday and Thursday, 7:30 a.m. Central, and join the conversation on all PowerBI.tips social media channels. Thank you all so much, and we’ll see you next time. Heat.
46:59 Heat.
Podcast Info
- Find us on Apple Podcasts, Spotify, or wherever you get your podcasts
- Submit questions and topics at powerbi.tips/podcast
- Join the live show every Tuesday and Thursday at 7:30 AM Central
- Follow Mike, Tommy, and Alex on LinkedIn
