Databricks, Fabric, Development and Vibe - Ep.523 - Power BI tips

In Episode 523 of Explicit Measures, Mike Carlo and Tommy Puglia unpack the latest Power BI and Microsoft Fabric topics from the show. You’ll get a quick read on the episode’s biggest ideas, why they matter, and where to dig deeper in the full conversation.

News & Announcements

• No linked announcements were available in the episode description for this post.

Main Discussion

This episode covers the major themes, opinions, and practical lessons Mike and Tommy surfaced during the conversation. The transcript below captures the full verbatim discussion if you want the exact phrasing and context.

• Mike and Tommy react to the episode’s biggest Power BI and Fabric developments and explain what stood out to them.
• They connect product announcements to day-to-day practitioner decisions instead of treating the news as abstract roadmap chatter.
• The conversation highlights where teams can move quickly, where they should slow down, and what tradeoffs deserve attention.
• They share candid perspective from real project work, which gives the discussion more practical value than a headline recap alone.
• The episode mixes tactical advice, opinionated takes, and a few forward-looking predictions about what listeners should watch next.

Looking Forward

If this episode’s topics affect your current Power BI or Fabric plans, use the transcript and linked resources to identify one concrete change you can test with your team this week.

Episode Transcript

0:02 fix, fabric in I get your fix. [music] Explicit Explicit measures, drop the beat now. Hop and skip and feel the crowd. Explicit measures. Explicit [music] measures. Drop it loud. Good morning everyone and welcome back to the Explicit Measures podcast with Tommy and Mike. How’s it going, Tommy? Dude, it’s good to be back. We are back again. All right, jumping back into the episode today. We have a couple news items.

0:34 today. We have a couple news items. we’ll go through those together. But before we do that, let’s talk about the main topic today. Our main topic is going to be around Dataverse, Fabric, Fabric, and development, and your vibe. The vibe coding experience. And your vibe, what’s your vibe? What’s your vibe? Tommy, I also want to point out one thing. Have you seen this shirt, Tommy? Filter. Check it out. I didn’t notice it. Tommy and I’ve been talking all morning this morning. We got on the call and he hasn’t even noticed the shirt that I’m wearing. Oh man, that’s like when my wife gets a haircut I don’t notice it. I [laughter] I’m feeling the same guilt right now.

1:06 I’m feeling the same guilt right now. So I So I I looked you in the eyes. Tommy a while ago had said something. He was like, “Oh man,, it really means filters maketh the model.” I was like, “Yeah, that’s true.” When you think about a data model, it’s all about filtering and aggregating. Like that’s the the Yeah, there’s an old English expression that says manners maketh the man. Correct. Correct. And obviously because we’re nerdy I was like, “Well, I think there’s another way to put that. Filters maketh the model.” How do you filter Yeah, and that’s phenomenal. So the shirt the shirt has a vibe. vibe. Wow. It’s I got a nice big print on

1:38 Wow. It’s I got a nice big print on it. I’m trying a new company out. So we are So today is filters maketh the model. model., that’s going to show our true fans if you ever walk down the street someone goes, “I know what that means.” Does it Does it have any Power BI tips, Power BI Fabric No, it’s just straight Tommy’s phrase. I almost put a little like like, T Puglia underneath the bottom right-hand corner for a quote. Yeah, so no. Make my toilet paper? Yeah. No, we’re very we’re a very niche group here and

2:09 very we’re a very niche group here and we have our own inside joke. Anyways. I am I am honored. I’m humbled by that shirt. shirt. It’s really good. Anyways, that was our our fun little intro. So Tommy, you found an article here. here. this is Inside Notion on the Colossus. I guess it’s the Colossus blogging app. I guess that’s the the website that’s that’s coming from here. give us a little summary here. What’s Why What’s your infatuation with Notion, Tommy? Tommy? So I’ve talked about Notion a dozens of times on this podcast and it’s definitely ramped up recently. I would

2:40 definitely ramped up recently. I would say in the last two, three months. me just having to explain it on the podcast and a big reason is they have incredibly innovated over the past couple of couple of really last 45, 60 days around this concept of custom agents. And I’ve talked about it how it’s changed my own workflow. Yeah. Well, what’s nice to know is I’m not the only one who’s seen this incredible change. Notion has been around for a long time. It’s pre-AI. It’s And again, it’s this idea where you

3:10 It’s And again, it’s this idea where you have custom pages where you can put these blocks of words, databases, these modules so to speak is what Loop has been literally has tried to copy and paste. Mhm. but they took this incredible spin when ChatGPT came out, had this AI chatbot, but really I’ve seen these amazing changes and updates where now Mike, before I liked Notion, now I rely on it. And

3:40 on it. And the article’s talking about the back end of how that started. And there’s two major points here I think that are going to resonate with you from our own conversations. And one of the big things here is you and I have lost or are losing tools that we used to use. We’re trying to migrate some of our even existing podcast applications for things that are built because of the synergy. And I think more and more, Mike, you and I are getting to this

4:10 Mike, you and I are getting to this point where AI doesn’t make sense if it’s isolated. Yeah. And I I’m I’m coming back to this theme more and more because because you can use ChatGPT, great. You can use Copilot, great. But if it only works in that isolation of it that own context, it’s not super helpful. And the thing with Notion that makes it so powerful is how these custom agents relate to other pages, the other my notes, AI meeting notes that does the transcripts. It’ll talk to Claude. It will talk to all my

4:40 talk to Claude. It will talk to all my existing resources. And it has this synergy and it has this ex- web of context that Mike, like to like I said, has completely changed the way I work. And if they tripled the price today, I would go, “Cool.” Like it would not I would not even blink an eye on what this is subscription is because it’s essential in terms of how what my workflow is. And it’s neat because they talk about here even though it was a note-taking app,

5:10 even though it was a note-taking app, they in 2022 realized that, “Hey, “Hey, we’re an AI company now.” And what that’s really changed is their core principles and also two two major things and this is where it’s going to really resonate with you. This idea of what they call wartime mode. mode. They are operating always in this constant state of wartime. Constant urgency. They are iterating codebases and projects that they’re working on will completely get scrapped or

5:41 scrapped or Yep. for the sake of what’s really coming up or where the progress is. So they celebrate discards as progress rather than,, Yeah. Every decision that’s made move to to move forward is,, or anything you cancel and stop working on is a decision made to Well, that wasn’t really what we wanted. It was a good idea, but you learn something and you move on and you build something different that maybe is more of the right option. This sounds very familiar with an article I just read about how the developers speak about Claude code and how it works. Claude code has a very

6:11 and how it works. Claude code has a very similar aspect. They want to encourage like I think it’s like everyone’s afternoon is just build things. Go build stuff. The Claude code the app was actually coded with someone in an afternoon. They’re like, “Huh, this would be interesting. I should I should do something like this.” And someone built the solution or the app. And then from that they give it out so they Someone built a solution, you present your idea, and then if it goes well, you give it to the rest of the company. And then they just watch internal usage. If internal users use the thing that you just built,

6:42 use the thing that you just built, they’re like, “Okay, this is good enough. Let’s turn it into a product.” And then they go all the way and and finish out the idea. So this to your point, Tommy, like I think Notion here is internalizing this like rapid prototyping, try lots of things., there’s lots of less barriers around AI and code things. You just got to like move quickly and just go. Don’t don’t just sit still contemplating the perfect solution. Right. We’re now in a place where we can build lots of things quickly and then from what works, pick what works. Yeah, we we we are not having to do this, you

7:13 we we we are not having to do this, you we we we are not having to do this,, calculation in our head around, know, calculation in our head around, “Well, I can only create so many things because we only have so many resources.” It’s very much an iteration. But the real part to you, Mike, that I think you’re going to really hit home with is what they call it’s all the motto is it’s all jazz now. And it’s really their planning, their even organ org chart, and the roles at the company are intentionally minimized. And the reason why is why is the work is really emerging from ideas

7:43 the work is really emerging from ideas and prototypes that are organic. The you and prototypes that are organic. The, what are the customer pains? What know, what are the customer pains? What are things that spark people? So it’s not like, “Well, there’s a manager who manages databases and there’s this person who manages this.” Obviously there are skilled different areas, but the team’s self-organized. The ownership’s very fluid. duplication is fine. And they really have this jazz model building the application. The jazz model building. So really their internal model right now is it’s all jazz. Yeah, so the the the header I think

8:15 Yeah, so the the the header I think really resonates a lot with what you’re talking about with this idea. Planning is over. The org chart is over. Roles are now over. And it’s all jazz now. And I I think, this is not I think as companies start engaging with AI and interacting with it more, it’s going to open up the opportunity for more non-technical individuals to build more technical type solutions, right? I think this is what This is the same thing that Power BI reports I think were doing for

8:45 Power BI reports I think were doing for the ability for people to build reports. Like in the early days, right? Here’s this free program to go to go use and from that free program you can go build go build Power BI reports. You have a semantic model. You can load as much data as you want on your local machine, right? That’s That was brilliant, I think. I think that whole idea of like giving away a product for free, letting people understand how it works and and build with it really helped them become a dominant leader in this space. And I think to your point, Tommy, maybe Microsoft has a bit more structure and plan around what things need to be

9:16 and plan around what things need to be built. But those early days of Power BI, it was like every day it was like people building new features, getting stuff out the door, lots of listening to customers. It was It felt exciting. Like that was exciting to be a part of something that was changing like that and and happening that quickly. And you want to be in a place where the best idea wins. And I I such And the problem with even in our space I literally I the training I did last week and if anyone’s listening from that, shout out. was they were having the same issue that I’ve heard over and over again where

9:46 I’ve heard over and over again where they had a data team, data engineering team, but they’re like sometimes it’s a little difficult to talk to because they’re technical. They’re incredibly technical. Not that there’s anything wrong with that in any in of the imagination, but the where the business wants to go, where ideas go get stopped because of that technicality. And Fabric, you’re seeing the walls getting,, really torn down there because I don’t have to have a SQL database administration background to be able to create a

10:17 background to be able to create a database for a great idea to work with our product. Yeah., because before you’re like, “No, this would actually work except well, I need to go through licensing and then I need to get,, a DBA who’s willing to work on this project with me.” Sure. And hopefully we’re seeing that more in our space, too, where people, regardless of their role, if they have an idea of, “Well, “Well, couldn’t we have this talk to this? I don’t know how we can do that, but I know this would be amazing for our company, amazing for the team and being able to get that done.”

10:47 able to get that done.” I I Yes, I agree with these things, Tommy, and I think the other later on the article I was just talking about like this high level of,, some some challenging questions that people ask against the company, right? Well, what about duplicate efforts? When two people learn or two people are building the similar thing. What about How do you resolve a dependency two dependencies between two different people? What’s What is lack of ownership? Like if you give this, you ownership? Like if you give this,, you’re letting everyone to build know, you’re letting everyone to build and create and build what they think is next. Who actually owns anything? Does if no one owns anything, does anyone even own

11:17 one owns anything, does anyone even own anything? Like that’s These are the kind anything? Like that’s These are the questions I guess you would of questions I guess you would traditionally look at business and say, “These are challenges we had to work our our way through. What about wasting time on bad ideas? Like we only have so many people, so much time and talent. How do we How do we narrow down our focus so that we’re just building what people want?” want?” And I think these are great questions to ask, but they’re they’re talking a lot about like like the culture at their company. Yeah, right. right. And it talks a little bit around like lack of honor lack of ownership was one

11:48 lack of honor lack of ownership was one of them. of them. Lack of honor, yeah. Right. And then and they’re like, “Well, everyone has a lot of high agency here. Everyone the whole goal of the culture of the company is if you don’t have high agency, if you don’t have the ability to like self-task, self-align with something that needs to be getting out the door or something that you’re working with inside the product, you’re not a good fit for the company. Like that high level of agency is required inside the business.” And I think that’s I think what we’re going to see though is these kinds of personalities or these

12:18 is these kinds of personalities or these kinds of companies are going to find that moving quickly, lots of iterations, testing things out, seeing what works fast is going to that’s always been

12:28 fast is going to that’s always been an a success win for companies. Companies that move quicker than ones that move slower seem to be able to do better long-term. So, I I this is a really interesting article, Tommy. I got to really like decompress. Like it was a long read. So, I got like It’s a bigger read. partway through it, but it seems really good. I like the idea of this. I haven’t delved in very deep on Notion. I’ve been a lot more using agents in other spaces, but anyways, really neat article. Some neat concepts there. it would be neat to see how we could apply those

12:58 neat to see how we could apply those concepts to report writing and report building and like what does that look like for us now because I think I think we’re starting to see for me it has been ever since Fabric conference came out. It’s been AI a lot of talk, but this time it was a lot of AI and agent talking. Like The agentic side. The the creator creator creators with AI. So, anyways, let’s move on to the main topic, I guess. Yeah. I do want to and I want to keep baking on this because I think there’s a lot of this we can apply to the

13:28 lot of this we can apply to the what we’re doing, but yeah. And yeah, so we got a we have a great mailbag again. again. Another mailbag. this is going to be a a really I think maybe heated debate a little bit. Tommy, go ahead and give us the the mailbag for today. Real quick, you you’ll appreciate this. I got to give another another very quick shout-out to my father because I saw my father this weekend. We were coming to town for a family thing and the first thing he said to me was, “What’s with your eyes?” I’m like, “What are you talking about?” He’s like, “Your podcast. I don’t know where you’re looking.” I was like, “Oh, no,

13:59 you’re looking.” I was like, “Oh, no, not you, too.” So, he also notices that time to time my eyes tend to dart during the podcast, which I did tell him it’s a audio podcast, but people notice. I’m working on it. I have a lot to manage. I got the agenda, I got the mailbag, so sometimes I am not looking. [laughter] That does not mean I’m not paying attention, people, okay? That’s funny. Remember, I don’t know how many people are looking at my face when I’m talking. That’s great, but there’s not much to look at here, but I how someone else notices it cuz sometimes Tommy’s like looking at a

14:29 sometimes Tommy’s like looking at a different screen and I get it. Lots of screens. But like I try and put all my things on the screen in front of me where the camera is so I can like look at you and talk on the camera. I’m trying to make use of my five screens the most I can, but I get it. So, my own father You didn’t even say, “Hi, how you doing?” first. It was, “What’s going on with the eyes?” [laughter] Oh my goodness, Tommy. you would appreciate that more. I’m like, “Mike’s going to love this.” All right. So, yeah. Yeah. You had it coming, Tommy. You had it coming. Yeah. Let’s do the mailbag and this is from Merit. He’s from Denmark, so

15:00 is from Merit. He’s from Denmark, so Merit, I am definitely saying your name wrong. I apologize. “Hi, Mike and Tommy. It’s Merit from Denmark. First of all, again, thank you for the pandering. Thank you so much for your fantastic podcast. It’s a great opportunity for me to catch up catch up with the latest and greatest developments not only in the BI world, but also in AI. All right, Mike, we’re making an impact. Yes, there you go. I’ve been following you since the very first episode. Let’s go. Back when you were focused purely on Power BI and it’s really great and inspiring to see how

15:30 really great and inspiring to see how over time you’ve expanded the scope to include Microsoft Fabric, data architecture, people, process, and now AI-driven ways of working. Every episode feels well thought out. Fooled you. relevant and very connected actually [laughter] to what’s happening in the real world of data. I listen to every single one and always look forward to the next session. On a related note, I wanted to share some thoughts and also ask your perspective on how Databricks and Power

16:01 perspective on how Databricks and Power BI are revolving to some extent overlappingly lately. Oh, yeah. I work in one of the Nordic leading insurance companies with modern data stacks where Databricks serves as a lakehouse, DBT handles transformations, and Power BI provides powerful visuali- visualization and reporting. But recently with Databricks introducing Genie code and investing heavily in AI-driven analytics and BI capabilities, it seems that Databricks is trying to

16:31 it seems that Databricks is trying to close the gap on the business intelligence side. It almost feels like Databricks is moving from being just an engine for data engineering and storage to becoming a more complete end-to-end analytical platform. Somewhat competing with the presentation layer of Fabric, Power BI. Now with Databricks on mobile, the gap is even smaller. I’d be very interested to hear your take on where this is heading, whether you see Databricks and Power BI ultimately competing.

17:02 competing. Thanks again for all the great work and continue to share such valuable insights with the data community community, Merit. P. S. What a great intro you’ve gotten recently sometimes. Keep rewinding, just listen to it.” I guess he’s talking about the song. Oh, yeah. Yeah, that’s a fun song. Cool. First off, Merit, what a great What a great mailbag. Okay, there’s a lot to unpack in this one. So, we’re going to we’re going to have to go slow and pull apart some ideas. So, first off, thank you very much for the being a listener. We we appreciate having long-term listeners and and we do really want to try to

17:33 and and we do really want to try to have a balance of like all the different platforms and the things that are evolving. And this To be to be frankly honest, a lot of what we speak on here is what we’re physically going through, wrapping our heads around, just struggling through, building through in our own careers and our own businesses that we’re doing right now. So, a lot of the reason you hear us talking so much I I saw another comment on YouTube similar nature here like, “Wow, you guys talk so much about AI. Why When do you guys stop talking about AI and actually talking about like DAX or other things?” I’m like, “Well, we’re really shifting how we build these

18:05 we’re really shifting how we build these days and and we’re not building as much of these of these other parts of the language and other parts of the system. We’re we are building it, but we’re using agents and things to build on top of this. And so, And so, this is an evolution of what we’re interacting with with Fabric and Power BI. BI. And I’ve asked I’ve asked the same question, too. I think when we started really diving or being more heavy on AI, not just again, there’s two sides of this one I feel we talk about it. There is the developer AI experience, how I

18:36 is the developer AI experience, how I use AI to build the data platform, and then there’s also what we are doing for our customers around AI because most organizations who want any anything agentic or AI capabilities need to use their own data. So, we are trying to have both sides of the conversation and where I’ve really been led, Mike, is the it’s not going away and it’s going to be part of the business intelligence person. Like that career, that skill

19:06 person. Like that career, that skill stack how is in my head 3 years from now, I am expecting to see that on resumes, your ability to work with AI or to provide AI-like solutions for users, whether it’s through Copilot Studio or whatever’s going to be out there. There’s no getting around it and I know we’re going to keep talking about it and for those who are not using it at all, I’m sorry, but it’s the same fashion where the world runs on data and data runs and AI runs on data as

19:37 and data runs and AI runs on data as well. So, well. So, Yeah, I agree. So, let’s let’s unpack some more of this Databricks story. So, so there’s an observation here, Tommy, that I think is very very intuitive based on this question, right? What’s actually happening in the real world of data? And I think the the note here I want to point out is Data bricks and Power BI are now evolving and it seems like the the comment here around seems like Data bricks is moving more into the BI space, the visualization layer. I said this a long time ago, Tommy, probably a year or so ago. I said the

20:08 probably a year or so ago. I said the only real only real let’s think about Power BI, just specifically Power BI. There hasn’t been a lot of competitors to really come into the market since Power BI came out in the marketplace. There was Tableau, there was maybe a couple other ones, Qlik and some other programs out there. But once Power BI really gained some momentum, it started eclipsing all other visualization programs out there. And it was like, okay, you’re either like a Tableau shop or you’re a Power BI shop. I can’t tell you the number of migrations I’ve done from Qlik into

20:38 migrations I’ve done from Qlik into Power BI Power BI because that was the eating dominant force was move over to Power BI. BI. So, Data bricks has always focused themselves on So, let’s talk a little bit about Data bricks history. They’re the team that built Spark. That’s the team that that built So, they turned themselves into an organization, a company. Data bricks is that solution and now it’s used everywhere. Tommy, you use lakehouse, I like notebooks, Fabric has a solution to this one. And also around the same time when Data bricks was building this like data

21:08 bricks was building this like data engineering space with their solution, we see this thing called Snowflake come out. You’re like, oh, Snowflake. And it’s a mystery, like how does Snowflake work? What is it What is it all Well, it’s just Spark again, but with a different file format underneath the hood. It’s Iceberg versus Delta formats. So, the the argument has for Data bricks has been like, we’re the we’re a data company. Build our notebooks, build pipelines. This stuff they’re coming out right now, they definitely are using AI to help them build because they’re coming out with some really neat

21:38 coming out with some really neat features features very quickly. I just saw another graphical data engineering space for like business users where they have like tables, you drag them in and then you apply filter and then group and like very common SQL-like functions. It’s almost like a very graphical nature. That to me, I looked at it and go, oh my goodness, they just built Power Query. Like in a in a graphical interface. So, from that perspective, like Data bricks is is really, I think playing catch-up here

22:08 , I think playing catch-up here in the in the business arena, but they’re making really good ground very quickly. Let me just pause right there, Tommy. What are your thoughts? there’s actually a lot of similarities between our news article here and I think what’s going on with Data bricks because and I’ve actually seen that here. It’s interesting because Mike, honestly, you and I know the telltales. If you want to know whether a company is using AI to build their product, look at their development cycle. For example, if you and I start doing eight episodes

22:39 if you and I start doing eight episodes a week, odds are eight six of them are AI generated that we’re not even talking. But the point being like Data bricks is finally doing I think now things they probably wanted to do, those ideas, but were limited based on resources. And it goes back to this whole idea expressing in the Silicon Valley, evolve or die. Right? Where Data bricks has existed greatly as a data engineering platform. When you think Data bricks, you think data engineering.

23:09 engineering. But Microsoft was known as, misled, most people thought of as a reporting platform, it was more, but you ask 100 people on the street, they’re going to say Power BI was for reporting, bar charts and visuals. Microsoft has been trying trying to change that tune because I don’t think for these types of applications or these systems, you can’t exist in a rigid I do one thing really well mode anymore. So, we’re seeing that with Data bricks. We’ve been seeing Tableau try to copy

23:39 We’ve been seeing Tableau try to copy that and did a horrible job trying to expand their product through which, yeah. Did they try to do like a whole data engineering a little bit of like Yeah, like they try to get in that space and they and honestly, every data coming out of any system, there needs to be some level of data engineering and the more you can make that accessible to business users and and easy to engineer the data, the data, the better your reports will be able to come out as well. So, yeah, it makes sense. It needs to be it needs to be a a full solution around this. Part of this is I think Microsoft’s fault or Power BI’s fault because of the ease of use,

24:10 BI’s fault because of the ease of use, but I think the expectation for customers, for companies, is that when I connect and I’m buying something to with my data, it should be able to do end to end now. Where no one I I I Okay, again, I’m not going to make blanket statements here, but very little very small amount of companies are going to buy a data product that only does reporting and has an expectation that the data is already cleaned, I think today. Like, there’s not a lot of

24:40 today. Like, there’s not a lot of people using Google’s AI Studio or Looker Studio, whatever it’s called. Yeah, Looker. Yeah, Looker. And it does some transformations, but not really and it’s not great and But Looker’s good because it pairs with

24:54 But Looker’s good because it pairs with like the back end of what Google’s doing on like Google Analytics. Like so, they already have a great analytics platform. Yeah, exactly. There’s the platform that they have all the or Google Cloud or Google GCP or,, that is you pair that with this little thin layer of visualization and then now you have the solution. solution. Sure, right. But that’s again very small amount of people. Mike, you and I talked to a lot of companies, we do the networking, we have the user groups. How many companies do are using Looker but with their own databases, not

25:24 Looker but with their own databases, not because they’re already using Google Analytics or because they’re using BigQuery? BigQuery? I have not met one. It’s been very specific. Like if you’re doing if you’re aggregating data around advertising, then you’re using Google platform because it’s all Yeah, cuz all the data comes into the into their platform and use it from there. Yeah, double cents, yeah. Ad Yeah, Ad Sense. So, but again, I think there’s this expectation now where even with Data bricks, even with data engineering because of Fabrics,

25:55 data engineering because of Fabrics, like, wait, it doesn’t do,, reporting or my model, it only does the lakehouse? Well, I don’t want that because it should be easier and I think there’s this expectation that data should be easier in 2026. in 2026. What’s your thoughts there? I I think that’s right., I and I’m going to go back to more of like what Data bricks is doing, right? I I feel like this has been a tale of two worlds, essentially. So, Power BI kind worlds, essentially. So, Power BI came out first. We had,, of came out first. We had,, some attempts by Microsoft to build like a data engineering platform. Initially,

26:25 a data engineering platform. Initially, it was Synapse, all inside Azure. Didn’t really go so well. We really started picking up momentum when the data platform pieces started moving directly over to Power BI. So, let’s let’s, you over to Power BI. So, let’s let’s,, if if the landing destination for know, if if the landing destination for all of our reporting is going to be a Power BI report, why not bring this other thing called Fabric right next to it and make it all just work together, right? And I think honestly, too, Microsoft had a very unique billing s- solution here, which was compute units, right? This compute units thing, not a lot of other companies were

26:56 not a lot of other companies were saying, well, just buy anything you want and it’s all part of this compute units bundle and all it works. So, from that that was a novel idea or experience there. But But Power BI was like the front and center, the semantic model, the Excel sheets, the reporting. That was that was a very mature product that was there and we’re building other mature products in Fabric or in Azure that were being lifted, moved to over. So, not to say that, you moved to over. So, not to say that,, pipelines and notebooks weren’t know, pipelines and notebooks weren’t discovered and known item, it’s just

27:26 discovered and known item, it’s just they were in somewhere else in inside Azure and not in a Power BI world. So, contrast that to Data bricks. Data bricks didn’t really ever have a good visualization layer. They really focused on the technology stack of the Spark engine. And so, a lot of what they’re doing, they have two compute engines, they have a Python or Scala one and then they have a SQL engine. They They’re basically they’re managing like two engines, Spark and SQL. That’s what they’re the two things they’re Spark SQL is what they’re running. And then they’ve started maybe

27:56 then they’ve started maybe moving more towards, hey, here’s something for the BI people and the BI teams and there’s this serverless thing. You don’t have to worry about the cluster configuration, we’ll figure it out for you. Right. And I, back to my comment like what I was saying what I was talking about like a year ago was Microsoft’s biggest competitor will be Data bricks if Data bricks really steps up to the plate here and says, we’re going to provide a visualization experience on top of the data that we’re already engineering. Like that’s you need someone in this market to really push you to stay competitive, so

28:27 really push you to stay competitive, so you really hone in on what are feat- what are what are customers looking for? What are the features we’re looking for? So, while I think Data bricks is making a huge move, other side note here, I’m looking up Googling some of these other articles from Data bricks. And one thing I think is interesting is on their community page, they have Data bricks community, there’s now a section for MVP articles. So, Data bricks now has Data They call it,, Data bricks MVPs. That

28:57 it,, Data bricks MVPs. That looks very similar to like a Microsoft MVP, experts in the space talking about their product on their website. People can view, comment, like, and thumbs up that their articles they’re writing on their website., how Genie code is doing this, lake flow designer, you is doing this, lake flow designer,, opinions about that. know, opinions about that., this is it looks like a lot of things what I’m seeing Data bricks do is they’re copying a lot of what Microsoft’s doing. let me pause there. Yeah, I I I need to hear your thoughts or your opinion because would Data bricks move in this direction

29:28 would Data bricks move in this direction had Fabric not come out? Or are they naturally evolving because of what AI is giving them? And I’m going to expand on that before I I get your thoughts here because I want to make sure sure I’m clear here because to me, Microsoft realized at some point that they cannot survive with Azure. Azure was just not going to last. So, they built Fabric. And I don’t know, Mike, in 2028 or wherever we head from here, that Azure in that platform in the way it was

30:02 Azure in that platform in the way it was would survive. So, they’re like, “Hey, we need an end-to-end solution.” Maybe AI was part of that, maybe not. But obviously that stepped on the toes of Databricks in in a complete sense., they literally put PySpark in the lakehouse and data engineering in the product. Well, Databricks is like, “How are we going to survive if we have that?” So, or do you see it the other way where Databricks has always wanted to do these things, and AI is just giving them the opportunity to do so?

30:34 opportunity to do so? I think AI is accelerating this challenge at this at this point. it’s also a little bit easier for So, also thinking too, Tommy, like when you are looking at the the incumbents, right? So, Microsoft Power BI was kind right? So, Microsoft Power BI was like the the staple right now. And if of like the the staple right now. And if you’re Databricks, you’re doing the engineering work. you you could do a lot of that data engineering work, but you’re looking at the at the rest of the market, right? So, let me give you another trend here that appeared here that’s now causing a lot of friction in this area.

31:05 friction in this area. Power BI has always had this concept of the semantic layer, right? These data models. models. So, the the semantic model is this relationship between tables and measures. And you can define like a structure around here’s the metadata, the semantics layer on top of that. Well, Microsoft’s had this out for a number of years, and now recently you’re seeing a now a shift where Databricks is now stepping in and saying, “Hey, we now have this thing called the metrics view.” And it’s very similar. It’s defining

31:35 And it’s very similar. It’s defining relationships. It’s defining basically KPIs or calculations. It’s defining basically the same stuff that you have inside the semantic model, right? And what’s happening is I I I think initially it was Databricks was trying to attempt and they were initially, “Hey, throw an AI at your data tables and it’ll just figure things out for you.” And they really didn’t go very well, I don’t think. I I don’t think that’s ever a good solution, honestly. And there’s a lot of articles from the Power BI and the Microsoft side was like, “Look, we actually need the semantic layer to describe the tables.

32:05 semantic layer to describe the tables. Why are they there? What are the columns? How they get there? What kind columns? How they get there? What calculations we want?” Now, for me, of calculations we want?” Now, for me, the big difference is when you look at the Databricks world and these and these the metrics view, and you look at Power BI semantic models and what they’re doing, the strength of Databricks has been you can divide a semantic semantic layer of information across every table in your in your database schema. It’s it’s a huge definition. It’s almost like we have a semantic model with no data in it, but describing

32:37 model with no data in it, but describing everything and how it’s all related. On the flip side, we have Power BI and semantic models. Well, semantic models are really good at describing the semantics of a data domain, but when they get really large, they’re very difficult to work with. And so, we’ve we’ve moved into a Power BI world where we’ve made these domain-specific semantic models, but the data’s included in them. It’s importing, it’s caching. So, it’s just a different approach on how to serve these models. And what Power BI lacks is this whole like enterprise model that’s on top of

33:08 like enterprise model that’s on top of it, right? There there is no one semantic semantic virtual layer for everything that’s in your organization for all semantic models. models. Isn’t that what the ontology’s trying to be? I don’t know yet, Tommy. Like and and I I’ve tried to play with the ontology piece, like the Fabric IQ. I think that’s maybe a portion of what it’s trying to solve. but I have a very firm belief that Power BI needs to start evolving into we need to take a lot of these already built semantic models, aggregate them together into a larger no

33:38 aggregate them together into a larger no data-related semantic layer, and then we should be able to auto-build semantic models off that generalized semantic layer, that semantic model. So, I’m I’m becoming more,, pointed around that’s something that we need in the Power BI side, which is missing at this point. Do what ontology and metrics feel like right Ontology feels to me like a junior high kid who’s now 6’4 and you’re like, “I don’t know if you’re going to be great at basketball or just be gangly your whole life.” [laughter] We’re they still out? Like you could be really like,, college Duke,

34:09 really like,, college Duke, Kansas basketball player. You still got two left feet. Or maybe you’re, you two left feet. Or maybe you’re,, you’re just tall. Like I So, it’s know, you’re just tall. Like I So, it’s in that weird adolescent stage for me because to your point, Mike, I don’t know. Like I I can see what it where it wants to go, but I think we’re at a good point. Yeah, yeah. And I think Fab it’s it’s ontology is still figuring yourself out. It’s still like, “Okay, I got long arms. What can I do with these?” I think the question though is is Databricks competing with Power BI?

34:39 Databricks competing with Power BI? I think it’s competing with Fabric, but in no means do I see it competing with the Power BI world right now. Fabric, absolutely. Like from the data engineering, obviously from the semantic model, but to me, when you look at what makes Power BI Power BI truly successful, what it makes it truly the tool that it is today and the platform that it is today is the ability for anyone at the company to view, consume seamlessly through what they’re

35:10 consume seamlessly through what they’re already using with Microsoft. So, actually, case in point, I had a potential major project, but the company moved to Google. Like so, they’re in Gmail, all the things in Google, that’s which is totally fine, but there’s no Power BI there, right? Because they don’t have a light Microsoft licenses. And you realize very quickly in those moments, Mike, that the ability for if I have Office, I can have Power BI, and I can have the whole enchilada, so to speak,, as a

35:40 enchilada, so to speak,, as a consumer or a developer, is so powerful. So, even when you have these outside products like Databricks, and I’m calling it an outside product because again, you want to access it, it’s not through your Office,, ecosystem. It’s outside the ecosystem. It’s going to be very hard for Databricks to compete with that or any product to compete if you’re outside an ecosystem in today’s world. Again, this goes back to most licensing that companies buy

36:10 most licensing that companies buy expects some type of end-to-end, be it data, note-taking,, project management. It can’t just be an It can’t be just really good at project management. I’m I’ve met actually in the last 2 months, three clients who are moving off monday. com. It’s a great tool, but again, it only does project management in that space, and they don’t want to live there. They want to live in something that talks to everything easily. easily. And to me, just answering that

36:41 And to me, just answering that question or looking at the question that Merrick Merrick put together is yeah, it’s probably doing some things better in than Fabric in data engineering, but by no means, I don’t know how much it’s going to take out of the pie of the reporting in the BI side of things. What’s your thoughts there? Yeah, I think What’s your take? Really really the trick is this now, how do you manage this with multiple teams? And what tool what teams are these What tools are these teams using? Right? So, I would argue if you’re a greenfield

37:11 I would argue if you’re a greenfield company and you’re starting from scratch, right? You don’t have any existing stuff. You’re on prem, you’re moving into something cloud, and you’re going to make a decision. Right. I would argue the decision for you

37:22 I would argue the decision for you should be really focusing on like you could you could have a lot of decision points. Databricks does this well, Fabric does this well., Fabric does this really well, and Databricks does not do that well, right? You can go back and forth on a lot of these pieces. For me right now, and this is evolving, right? This is an opinion that is changing over time as the products and the tools change. This is a fluid movement, right? Right now, as I see it, to me, I like looking at this going, “Well, what is what is what shop are we? What company are we going to become?” become?” Right? And I and I ask that question is,

37:52 Right? And I and I ask that question is, “Are you a Microsoft shop?” If you’re already a Microsoft shop, you already have a lot of Excel users, that’s where your knowledge lives. You already have people using Power BI, or you’re using data and analytics from the Microsoft stack. If that’s the chosen stack of your company, you want to think about, “Okay, what’s my control plane? How do I control access to items and things and licensing and all those things roll up?” Sometimes people’s hands are already tied towards the Microsoft arena because

38:24 tied towards the Microsoft arena because we’ve already bought all of our employees an E5. Well, therefore, we already have free Power BI Pro licenses for every user in the company. Therefore, why would we not use this program that we’ve already paid for? So, in some situations, when you’re already heavily leaning into the Microsoft stack, I think you need to continue to lean into it and go harder on leveraging that stack for your advantage. Now, here’s where things I think get tricky. tricky., other team members and again, I’m to be very clear here, I’m going to be very transparent. I’m both a Microsoft partner and I’m a Databricks

38:54 Microsoft partner and I’m a Databricks partner. I’m a Microsoft MVP. I am not a Databricks MVP, right? So, I I’m I’m leaning more towards the Microsoft stack heavily, but I fully recognize as a developer, my team absolutely loves working in Databricks. The notebook experience is really good. It’s very fast. it’s a pay-per-use model, right? You you only pay for the compute that you’re using consume. So, if you have a little bit of data work, you pay for a little bit. If you have a lot of data work, you pay appropriately more of that data work. And you can make then the business decisions around how much

39:26 the business decisions around how much spend do we need to make. That’s a That is a something that Microsoft of that Databricks formed, Microsoft copied it, right? So, now you have Microsoft auto-scaling on Spark. So, I look at these again, most of the items between the two companies are becoming in feature parity. So, my opinion right now is what’s the control plane? What are you going to use to control how people get access to it? Do you want to manage a control plane in Fabric? Right. And in Databricks? That’s a good point. Or do you want to manage just a control plane in just

39:57 manage just a control plane in just Fabric? Now, right now, it makes sense to just pick one platform. Databricks is not making it easy to manage and delegate Unity Catalogs in the same way you would delegate to Power BI. Also, think about your consumption layer. How do you consume this data? And this is when I’m going to give a a big shout out to our product that we made because we we’re betting on both horses in this race. So, we have a product called Intellextus. Intellextus allows you to embed Power BI reports. It’s a white-label solution for building

40:27 It’s a white-label solution for building Power BI embedded reports. So, you want your own branding, you don’t want the customer portal from powerbi. com, you can use that solution for Power BI. We also recognize that AI BI dashboards are very powerful as well. So, you can also embed in the Intellextus product a AI BI dashboard. So, That’s a lot of right now, if you if you go into if you go into powerbi. com, you can render Power BI reports, paginated reports, and table building experiences with Data

40:58 table building experiences with Data Explorer, right? Those are what you can do today. do today. Yeah, it’s cool. That’s cool. That’s the rendering you get. You don’t get the ability in Fabric to render a Databricks report inside Fabric. So, there’s not one surface area of control, okay? People would have to go to a different portal somewhere else. If you add Intellextus on top of this, it changes the licensing model. You’re now paying for premium. You can have a whole bunch of free users down to much lower-level SKUs inside the Fabric ecosystem, but now you can embed a Power BI report

41:28 but now you can embed a Power BI report and a Databricks report right side by side. And so, now you could split this work across teams. Now, this is this is you can do it. It’s possible, but this is where you have to really think about the business goals of what your company’s going to be doing. so, let me just pause right there on the surface control plane. So, So, first, awesome thoughts, so I’m going to try to categorize these because there are a lot of great things that I really

41:58 are a lot of great things that I really want to mention here. The first one is the elephant in the room, and I cannot wait for whenever we solve this, but we got to do better than AI BI. It’s a lot. That’s not catchy. Here’s what I’m going to propose. I’m going to say ABI. There you go. ABI ABI reporting. a little ABI reporting. Anyways, but I I want to lean heavily First, I want to touch on the security part of this, where let’s also not forget you’re not just building chat GPT things. We are dealing with data, security around data, and the flow of

42:29 security around data, and the flow of data, which is essential to a company. And that whole who can access, how the data gets accessed, that that flow, regardless of what tool you used, that’s such such a big part of the plane. And I think what you said there, Microsoft right now to me has the advantage because again, it flows through Entra ID, flows through things that are already commonly known. What I do want to lean heavily on something that you mentioned, and I think this is going to be very very big for you and I and

42:59 be very very big for you and I and anyone in the data space here is the fact that you mentioned the ABI dashboard. See, I’m already using it. That someone can render this,, using AI, or the ease of use connecting to and transforming data using AI. using AI. Mike, Mike, from a personal note, I have now begun to just assume no one is going to hire me for my services around technical capabilities.

43:30 capabilities. And I think this is this is at least the way I’m viewing myself as a consultant, my viewing myself as a professional. Again, I’m going to say that again. I am no longer in the belief that I’m going to get hired purely for my technical skills alone. Really, because that’s where we’re heading, where if I want Jim from operations to help build data, that ease of use is going to be there. Where I see the true skill and where someone’s going to need the expert,

44:01 someone’s going to need the expert, because that’s what we’re talking about here, whether you’re an expert in Databricks or in Fabric, is there’s just the solution side, it’s the strategic side. Because you may be able to isolate and build a lakehouse Mhm. in Databricks or Power BI. Great. And you can do it without any consultant or training. You without any consultant or training., you don’t have to hire me or Mike, know, you don’t have to hire me or Mike, and you can do it. However, this, there’s the web of politics, there’s the web of the flow of data, there’s the data governance behind

44:31 data, there’s the data governance behind this, and there’s the actual adoption. Do people actually use it?, so there’s that whole technical layer that is becoming less and less and less important to know, where whether or not I’m certified, okay, but it’s more on what’s the plan and the blueprint. How are your things getting rolled out? Who’s aware of this? That I think is where the I’m at least putting my money down on where my skills are. And and really, to be honest, how I’m marketing myself. Because

45:02 Because I think you’re seeing both this redundant not redundancy because you use that word, this deprecation of the needed skill in Databricks. If I tried to use Databricks 8 5 years ago, and I didn’t have any PySpark experience, I was lost. You would be lost at sea. There’s nothing I could have done in the tool for it to have any value to me. But then we’re no longer there. And this more or less in the sense of Fabric, too, and I think

45:38 your point here around the idea of where we’re heading and where these tools are heading is there’s absolutely redundancy. Mhm. But there’s still a navigation side of this. So, I want to get your take on that on where’s the importance then lie in for the professional, whether it’s Databricks or Fabric? Let’s talk about some friction points here. I was able to land that because I got lost in that real quick. [laughter] So, let’s let’s talk about a couple architectures here and like some things that you can decide to build one way or the other. And I think this is just kind the other. And I think this is just important to note, like, okay, you

46:08 of important to note, like, okay, you of important to note, like, okay,, you can you can build know, you can you can build in a in a If you just picked Fabric only, or you picked Databricks only, you’re probably going to be able to get a lot of things you can get everything done you need to, I think. At the end of the day, you can get,, accomplish what you need to in both solutions now. If you would ask me this about a year ago, I probably would have said Databricks is much weaker on the visualization and the and the front-end side. just recently, I believe in March, Databricks released a mobile app that you can talk to Genie or talk to your data and get views and visuals back

46:40 data and get views and visuals back to your phone. That was something Microsoft has had for a long time prior to this. So, it it works. It you can get it on your phone, so everything you build will work there. And I I’d be honest, I every so often, I go to a couple reports and I check things on my phone. I’ll do that. It works for me. So, that was a feature that was on the Microsoft side. So, if you then feature compare one to one what’s going on here, Databricks has continued to bolster their their features on the AI side. Now, I went to a Databricks conference

47:10 Now, I went to a Databricks conference about 2 years ago. It was in Chicago, and I was just listening to what they’re talking about, and they made this really big announcement of we are going to double down the investment on the Unity Catalog. Unity Catalog is going to be like their place to help with governance and management, and everything will be built on top of this. You it’s like a strategic move for the company. I’m like, okay, we’ll see where this goes. Here we are like 2 years later, and Unity Catalog is becoming this really central place that makes a lot of sense for them. And so,

47:40 for them. And so, now when I step back and look at like the broader ecosystem of where Microsoft and Power BI are, I’m going to pretty much say they’re they’re almost feature parity on if you just picked that solution alone, and that was your only solution. They’re they’re pretty much there. Let’s now talk about the conflict around in mixing the two solutions, right? Right. Databricks Databricks’ strength is in the notebooks, Spark runtimes, and they’re coming out with the newest stuff

48:10 they’re coming out with the newest stuff all the time. They have they’re very innovative in streaming data sets and doing microbatching and like from their from my perspective, a lot of they had materialized views way earlier than than Fabric had them. And then the request came back to Fabric, hey, you need to get materialized views because look what Databricks is doing. So, again, you keep seeing these features like volley back and forth. Someone comes out with something innovative, the other company watches it, and they have an equivalent type product piece. So, So, when I like to merge these two, right? I think of right now my mental

48:41 right? I think of right now my mental model is bronze, silver in Databricks, gold, gold, semantic models reporting in Fabric and Power BI. That to me, that’s a good drawing line. Another thing I would also argue here, Tommy, that we talk about when we mix the solutions together, is data engineering should live on the Databricks side because Databricks has got a really good data engineering solution. Not that you couldn’t do it in Fabric. See, yeah. different, and it’s really like again, as an engineer who’s done a lot of work

49:11 as an engineer who’s done a lot of work in data engineering, it’s a really good experience. On the Fabric and Power BI side, you can still do that, but there’s always a couple missing features, or might be a little bit of extra friction, a little bit in some of the notebooks. But to argue now, looking at it right now today, I’d say most of that friction has been removed. Like it’s almost to me feature parity between what Power BI and Fabric can do with the with the the notebooks versus Databricks. So here this is where I see the friction. The friction becomes when you

49:42 friction. The friction becomes when you stream data into these tables in bronze and silver and you want to do the last step over to gold to distribute and

49:49 step over to gold to distribute and serve the data. I’m and by gold I’m defining gold is like the semantic tables. It’s like the dimensions, the facts. You’re trying to do as much star schema or, three or four fact tables with a bunch of dimensions on top of them. You’re trying to focus on star schema built things. That’s what by gold, okay? Because of that, right? right? There’s some technical pieces of streaming data that Databricks does not expose as public APIs or open APIs. And so there’s always a bit of friction between how do I get the tables

50:19 between how do I get the tables optimized out of Databricks and into Fabric with V V sort orders, sure. Because that’s for the VertiPaq engine. That’s for what we want to do for the semantic models, right? There’s there’s a little bit of secret sauce that Microsoft’s using to make the semantic models efficient to use DirectLake on these tables. So all this being said, the tools do work together. There just seems to be a substantial amount of friction right now when you transition from one tool to the other. other. Fabric can’t read these streaming

50:50 Fabric can’t read these streaming tables. tables. It doesn’t It doesn’t You can’t read them from the shortcuts. It’s not a possible. so that’s I think maybe a different methodology. One other key observation I maybe will point out here too, Tommy, is each team, Databricks and Power BI have a little bit different approach on how data is being served to the user. Okay? In Databricks world, you’re using metrics views and you’re using like a SQL SQL to make these metrics and KPIs. So you define SQL

51:20 define SQL to write the KPI in Databricks. In Fabric, in the semantic model, you define DAX. And DAX is a is like a mix of MDX, SQL, and Excel functions. It That’s what it was born from. Yeah. Right? So because of those extra functions on the on the DAX side, the DAX language, although be different than SQL, it’s actually a bit more versatile, I feel like, for building like reports, visuals, and being very dynamic because

51:51 visuals, and being very dynamic because the filter context of the measure is now generated when you drop the measure onto the visual, right? That’s something that SQL doesn’t really have the same concept to. So maybe Databricks will get there with their side of things, but if you look at the two worlds, my big picture brain says, we shouldn’t be defining any enterprise semantic model in either platform. We need a generalized semantic model that is these are our dimensions, these are our fact tables, these are relationships, and this is how you calculate this KPI.

52:23 this is how you calculate this KPI. Total sales, Sounds like metrics that’s Well, yes, to some degree, Tommy. Like it’s defining but you can define this by this measure is doing this. It’s adding this table up, it’s filtering by this and this, it’s grabbing this data. So the the measure is doing some capturing of of Right? Back to my shirt. Filters make [laughter] the model, right? So the DAX is manipulating filter context. It’s all about filter context. So if you if you really step back big picture this thing

52:53 really step back big picture this thing and say, the filter context is being captured by these measures. A lot of models I see these days are specific calculations that are doing lots of date time filtering. And there’s a lot of those measures, right? What do I want? You just drag it on the page, it just works. works. So because of that, you have two worlds. The Power BI side is running DAX. The Databricks side is running Spark SQL. And this is where it gets tricky because you can write more complicated metrics with SQL, I believe, than you can with

53:25 with SQL, I believe, than you can with DAX. But DAX does things that are very different than what the SQL side is doing. So we have this world of like you need to define the same calculation in both languages across the same tables and the same relationships. So that’s what I would like. I I I feel like there needs to be like a third middle layer in here where the semantic model can be like generalized for both tools. And it doesn’t matter which solution you’re building, you can get measures into both of these. And to me right now, with my

53:55 of these. And to me right now, with my customers, this is a major friction point, especially the ones that have Databricks and Power BI together. Sorry, final thought around this one cuz I’ve been doing a lot of work on this as well. Final thought. Clearly the Power BI side caches a lot of data into memory. Yeah. Like the semantic model is designed to import or DirectLake it, bring it into the memory of a machine, columnar store it, rip through the data, right? Databricks is also using columnar stores, but I don’t

54:25 also using columnar stores, but I don’t see Databricks doing like this in-memory caching type system the same way that semantic models are doing. So I just see the Databricks side right now, again, they doesn’t mean they can’t change, but right now a lot of their system is not building building a a in-cache memory solution like a semantic model. They’re just running SQL back against Databricks. And so as you interact with these dashboards, you’re just getting a bunch of SQL database queries or SQL queries back to the engine and it’s fastly returning these datas. datas. But even me, as a developer,

54:55 But even me, as a developer, when I embed cuz I have a IntelleXo solution that embeds both side by side, Power BI Power BI almost renders instantly. It caches that data. It comes up. It loads the data quickly. Boom, done. Very quick. The Databricks side, even though you click on it, there is a little bit of a warm-up, slower warm-up time for them to get the data and get the queries run to start showing things on the report. So when I look at them side by side on things, things, Databricks still is a little bit more of a lead time to get the datas rendered. And I think that’s pri- for me

55:25 And I think that’s pri- for me primarily, it’s not the caching thing. So let me just pause I said a lot of things there. We pause. I I I I don’t feel as bad rambling anymore, that’s for sure. But no, but honestly though, I Mike, I first want to say it was a good run, our 6 days of not of not disagreeing, of being completely on the same page. Like what, 6 days? It was like 1 day. Well, it was one podcast, but I’m going to count to count One podcast. Thursday, so about the 5 days that went in between, it was a good run. It was good.

55:55 good. There are two things that you said that are really one major point that I’m going to vehemently disagree with you on. And then one I’m going to tie it back that I think you’re going to I think will align. The first thing is you were like, well, Databricks is better for engineering. It’s only better for at this point in time, Mike. It’s only better for engineering for data engineers. I cannot disagree with this more with Fabric because the point of Fabric that we’ve talked about I’m seeing more and more. It is for the masses. It is so

56:25 more. It is for the masses. It is so much more much more for the teams that have a data engineering team who can’t get their stuff done, who have bottlenecks. Right? Because if I’m a data engineer and I that’s been my world, yeah, of course Databricks is going to be better. It’s my experience. But I’m not going to bring marketing or sales or even a BI team necessarily straight into Databricks because it’s a different environment. Where I have Fabric, sure. Does Databricks do a lot better both from back end of processing and user experience for a data engineer? Of

56:56 experience for a data engineer? Of course it does. That’s its thing. You like that’s literally you go see a guitar player, you’re not expecting them to play a piano. So that’s what it does, but Fabric, I will give Fabric its flowers in terms of its ability for a lot of data in a lot of use cases to be able to handle that where I can now circumvent the Databricks side of things or circumvent that heavy data engineering thing. thing. When you do get into more enterprise data, Mike, and I will not disagree with

57:26 data, Mike, and I will not disagree with this. this. Yeah, Databricks is going to be a better solution. When you get into a lot more heavy processing, a lot more rules you need to put in. However, majority of things that people are trying to accomplish, Fabric is an amazing tool for that business or even technical user. We may disagree there, but I think there’s something you said that I’m going to propose to you Mhm. that maybe you and I can build and then it releases into the world. Sure. You you talked about this idea of

57:56 about this idea of there’s we need this universal definition of a model. Yes. And the problem is and I first was going to disagree with this, too. This was going to be my second thing I was going to disagree with going Sure. It can’t It can be in Fabric, but I think what you’re really looking for, Mike, Mike, you’re we’re looking for an agents. md version of the semantic model. So I’ll back up here to explain this. this. Back in the day or the skill. md, really, is what we’re looking for. And so I’ll I’ll preface this for those

58:26 And so I’ll I’ll preface this for those who are unaware of what the know what the heck I’m talking about. Claude invented this idea of a skill or and also this agents. md file. This was the initial thing pre predating the skill and agent skill. And the thing was it worked great in Claude because Claude made it. But what realizing that every AI platform was doing some version of the same thing. So if you had a GitHub repo, you’d have agents. md, claude. md, you

58:56 you’d have agents. md, claude. md, you you’d have agents. md, claude. md,, there’s like,, know, there’s like,, arrow. md. And they’re all having their own version of this. Claude actually open-sourced the agents in a sense instructions to say, hey, what we’d like to propose to all the AI platforms is a single rule set, guidelines that any AI platform could follow. And it’s called and you’ll see this if you download or clone a Git repo called agents. md, an agents markdown file. Well, they since open-sourced skills. So you you’re

59:27 open-sourced skills. So you you’re seeing now Copilot adopt skills. You’re seeing this available in cursor. You’re I don’t know if chat GPT and Codex has has adopted it yet, but this idea of I can reuse this. I don’t have to now rewrite it for Claude and then rewrite something different for chat GPT. To your point, the we talked about this that awesome co-pilot or awesome agents. Where that can work in any platform. I think Mike, we need to have the same thing for semantic modeling.

59:57 thing for semantic modeling. Where regardless of the platform or where it sits to have that standard definition. Obviously, there are things can be custom, but to me, that’s what you’re looking for when it comes to the definitions around a company. I think you’re right, Tommy. And And I think this is where ontology is trying to support some of this., and and Donald, you’re you’re right on point here. So, Donald’s in in the chat here talking about things as well. Donald, I’m going to call on a couple of your comments here. One of them you just kind comments here. One of them you just mentioned was DAX is more powerful of mentioned was DAX is more powerful than the SQL in the analytics space

60:27 than the SQL in the analytics space right now. And I I would agree with that. And to your point, Tommy, right? I think ontology is trying to fill some of this gap around discrete So, this is why I go to the ontology item and say, make ontology from some like go into the semantic model. Let me Let me rephrase this. You go to the semantic model and you create click one item Hey, I’m going to click on this semantic model and add that into Right. my ontology. ontology. When that came out, I was like, great, interesting, but I need an ontology

60:58 interesting, but I need an ontology across all my domain models, right? So, Mark on LinkedIn is talking about, hey, you should consider developing semantic models for domains or or each vertical in a company. And now I would argue that that’s domain-based stuff, right? So, when we pulled the picture way back out, right? When we zoom out on like the big picture of what’s going on in your company going back to Donald’s comment Donald says, we just make it a data warehouse. Yeah, we are. It’s a bunch of tables. We are finding relationships between them. We need to describe what those things

61:28 We need to describe what those things are, right? And to your point, Tommy an agent needs to be aware of those things. things. And this is where my my thinking goes to your point here, Tommy. I want a generalized solution, right? If I if I look at the two different systems when I’m building tables in Fabric, if I’m building tables in Databricks I can get to the same resultant dim customers. I can get to the same resultant dim product tables in both solutions. Those tables can be described in both solutions with relationships. Right. The

62:00 solutions with relationships. Right. The same same same thing. It’s literally the same stuff. Like I’m getting to the same answer in both solutions. Now, in Databricks, I have like the control plane of like what Databricks says is control, and I go to Fabric and have its own control plane around how it thinks. So, when I look at this at a holistic

62:16 So, when I look at this at a holistic view view I have things I want to calculate in the measures world but the calculation should be the same. If if I have, lead time calculation or product lead time calculation, that’s something that’s in in that logic is built into a measure in semantic models, the logic exists regardless whether it’s a measure or SQL. Honestly, you can write it in both ways. So, what I want is this general way. Now

62:48 So, what I want is this general way. Now when you do this, this is where I think things get really challenging here, and this is where I think the friction for me right now lives. Let’s just say all things being equal. We make tables, we make relationships, and we define this generic measure calculation that can be written both as SQL and as DAX. The trick of this would be is what tool can I produce or what thing can I put in front of me that verifies that that calculation that I wrote is the same calculation in both platforms?

63:18 calculation in both platforms? And what by this is I have the same dimensions. Like I need to be able to If you’re going to do this, right? You need the ability to test this. You need the ability to say, I’m going to run this dimension and I’m going to have a column for the SQL calculation and I’m going to column for the DAX calculation. And you need to be able to see that they both match and it verifies that the the answer is correct. Now it does not change the fact that your measure, the calculation you’re trying to define, has rules around it. It’s generic.

63:49 generic. How you implement those rules, whether it be SQL through Databricks or it be DAX through Power BI and Fabric it does not matter. And so, that’s I think to me when I look at this, that’s the friction point that I’m facing right now across these two solutions. And so, Databricks is saying, come over here, we’re building all the things that Power BI has, but I’m like, that’s not how I serve my models. I go through the Microsoft stack ecosystem. Like I want to stay over here on this other side. So, it’s very difficult on what that looks like. I know I tell my kids it’s impossible to

64:20 I know I tell my kids it’s impossible to do anything you want, but this is [sighs and gasps] you can’t with DAX. And this is the problem I have and as much as I love the DAX language and I would love for everything to be universal to be adopted, it’s the running Power BI joke. Hey, what’s the answer What number is that measure? It’s a trick question because under what context, right? DAX survives and exists because like there is no number of a measure. That’s why metric sets didn’t work. And I’ll admit that, too. Where metric sets failed or

64:50 that, too. Where metric sets failed or just wasn’t the right solution was because you I think Microsoft and the team realized very early on that we can’t show a final number in a metric because it’s always based on its context. And yeah, you could maybe translate the SQL, but the DAX language survives and is built because of a valid filter context, evaluation context. And I know what you’re saying but there we still have some barriers to go there. I would like the same thing.

65:22 go there. I would like the same thing. But part of Power BI’s fluidity and part of Power BI’s flexibility is because it’s designed for evaluation context, right? It’s not just designed to have a definition. It’s designed so it can show under any of these dynamic contexts that I need it to be. Which SQL is not meant for that at all. So, there’s other rules that you would need to have under that. what? And that’s and that’s where I think to me my mental model starts shifting. And I think this is what Donald was maybe pointing out as well, right?, the measures defined in like

65:53 , the measures defined in like so, when I look at a SQL statement, select these columns with this sum of whatever the thing is, the the mathematical piece, the the measurement piece, right? That you’re adding to these dimension elements is is is is written out. And when you write the SQL statement, you have to update the columns in the SQL statement to get a new value. The metric sets in Databricks is a way of you being able to say, look, I can define a common metric. And I’m going to be very simple here, right? There’s a a column called sales and I want it to always aggregate sales

66:24 and I want it to always aggregate sales up to total sales, right? That’s just a sum of the column. By doing that, you can dynamically build the SQL using, hey, this is metric is going to be used and I can select these columns. And then you can pick any other new columns you want to select. The advantage of this is DAX is doing that, but DAX is leaning on this MDX language. MDX doesn’t have to define it didn’t have to define the measures, the dimension side with the with the measures all in the same thing. You could define the measure, the calculation for the

66:54 measure, the calculation for the measure, and then arbitrarily grab whatever dimensions you wanted and would just put it there. You could still do that with SQL. It can be dynamically generated, but then you need tooling around that and it just it just operates differently. I know. I know. So know. So this is why, this is why DAX was created. The Someone looked at this and said, SQL’s good. good. We need something else. MDX is good, but no one knows how to write it. Excel functions are good, but that’s business-user-centric, right? So, there’s a lot of these tradeoffs that

67:24 there’s a lot of these tradeoffs that are all happening at the same time, and I think that’s where the DAX language evolved into. This is a this is really flexible filter-context-driven language. That’s why it exists. We don’t have the equivalent DAX thing inside Databricks. And so, this is where again, this is where the struggle becomes. Like when you’re trying to blend the two platforms, we get friction. what we need, Mike? We need a model. md or model. md type thing. And I think I’m going to I may have to start dabbling in this where

67:54 this where a model. md would have the instructions, but also have because if you look at a DAX query, even in a simple bar chart, it’s not simple. Like if you look at the back end, the actual query that evaluated that statement, it’s long, right? And it’s not again human readable. I think it really to your point, you’re trying to accomplish two things. I know we’re getting way past time, but love this. We need something that’s going to be human readable, but computer readable as well. And it’s very hard right now with DAX and SQL to accomplish both those things. So, I think we need to create a

68:25 things. So, I think we need to create a agent that can help us create model. md. Yes. Yes. I’m saying here? like where it’s going to have the business definition and rules, but also help the model translate DAX to SQL and back and forth. I don’t think an MD file is going to be able to do everything And I’m not saying a markdown, but you see where I’m Well, and I think I think this is where So, if I’m reading between the lines on some of these things Yeah. this is where , the ontology stuff is trying

68:57 , the ontology stuff is trying to fit. If if I had to read between the lines, right? So hoping. Right? I need something that is agent-centric. I need something that is very comprehensive of all the things I want to build. And I need an agent to be able to go look at that and say, here’s things in this common ecosystem, what can I pull out and what can I use? So, again, my my big dream here is I’m I’m I’m we’re going to get Someone’s going to build it. It’s going to happen eventually, right? It’s going to it this is going to appear. Someone’s going to be able to build this common common semantic model, enterprise semantic

69:29 semantic model, enterprise semantic model, whatever you want to call it. that already exists. Yeah, no, not that. We’re There’s so many names for things at this point. Like, let’s use Let’s use Donald’s language here, right? There’s going to be a data warehouse built, right? The data warehouse will have all the things that we need to define, the tables, the relationships, and the measures that we care about. What I think needs to be developed here is an agent that can come in and look at that and help users I really want this This is my idea, Tommy. I want to bring my shopping cart to the data warehouse, just like I would

70:00 to the data warehouse, just like I would go to like a Costco. I want to look at all the measures and the KPIs on the on the shelf and just pull into my bin or my basket, “Hey, look, I need these tables, I need these measures, I need this thing.” And boom, here’s Here’s the key elements I need. Agent, go figure out how to build a semantic model that makes all this stuff work. Like, that’s That’s where it should come from and it should come from a centralized model. You’re going to love this. I’m going to tie it all together here and I’m going to blow your mind, okay? You and I are going to create our own version of Delta, Lettuce, whatever it’s called,

70:31 Delta, Lettuce, whatever it’s called, right? But we have to combine the two. So, just hear me out. You’re going to I think this is amazing. Okay, so you have Delta and you have Lettuce, right? That’s the one for Snowflake. Oh, no, no, not Lettuce. It’s Iceberg. But it’s it’s a form of Lettuce. Let’s say Iceberg. Okay, fine. So, we need something that’s a food, right? But also what is What’s a Delta? What shape is a Delta symbol? Upper case. It’s like a triangle. Triangle. Okay, well, we need a food. What food is a triangle form? Triangle form. Food comes in triangle form. Watermelon

71:02 Food comes in triangle form. Watermelon when you cut it. Pizza. Oh. We’re calling our model pizza. The pizza where the What is it? So, you have the The pizza warehouse tables, you have Snowflake, and you have pizza. You have pizza and it’s going to be our combination of this model definition. Pizza. Golden. It’s gold, Jerry. It’s gold. [laughter] Well, now that we’re using our pizza models and it’ll all work.

71:34 That’s amazing. Amazing. Awesome. Well, this was a really good engaging discussion. There’s a lot to unpack between Databricks, Power BI, Fabric, all the things that are happening here. This is a very dynamic space. It’s changing all the time and I really see a healthy At the end of the day, it’s a healthy competition between Microsoft and Databricks. I think this is a really good thing for Microsoft Fabric to be seen so when as large as Databricks coming in and competing and pushing and helping us build new features, I think regardless at the end of the day, competition is always a good thing. It’s definitely different how

72:05 thing. It’s definitely different how both teams are approaching the world. It’s There’s a lot of,, friction as you try and jump from one platform to the other one at this point. Maybe that’ll get better in the future. But for now, it’s a good good thing to unpack and I still I still think both tools serve really good purposes. purposes. It really It really I’m not going to be able to recommend a single solution for any organization, but I will say there’s a lot of things to consider here as you think about Databricks, DBT, Power BI, and Fabric. We actually didn’t talk too much about

72:35 We actually didn’t talk too much about DBT at all, Tommy. That’s another kind of That’s another day. another another solution here. Okay, that being said, thank you all very much for hanging out here at this a little bit later today on this episode. I hope you found this to be informative. I hope the discussion here was helping you unpack a little bit of what’s going on in these models in this space. We try to be thought leaders in this area. We’re just sharing what we have experienced and we also love the chat here. So, thank you, chat, for being very engaged and sharing what you’re thinking about here as well and and the areas and and parts of friction that you’re seeing as well

73:05 friction that you’re seeing as well between these two platforms. Yeah, thought leaders. There’s a lot of Well, maybe maybe just thoughts. Maybe not We made up our own pizza model. But here But hear me out here, dude. This is another I thought of another thing. You have what? Bronze, silver, gold, crust, cheese. It’s slices. Which slice are you talking about? got to know. Crust, sauce, and cheese, those are your layers. Oh, boy. We can Gold, Jerry. everything And then each of the each of the domains could be a slice, right? And then you you put them all together to become It becomes the pie. Yeah. Oh my

73:37 It becomes the pie. Yeah. Oh my goodness, Tommy. Oh my gosh, this is incredible. I’m going to going to go go the data world. [laughter] Everything’s pizza. Everything’s pizza. There’s Put it on a T-shirt. go. Everything’s pizza. I love it. You can find us on Apple, Spotify, wherever you get your podcast. Make sure you subscribe and leave a rating. It really does help us out a ton. And you want to help us make pizza? Go to powerbi. tips/podcast. Leave a name and a great question to submit to mailbag. And finally, join us live every Tuesday and Thursday

74:08 live every Tuesday and Thursday a. m. Central and join the conversation on all Power BI Tips social media channels. Awesome. Thank you all so much and we hope you have fun with your Lettuce standards now, aka Iceberg. That one’s funny. That was good. Awesome. [laughter] Thank you all so much. We’ll see you next time. Flame. Flame. Explicit Measures, pump [music] it up be a hype. Tommy and Mike lighting up the sky. Dance to the data laughs in the mix. Fabric and data, get your fix. Explicit Measures, drop the beat now.

74:38 drop the beat now. Pop your screens, [music] feel the crowd. crowd. Explicit Measures

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.