PADU and Fabric - Ep.570

Fabric gives a team several ways to store data, ship a change, and let an agent touch production. Tommy Puglia and Mike Carlo sort those choices with PADU: preferred, accepted, discouraged, and unacceptable.

News & Announcements

• All About the Power BI / Fabric MCP. The next Chicago Fabric and Power BI user group is October 22. Tommy’s session installs the Fabric remote MCP and the local Power BI semantic-modeling MCP, then walks through what each one can do. RSVP with a first and last name. A handle is not enough for the ID check at the door. The same morning they also have a 10 a.m. Central live stream on Skill Vault, Tommy’s app for moving agent skills between tools.

• Trusted AI starts with Microsoft Fabric, Real-Time Intelligence, and IQ. Tommy reads the post as Microsoft placing Fabric inside a wider IQ story: Web IQ, Work IQ, Foundry IQ, and Fabric IQ. The line he pulls is Fabric as the operational nervous system: observe signals through OneLake and Real-Time Intelligence, analyze them with Fabric IQ’s business context, decide with AI-assisted reasoning, and act through people and agents. Mike hears event-driven work, not a truck streaming GPS all day, and he does not want an IQ brand on every surface. He wants an agent that can reach the data. Tommy’s counter is enterprise permission: a governed bot is not the same as a consumer connector. Tommy also counts the word ontology 14 times and finds an ontology MCP mentioned without a clear install path.

• Understanding DAX contracts when using AI. Marco Russo and Alberto Ferrari argue that the important artifact is not the DAX. It is the contract: a structured description of the business rules the code must follow, sitting between a vague model sketch and the measure itself. Their customer-ranking example starts by asking what the rank means (who competes, which filters count, how ties break) and only then asks the agent to write the measure. Mike takes that as confirmation that business logic buried in DAX, SQL, and Power Query has to live one layer higher, in a file both people and agents can read. He suspects semantic views are what pushed the article into the open. Tommy calls the skill a conductor’s skill: move context to the right tool, check the output, and keep the token bill honest.

Main Discussion

Topic: A PADU scale for Fabric, three years in

Tommy found the scale in Fatal Risk by Roddy Boyd, in the section on AIG. A preferred practice might be quarterly statements online. An accepted one might add a printed annual report. Defaulting to paper was discouraged. Paper only was unacceptable, and it was removed. Unacceptable, as he reads the book, was close to a firing offense. Preferred is not a mandate. It is the pattern that gets people hired and promoted.

• The first weight is cost, and the cloud flipped it. Mike’s high-level cut is that storage is cheap and compute is expensive. On premises, the machine was the expensive purchase and idle CPU was waste. Drives were the hard part to add. In Fabric, a gigabyte for a year is close to nothing, so the design computes once, stores the result, and reads it back. Bronze, silver, and gold, notebooks that land a table, and in-memory semantic models all follow from that. Tommy agrees on capacity, then refuses to let CU be the only weight. Customer experience and trust in the data sit beside it. A cheap pattern that nobody trusts still fails.

• AI is an organizational problem, not a COE sidebar. Mike’s third weight is that employees will use agents whether or not the company buys the seats. The old shadow-IT complaint about Excel is the same pattern: people use the tool they have. He is wary of backends that have not been through Fabric’s compliance story, because the security surface just got larger. Tommy’s list for the rest of the hour is customer experience, capacity, and AI integration.

• Storage does not get one grade. Both put the lakehouse at preferred for batch analytics: SQL analytics endpoint, notebooks, and agents that can build on the pattern. SharePoint lands at accepted, especially with OneLake mirroring. A personal gateway pulling files off a laptop is on the low end, discouraged and close to unacceptable. A Fabric SQL database is preferred when the job is an app or a form that writes in the moment, and only accepted when the job is analytics. Mike notes a floor: even an F2 feels the database. He has heard of on-demand billing and has not seen the numbers or a blog post.

• Git is preferred. Publishing around the pipeline is not. Mike lives in GitHub now because an agent can write the README that tells the next person, or the next agent, why the project exists. Deployment pipelines and the deployment planner are accepted, with the open question of how data moves from dev to test to prod. Publishing a report straight into production, outside the pipeline, is discouraged. It can break the link between the report and the semantic model. Developers should live in dev. An agent on a live XMLA endpoint can delete a table before anyone says stop, which is why version control is no longer optional.

• The lightning round splits them in a few places. Export to Excel is discouraged for both. Analyze in Excel is accepted. Paginated reports are accepted for Mike, and only discouraged-with-exceptions for Tommy (invoices, email, automation), because the UI does not match the engine. Dataflows Gen2 are accepted for Mike, as the path most Power BI people already know, and discouraged for Tommy on cost. Spark environments are preferred. Data agents are discouraged for Mike, because he would rather have an agent call the APIs or a Fabric app, and accepted for Tommy when they are used with care. User data functions are accepted, and preferred once Fabric apps need them. Fabric apps themselves are preferred.

• Ontology is not ready to be self-service. As of October 8, Tommy puts the ontology item at unacceptable until there is a plan. He does not want marketing to create one on a whim, and he is explicit that this is not a forever ban. Mike puts it at discouraged. The message is not clear, he does not see how the agent uses it to get a better answer, and he would rather call the thing a knowledge graph: business terms bound to tables and relationships. They leave the argument for another episode.

Looking Forward

Write the four buckets down for one workflow you already run, including the use case, before you grade the artifact.

Episode Transcript

0:00 Dance to the day to laugh in the mix. Fabric and A. I get your feels. Explicit measures. Drop the beat now. Kings feel the crowd. Explicit measures. Explicit measures.

0:28 Hello, everyone, and welcome back to the explicit measures podcast with Tommy and Mike. Hello, everybody. How you

0:33 doing? Dude, I’m doing OK., this is that time of year.

0:38 Baseball playoffs. Yankees just got eliminated yesterday, Mike.

0:42 So let’s talk power behind fabric because you don’t want to talk about you don’t talk about the Brewers,

0:47 Tommy.

0:48 Not really. It’s always October when I question the morality of sports.

0:54 Like, why should I care about a uniform and someone hitting a ball with a bat? Does it affect my life?

0:59 Does it save my soul or change if I have to pay taxes? It doesn’t.

1:04 And I always question that now, especially when the Yankees get eliminated in three games. But yeah, good on the Brewers. Good on you guys.

1:10 , Tommy, I’m sorry for you, but the Brewers made it through. The Brewers made it through.

1:14 Yeah. and then maybe actually do something with your first

1:19 place record so

1:22 we should try we should come on team keep it together here we did it in four so that was usually they drag this on to like every single game in the series so i’m actually shocked they were like oh got it done in in four games so all good on you yeah I don’t get to say this very often because very rarely do I make it farther than the Yankees or the Brewers make it farther than the Yankees anyways.

1:36 So,, we’re also like the underdog. We’re a very small team compared. You guys are.

1:51 You have the best record in the major leagues.

1:54 That’s not my record. We’re talking about money per like money per franchise.

1:56 All right. Tommy knows this.

1:59 I also know what the most wins out of 30 teams means.

2:04 Again, every time, even a squirrel can find a nut. Even a blind squirrel can find a nut, right?

2:13 We’ve done well this year. So anyways, we’ll take the wins when we get them because most other years we’re losing., that being said, let’s move on to our main topic today.

2:22 Our main topic looks like it has an acronym in it. It is.

2:31 So let’s explain that. Cause I had to look it up myself before we get going here, Purdue and fabric., what does that even mean?

2:40 So,, P stands for preferred a is accepted. D is discouraged and U is unacceptable.

2:48 So defining what we want to be able to do or not do in our organization,, how does a center of excellence define these standards, patterns on how you want to work with Fabric and or Power BI? What does that look like for your organization?

2:57 So I think these buckets will help us to categorize patterns or behaviors that we’re doing now. And what does that look like inside?

3:06 Potentially, it might even change a little bit between a small and large organization. So we’ll try to unpack some of these concepts around Power BI and Fabric and unpacking these acceptable or not acceptable patterns.

3:15 Cool. Tommy, you want to kick us off with your first announcement or news item?

3:25 Yes. So we actually have October 22nd is going to be the next Chicago Power BI and Fabric meetup.

3:38 It’s all MCPs all the time. We’re going to help users install it.

3:51 If you can join, we’re going to actually work through what it looks like to set up both the Fabric remote server and the Power BI local semantic modeling MCP server. and then actually how to use it the best ways to apply it what capabilities it has and that’s going to be our session so make sure the meetup link is on youtube in the description of the podcast make sure to rsvp before you got rsvp or you cannot come in so it’s your full name yeah

4:05 it’s not your name that’s you’ve got to be your full name on the rsvp because you’ve got to be let in the building so just fyi about those things

4:10 yeah So we’re really excited about that. So again, make sure to RSVP coming up in a few weeks, two weeks.

4:16 And you and I are talking today, aren’t we? On the podcast, we are.

4:22 No, yes, after the podcast. The live event.

4:28 So we’re going to showcase Skill Vault.

4:35 Yes, correct. Sorry.

4:37 There’s another live stream coming. There’s another

4:38 live stream. Yeah, we’ve

4:39 got it scheduled. What time are we talking about that one?

4:43 10 a. m.

4:47 Central. 10 a.

4:51 m. Central.

4:55 So we’re doing an early podcast now, but Tommy’s got a skill vault that he’s been working on with some AI solutions, pieces that you can go use to help move your skills between different environments, between different agents. We’re going to live demo that as well. Awesome.

5:00 All right, so let’s get to some news news. And what we have here, the first one is from the Microsoft blog.

5:18 This is one of those conceptual ones. And it is about trusted AI starts with Microsoft Fabric.

5:36 real-time intelligence and IQ. And what the main article is really saying, Mike, is really positioning where Microsoft’s overall plan, not just Fabric, play in their idea for really intelligence for organizations.

5:53 They’re really putting this together in the whole context of the umbrella of Microsoft IQ, which has four main components in it. Web IQ, Work IQ, Foundry IQ, and and Fabric IAQ.

6:11 And where they’re positioning Fabric, and I want to get your take on this, is Fabric is positioned as the operational nervous system to observe business signals through OneLake and real-time intelligence, to analyze them using Fabric’s IQ business context, decide with ai assisted reasoning and govern policies and act through people work through an agents and a big focus that i’m seeing here is about the real-time intelligence and that’s where they see fabric iq playing a role some of the so let me stop there so because again a lot of this is about the streaming connectors schema where event streams it’s really putting the money on hey fabric is for real time for your overall iq scheme What’s your take on that?

6:32 , I think there’s a lot of data that comes into the system here. I think a lot of this is very relevant.

6:39 There are organizations that are finding value in real-time analytics, but less than, I think maybe there’s a misnomer around what real-time analytics means, right? I think real-time analytics means doing things in real time, like meaning a file shows up and then you process the file as it occurs, right?

6:46 You may not run that process all day long and it may not even run until tomorrow’s file show up, right? Something like that, right?

6:53 So I’m looking at this going, I think they’re trying to move a bit more of the ecosystem more towards an event driven action based system, right? That makes more sense to me.

7:00 When I think people think about real time, they’re thinking like, oh, I’m going to have a stream of data coming from a truck that’s delivering packages, and I’m going to get the GPS location, and it’s going to constantly send data. I know when you process that data in real time, it’s expensive.

7:07 The more you process data, the more compute you’re going to incur. So one, I just want to preface this with, I think they’re trying to talk about here event-driven data.

7:14 I think that’s really the point. They’re trying to make the system be better for that.

7:21 And then when something shows up, you’re able to evaluate and pull in agents when needed. So I’m going to choose to interpret it that way.

7:28 There’s so many IQ things that they’re throwing at us now. I feel like when I buy a Claude or I feel like when I buy a Grokbot or…, an open AI dot, right?

7:35 If I’m buying a system, I don’t care about work IQ and web IQ and foundry IQ and fabric. I don’t care about any of that.

7:42 I care that my agent. whatever that may be, a bot or a dot, can just access the information.

7:49 So to me, this is a lot of fanfare around these IQ systems that are like, we’re going to put IQ on everything. I don’t really care about any of that.

7:56 I care that I can get some agent to talk to the data in the easiest way possible. And I feel like there’s just so much friction right now inside the Microsoft ecosystem to get anything to talk to each other.

8:03 I can’t work with this. It’s just hard.

8:10 I look at comparatively… I look at, I’m all in on Grokbot.

8:16 It’s so cool, Tommy. I’m automating a lot of my workflow with this thing.

8:23 It handles things for me. It’s doing reasonable stuff.

8:30 It’s at a good price point. It’s getting things accomplished for me.

8:37 So when I look at that Grokbot system, I’m able to give it. And when I looked at it initially, it was, hey, I want to have a calendar.

8:44 Great. Sign into Google Calendar.

8:51 It just works. It can read my calendar.

8:58 Hey, I wanted to go handle emails in my Gmail. Great.

9:05 Sign into Gmail. It works.

9:12 There’s all these MCP hooks that make these agents work on top of the actual data. None of that requires me to have Google IQ or Google email IQ or search the web IQ from Google.

9:19 Like the bot just figures it out, knows how to do it. So I think this is a lot of fanfare and marketing material on top of this.

9:26 That’s just not needed for me. I literally, all I care about is M365 agents.

9:33 Do I have an agent? Can I know what they’re doing?

9:40 Can I treat them like an employee? Period.

9:47 All this IQ stuff is not helpful to me.

9:54 Well, I’m curious if the devil’s advocate or the other side of the argument here to me, Mike, is with the Googles or the other connectors, right? They’re not, and I know you can say this about Google, but they’re not necessarily enterprise tools.

10:04 And when you think about the Microsoft ecosystem or the Office 365 ecosystem, right?

10:15 Yes.

10:16 You just don’t also want an MCP to be able to do whatever it can if you’re not an admin, right? different roles and departments are going to have different permissions or capabilities.

10:26 So that to me is where the IQ makes sense. Now, why there has to be a web IQ, that makes sense.

10:37 Like you don’t want your, your bot just running the web and an organization pulling data in that could be malicious or wrong. So having that governed by, but again, but it isn’t, that would be the argument. However, like I understand,

10:47 I understand your point. But I’m also at a miss at for, why do I need a thing called WebIQ to do it?

10:56 Aren’t I just governing the bot that does the thing? Or I’m setting a policy around, hey, here’s a policy.

11:05 Everyone can use these agents. And here’s places where we let agents search the web on or not search the web on or use it like…

11:15 I’m thinking about it less about the IQ surface area and more about the user and the user groups and the interest surface area. To me, that’s where it makes the most sense here because the story here is still, to me, it still seems disjointed at this point.

11:24 A hundred percent. Speaking of MCPs, Mike, I was going through the article and like, they got to talk about ontology somewhere in this article. How many times do they

11:33 refer to that?

11:35 Oh, more than once. That’s for sure.

11:38 And 14 times I searched for the word. It’s 14 times in the article.

11:42 Yeah. And talking about disjointed, if you look, you can also search ontology MCP.

11:53 There’s now apparently an ontology MCP, which has a direct link to defining business rules, but it doesn’t talk about at all a link to installing the ontology MCP. which I find this very strange.

12:03 So below in the trusted or shared context for operations and AI underneath one of the graphics, it says create ontology MCP tools. When I click on that.

12:15 It just takes you to the ontology agent.

12:17 Yeah. So, okay. There is another

12:19 article. They actually missed the article because right beneath that, there is another one that says consume the ontology preview as an MPC server. So

12:26 yeah,

12:27 there is an MPC something coming out. Now, one of these questions I’m worried about, like concerned about here, Tommy, is when I think when I see this thing, MCP server things, I’m hoping that that is using an API call to do things, to do actions on the data inside Fabric or getting things in and out of the ontology.

12:40 I’m hoping this is not like call the MCP server and then I’m calling more CUs because I’m now using an agent to do something My impression of the MCP servers are, yes, it’s a library of tools. It has a handful of API calls.

12:53 And I’m not spending money on using an MCP thing to go spin up another agent to go talk to the ontology. That doesn’t seem right to me.

13:06 That’s not my understanding of how other systems work with MCPs.

13:20 A hundred percent. And here’s the crazy thing.

13:24 You’re not creating MCP for your ontology items. You would have to create an MCP per ontology.

13:28 Yeah.

13:29 So how you actually install, which is insane to me to think about what, I don’t know. You can’t use a service principle to access it.

13:45 I don’t like that either. So yeah.

14:01 I’m sure there’s some restrictions on here. I think the overall picture, Mike, or the bigger picture here is they’re trying to fit where Fabric and the Fabric tools fit in Microsoft’s IQ and Microsoft’s overall vision for artificial intelligence.

14:16 That’s what we’re looking at here. And I don’t think that’s really…

14:32 It’s getting more defined, but I still think there’s still very… we’re in pangea now of you can tell i’ve been watching dinosaur movies with the kids but we’re i think still the tectonic plates are still moving to me so it’s not super clear so that is that article and mike i got one from an old friend someone we have actually haven’t done an article on in a while and this is mr marco russo in alberta ferrari okay And this is a great article that I think is going to be upgraded to an entire episode.

14:48 But their article here is understanding DAX contracts when using AI. And the idea here, the central argument is that when we use AI to build DAX measures, The most important artifact is not the DAX code or the function itself, but is what they call the contract that defines the business rules and the code must follow.

15:04 So the contract is a structured description of how the calculation would be within the semantic model. It sits between technical code and high-level business documentations.

15:20 And Marco and Alberto are arguing that historically business logic was buried in DAX, SQL and Power Query and Excel formulas. But AI can change that workflow.

15:36 So rather than writing,, create a customer ranking measure, basically,, start by asking what questions need to be answered to define customer rating. so i love this idea i love this idea there’s a few thoughts i had on top of it but i love this idea this is things we’ve talked about too with doing recons first but i think especially with dax i was mike i was literally talking to a team team yesterday that are you going to be using ai i said yeah if that’s okay i said and then we went on a conversation about The fact that anyone can use AI and get somewhat good results with Fabric and semantic modeling.

15:51 But there is, to me, a experience, a skill on providing the right prompts and context using the MCP servers. to make it work, and I think this is showcasing it.

16:07 So, Mike, looking at this article, talking, and also talking more about your experience prompting and context engineering, does this just show you more and more how important that element is to really just using AI?

16:26 There’s two thoughts that come to mind. My first thought really stems around, wow, have the tides have changed.

16:35 I really distinctly remember a very pessimistic view of AI earlier in the year from both tabular editor and SQL BI. Don’t use it.

16:45 That makes a bunch of mistakes and they weren’t wrong. They weren’t wrong about it.

16:54 However, I do believe if you were a good prompter of things, if you had a good knowledge about how to use AI systems, you got decent results out of the AI with DAX. And I was disagreeing with these opinions of don’t use AI.

17:03 You still need to have your hands on the code. You still need to write code on everything.

17:13 nope, you guys were wrong. And, and you guys, I think that the message really wasn’t like, don’t use AI.

17:22 It was use it very cautiously. I think that was the message there.

17:31 So to be very clear, SQL BI and tablet editor, we’re not like, don’t use AI. It’s awful.

17:41 But actually when I look at what they were doing, these were two companies that were talking technically about the language itself. They’re there to deal with the Timdal.

17:50 They’re there to deal with semantic models. They’re there to deal with DAX specifically.

17:59 And so their businesses have been built around writing code, basically. And so when you have this new AI thing comes in that’s basically attacking the core of their business, why would I need a book about DAX anymore when I could just go ask an agent about it, right?

18:08 So… while I do think they have pivoted very well, this to me, the messaging they’re giving out now feels much more in alignment with what’s going on, right?

18:18 We’re seeing Tabular Editor build CLI interactions to an agent that can then use AI to help them build, right?, I feel like I’m actually physically seeing both of these companies embracing the creator agent. We’ve been saying this for months now, Tommy, creator agent or the agent dev.

18:27 I don’t know how you want to spin the name here. I don’t know what name was going to stick.

18:36 But the creator agent is the thing that helps you build a repeatable system that is deterministic. It is written in a report, it’s written in DAX, it’s written in something else.

18:46 You use the agent to help you build the system initially, but once the system is built, you don’t use the agent anymore. The agent goes away.

18:55 And we’re starting to really see, even in Microsoft’s development cycles, you’re getting Raytheon, you’re getting all these other tools that are using AI systems right alongside building real solutions. And so I think what this is doing here is this is just one more step of, well, there’s a lot of business knowledge and context in your head.

19:04 And Tommy, I’ll give you another example here. I’m actively working on this.

19:14 My second thought is all the code that we’ve written already to date for DAX for data engineering, for M code. That is the business logic and rules that we use to get to where we are right now.

19:23 Right. The business logic is already buried inside the code.

19:32 It’s it’s the code is a realization of conversations with team leaders. how the data fits together, what relationships we have.

19:42 And I’m more keenly aware to this than ever before, Tommy, because I’m actively really pushing my mind around, there’s this concept of cataloging items.

19:52 Mm-hmm.

19:53 And then there’s this ontology thing. And then there’s this thing, what I think we actually really want is a knowledge graph.

20:03 I think the knowledge graph is what people are actually striving for, where you can link tables together and you can have descriptions of relationships. And we need all of these systems to work. Why do I

20:12 sound like another work stream’s coming? Workload.

20:16 Listen, I can’t, I can’t catch you now. It’s been 600 episodes.

20:20 I know the way you pitch things down. Like that’s not just an idea.

20:24 It is. It

20:28 is just an idea, but it’s turning into software because it, it makes sense to like, again, I don’t want

20:35 to,

20:36 no one wants to manually build all this stuff. I need to be able to, I need to be able to go into an agent and then just go play with it.

20:47 I need to tell the agent, okay, go visualize this stuff, go make these things, go create the thing that I need. Right.

20:58 So there is a whole realm of other information here. That’s that we have to get our hand around.

21:08 And what I think is what’s occurring here is. There is a new world, and we talked about this before in the podcast previously, Tommy.

21:19 Monthly active agents needs to be a metric. If you are building software, that needs to be a metric that has to be part of your design plan.

21:30 What does it look like for your software to be accessed by agents? not just an MCP server where you’re wrapping up some other large language model behind it, but like really thinking through, okay, I now have two types of users that need to access my information, a person on a screen that needs to be able to click and add and edit things, but then also I need some agentic system behind it that can then read the same information, understand the same data structures, get the information out.

21:41 And while I disagree with how Microsoft has implemented ontology, because that prior article you were talking about, Tommy, which is Trust in the Fabric IQ, the article was written by, I believe, the gentleman who’s actually in charge of all of the ontology pieces, They’re missing the mark.

21:52 Microsoft’s missing the mark on ontology. You don’t know what we need as business users, I don’t think.

22:03 You have something, it’s out there, but no one understands how to use it. It’s difficult to work with.

22:13 And it’s not meeting the needs, I think, of what people are actually trying to accomplish with what an ontology, or actually, I would actually argue, ontology is not what we want. actually is we want a knowledge graph.

22:24 We want to be able to mix terms of the business, how they work together. And this goes right back to Marco Russo’s comment in his article, which is your team understands your business. where does that code go?

22:36 So I don’t know what the name is officially going to be, but to me, this just confirms more and more and everything you’re talking about too, along with how Margaret Russo’s really framed. This is, we’ve talked right about the conductors era, right?

22:51 Like that’s the world we live where we’re conducting the orchestra to me, like to be very honest, to be very frank, Mike, I think my greatest skill right now that I can provide is the conductor skill. How do you orchestrate all these different systems flow context to an execution to actually apply, generate,, understand, and then get that context back in the right place with prop from a token optimization to making sure there’s no hallucination.

23:06 And so you also can understand that flow as well. Not just the agent, but, what tools and system to use creating skills and creating agents, your own custom agents, be it Grokbot or in Cloud, and not just a session.

23:22 To me, this is where I think you’re looking at Power BI and Fabric that there’s this layer above on the skill. Because to your point, I’m not really writing a lot of DAX, but I guarantee you, you could take- I’m reviewing it.

23:37 I’m looking at it. I’m looking at the output of it. Some of the- I’m

23:53 telling my agents, build tests around this. Hey, I want this dimension.

23:57 I want that dimension with this measure. Go write a test on it. Like, go build

24:01 it. All the business rules, right?

24:05 How do you actually provide that? Especially not just for a single measure, but across the board.

24:10 So who knows the semantic model- it knows properly. And again, Marco and Alberto speaking right to it properly.

24:14 What’s the functions meant to do to me? This is, and again, I don’t know how to measure this skill or how to define it, but this is why I think we’re a lot of people are missing the boat.

24:19 And again, commercials aren’t helping because chat GPTs, great commercials, like,, make a summary of the page and it makes this nice slides. Great.

24:23 Cool. Yeah.

24:28 Ad hoc task, right? Yeah.

24:32 It is. Yeah.

24:37 So I, I look, we may have to talk with that one more detail. That may be an episode.

24:42 There’s this world of information that’s happening, and this is where I’m going to push on. Ontology isn’t the answer.

24:51 It’s actually something more like knowledge graph. And the more I push in Tommy to this thing called semantic views that Microsoft is making, I think it makes sense to me because there’s a there’s a whole handful of things of like what tables are doing, what measures do you have, what relationships are there?

24:59 Those same tables measures and relationships, those are the same types of things I need to use in SQL. Those are the same things I need to use in KQL.

25:08 Those are the same things I need to use in a semantic model. And those are the same types of systems or those same pieces that I’m trying to use in Databricks or Snowflake or some other system, right?

25:17 They’re all there. They all exist.

25:25 It’s just we haven’t had a good way of describing them, right? And again, as we think about what agents are able to do, I think what we’re having to do is we have to rethink what we build because the speed in which we can build things is now faster than ever.

25:34 It’s so fast. We can create a lot.

25:43 And the problem I think that’s going to exist now is our creation side is going to be able to get immensely, we’re able to build a lot of stuff. The question becomes, what are you building and is it valuable?

25:51 And does it align to these,, the contracts? And again, also, Tommy, I’ll put out here, this contract concept is not new.

26:00 We’ve been talking about data contracts between departments for years. That’s been always a point of contention.

26:09 Who owns what data? How are you going to hand it off to the next team?

26:18 A data contract is the same thing. These are the columns.

26:26 We define what data is in them and we tell you when it’s going to get there, right? There’s a refresh cycle or how the data stays fresh.

26:35 Those are important topics to hand off to the next team as you build a gold layer of tables to the next team and you hand it to them. What’s their responsibility?

26:44 What are you gonna let the next team do? So that’s where things get really interesting here.

26:52 So I really liked this article, Tommy. I think this is very good.

27:01 I believe this is articulating the concepts really, really well. And all of this information that Marco is describing here, I believe needs to land inside a Markdown file or information directly inside the specification, something that both the people and the agents can go look at it.

27:10 And I don’t think, Tommy, all this agent, all this information here needs to live solely in a semantic model. I think this information that Marco is building here should live one layer up higher than just the semantic model.

27:18 Yeah. I also suspect that the reason this article came out from Marco is because of semantic views.

27:27 I think semantic views came out and this is shifting his mindset around what this means. He’s already preparing for what a semantic view will be doing and how it will impact the business because a semantic view is this information.

27:36 It’s a combination of files. It’s a combination of other information.

27:44 It’s that’s, that’s the, I think the missing link, a lot of this is the missing link to give proper agent access to like what’s going on inside your model. A

27:54 hundred percent, a hundred percent

27:55 or inside your business, not even the

27:56 model,

27:57 right? It’s bigger than that.

27:59 Dude, I love it. All right, dude.

28:04 Well, hey, I think that’s going to be… I cannot wait just to talk about Semantic Views.

28:09 I think that’s going to be a series at some point that we’re probably going to do. We haven’t done a series in a long time on the podcast.

28:15 We have not. So that may be one.

28:20 I think especially once those things start getting released. But let’s jump into the main topic if you feel so ready to.

28:25 I’m done beating up these news items. We’d spend 20 minutes here.

28:27 Let’s go. Let’s go right into the main topic now, I guess.

28:30 So, Mike, what I want to do today is and what we want to do is there’s this concept and I was reading this in a book and it was actually about the rise and fall of AIG. back when they were really good at what they did.

28:43 And then obviously when they really were not as great as what they did and failed massively. But the idea here really struck me.

28:57 And I thought we can really connect this to technology governance and Microsoft Fabric. To me, this was a perfect layaway that we are now at the point how three years into Microsoft Fabric officially, right?

29:10 I think it actually was officially announced in 2023. I think we are now at the point where we can actually look at what’s going on in Microsoft Fabric and actually really understand what is best to use, what are things from a governance point of view, what are things that we want people to do, what are things that are unacceptable.

29:23 from a best practices point of view, the lay of the land, she is clear now or clear. And what I want to do, I want to read to you from the art, from the book.

29:37 And then again, how we can tie this to Microsoft fabric and power BI from practices process and artifacts and technology. Before you do that,

29:50 do you have the name of the book? Yes, it is.

29:56 I

29:56 believe I’m looking at the topic and the context here, but anywhere in your description here, I do not see the name of the book that you’re referring to.

30:02 Yep. It is called,

30:06 I should know this, but Fatal Risk. Okay.

30:09 Gotcha. I’m going to go find that on Google and put that out here for people.

30:12 Do who it’s by? Yes.

30:16 The book is by… Roddy Boyd.

30:22 Okay. Boyd.

30:27 Got it. Yep.

30:31 Got it here. I will pass a link to people if you want.

30:35 So the book that Tommy’s reading here is Fatal Risk. There’s an excerpt from it that’s resonating with you.

30:39 And then I’ll put that link here in the chat window just in case someone wants to see it. Yes, from

30:43 the article. So AIG introduced a scale used by other companies that placed every service into one of four buckets, preferred, accepted, discouraged, and unacceptable, or PADU.

30:56 A preferred practice might be delivering customers quarterly statements online, eliminating paper mailings entirely, while online reports accompanied by a printed annual report was deemed acceptable. Defaulting customer statements to paper copies were discouraged.

31:09 Sending only print statements were unacceptable and thus eliminated. So, Mike, we have this element here.

31:23 And I really do love this framework because prefer does not mean necessarily mandatory. And while unacceptable, usually in the book, they went on to say more or less if it’s continued by a team, it was a fireable offense.

31:36 So unacceptable is really that bottom here. And I think when we look at the process that we have, the workflow that we’re using, the tools that we’re using in Microsoft Fabric, I think we can really begin to have this conversation of could we put together a padu framework, preferred acceptable, discouraged, and unacceptable, around what teams or individuals can do and what they should do in Fabric.

31:49 And I, again, I don’t think this exists today. There are, if you look at Microsoft documentation, they do have workflows, but there’s ones that are not necessarily highly discouraged.

32:02 And I think now we’ve been in this long enough to actually see what works, what are the things that are okay? We’d rather you not do it.

32:16 And what are the things that,, don’t touch it. And again, that’s not just the technology, but it could also be a workflow.

32:29 So Mike, just from that perspective, blurb from the book and you think about this and where we’re at with microsoft fabric what do you see as you’ve learned in the last few years on really framing because i’m sure you’ve had a mental model of this already

32:46 there’s a

32:47 lot to unpack here

32:50 i’m trying to summarize to a very large level what we’re doing in fabric think i’m going to argue to the point of whatever you’re doing in fabric at a very high level storage is cheap compute is expensive that’s different than what we used to be building when we had servers on premise yeah where compute was cheap but storage was expensive so we’ve flipped on the head of the system we’ve reversed how we’ve traditionally thought about building systems, right?

33:02 When you’re on premise, when you’ve bought the hardware, when you’ve capital expensed the computers that live there, right? Once the computers show up, you’re just paying for electricity and the people to maintain them.

33:14 It’s not just,, you are paying for them. But at that point, you want to use the max.

33:26 If you don’t use all the compute on those machines, you’re leaving money on the table every day, right? You don’t want to buy a computer that’s 10 times larger than you need and never use half of it.

33:38 Right? Yes, you’re growing for some capacity.

33:50 You’ll potentially grow into that because it’s expensive to get another one of those compute units, more compute on-premise to make it all happen. It’s expensive.

34:02 You’re physically in a building, you’re doing all these things, right? So the compute is expensive initially, but then once you have it, you wanna use it to the max capability.

34:15 On the flip side, your storage on-prem, you have to keep buying more drives and these drives are not cheap and you gotta buy a lot of them and you could have a whole, someone maintaining this. So there’s a lot of like, yes, the storage part is like more expensive there because it’s hard to acquire the storage capacity.

34:27 Flip that now to cloud, it almost feels like in cloud, they’re saying you can store anything you want, almost nothing, right? Store a gig for a year for 25 cents.

34:39 That’s incredible. That’s a lot of free storage that you’re,, almost cheap, nothing storage that’s there.

34:51 So that is happening in the cloud. And so because the storage cost has dropped down so low, you’re optimizing, you’re building systems that compute things one time and try and save the results.

35:03 And then you capture those results and you read from those results. And so most of what’s happening inside the cloud world is you’re optimizing, you’re designing around how can I compute as little data as possible so that way my systems don’t fall over. Yeah.

35:16 Well, I would argue, first, I agree with everything you just said, how the tables have flipped and how budget conscious we have to be of capacity. I would argue in the framework of today, though, that is one of the elements we’re going to use, but not the only, in a sense, variable that matters to deem something to prefer acceptable or discouraging.

35:40 yeah understood but i’m just saying i’m trying to roll up a very big concept when you think with that mindset if you start with that pillar first then you go into like why does that why are you building things the way you build them well okay now it makes sense why i have bronze silver gold now it makes sense why i’m picking up a table in a notebook and dropping it back down into storage now it makes sense why i’m using semantic models to in memory grab the data out because it can serve queries faster than having to go back to a sql database and continue querying and computing every time right so Everything at the cloud level is about speed.

36:04 And when I’m using compute, I want to use as little amount of compute as possible. And so that I think that overriding principle defines most of your experience and why you design what you design.

36:32 Again, from a technology-only point of view, I would agree. The thing I do want to make an argument, though, of is the fact that I would also lean heavily towards the customer and the developer experience, right?

36:44 If the only thing you’re… And again, I know you’re not saying this.

36:55 I’m talking in general here. Sure.

37:06 If the only thing you’re concerned about is the CEUs that you’re using, at the expense of the organization’s experience around data and quality… then obviously you’re missing the boat.

37:18 There may be things that are going to take more compute than you would like or that some other ways could not or be less capacity. However, the goal of this, again, and what I love from the article or from the book is it’s all about that customer experience.

37:30 What’s best for the customer? What’s best for how we are delivering our services to them?

37:41 And again, one of our services or products Things that we need that we have to take account for is our consumption of our capacity to make sure we’re staying within that budget. But again, the other side is that data quality and the reporting to make sure that we’re actually flowing things the right way and people can get what they need.

37:52 At that point, if that is wrong, no one’s going to care about the CEUs. The first element to me is always going to be about making sure that I can have a repeatable process that other teams can use that can be at scale.

38:04 But more importantly, there’s going to be trust in the data. And whatever that flow is, for me, it would be the primary, or at least 50-50 when it comes to CU.

38:16 So is there anything else that would be, in a sense, be a weight for you when you think of Padu and things in Microsoft Fabric? a weight just like monitoring or valuing the how much cus are being used for a particular workflow or

38:28 tool do i have to just go around cus or do i have to do

38:32 whatever you want so we define two we define our consumption and how we’re consuming based on our capacity i’m trying to not

38:41 answer based on that comment i’m trying i’m trying to think of other things that are like

38:50 And we have that and we have customer or consumer experience.

38:57 I, again, more recently now, I think,, as I’m looking at where organizations need to push themselves, right?

39:04 Mm-hmm.

39:07 I believe we are entering a new era where we need to educate not only, and this is spanning more than just the center of excellence for the business intelligence, right? It’s more than the COE for Fabric and Power BI.

39:16 This is an organizational problem. This is at the entire company level, challenges that are beginning to arise.

39:25 Organizations need to understand what is AI, What is an agent? And this spans every part of the business, all the way across it.

39:33 What are you spending on and how are your employees going to use it? Because they’re going to use it.

39:43 Right.

39:44 Whether you give them a subscription or they go off on their own and buy a $40 subscription on their own, on their other computer and use it separately to do things, right? We had, Tommy, we’ve had this complaint from people using Excel and calling them shadow IT, right?

39:53 Well, they’re only shadow IT-ing because you haven’t given them the tools that they need to do their job.

40:03 And they’re just

40:03 going to go take data and go do something with it. So at the end of the day, the requirement for the business is still get stuff done, still sell products, still do the job you’re meant to do.

40:12 That doesn’t change. If you don’t give people the tools, they’re going to build massive Excel documents because that’s the tool they have.

40:21 And that’s what they’re going to do to manage this data and information because they don’t have the tools they need at their disposal to build what they want. And you can talk all about security and these other things.

40:30 Sure, fine. If anything, you’re going to need to be more integrated with AI agents because they’re going to need to help you build stuff that’s even more secure.

40:38 The surface area of what means security now has just greatly increased with agents. against your organization.

40:47 And if you’re not thinking about security as a paramount solution for what needs to be happening in your company, you’re in a world of trouble. I get really nervous when you start using other backends other than Fabric because Fabric’s been battle tested.

40:56 I know it, it works well. It has access to the things I want.

41:05 You can go build things on

41:06 Airtable. You can go build backend systems on other solutions, but I feel like those are additional weak points.

41:13 Isn’t one of your favorite points if you’re talking to a client or a potential customer and they’re like, yeah, we’re going to use an obscure warehousing tool you’ve never heard of. You’re like,

41:24 why?

41:25 Why would you go build that from scratch? Right, right.

41:29 Or why would you go use some other tool that’s, okay, it’s got a couple interesting features on it. Understood, right?

41:37 But you’ve already got this Microsoft tested system that has all the compliance pieces that they’re trying to make it as secure as they possibly can. Why are we not just using the system that’s already there in place?

41:45 What if you need support? Who are you going to go to?

41:51 You can hire people or get a consultant in the Microsoft space. But that’s my own tangent.

41:57 I always love hearing that. So, Mike, what I want to do with you, I think we define the consumer experience or experience overall, capacity, usage, and consumption, and AI capabilities and integration.

42:03 What I’d like to do is go through a couple things in the fabric world, in the ecosystem, and I want to see where they land for you in the pato thing. So let’s define again.

42:09 Preferred, this is the recommended approach. We need everyone to do this.

42:15 We obviously can’t force them, but this is where people get hired and promoted because they’re implementing this. Accept it.

42:21 It’s fine. It’s okay.

42:28 No one’s going to make a big stink of it. We’d like you to do the preferred approach, but it’s fine.

42:34 It’s cool. It’s good.

42:40 Discouraged is if we see it, it’s something that we need to make a note of it. We’re going to talk to you.

42:46 And we’re going to say, Hey, we gotta, we gotta get to something else. And, but, and so it is, it’s not just a, we’d rather you not it’s listen, let’s not do this.

42:52 We got to move on. And unacceptable is a fireball fence.

42:58 Unacceptable is you need tomorrow. This is your priority. If this is going on.

43:04 Yeah. So

43:05 would you agree with those four levels of how for Padu? All right.

43:11 Yep. So let’s dive in Mike.

43:18 We have lake houses. We have warehouses.

43:24 We have data flows, gen two. We have for let’s start with the storage areas, SQL databases, warehouses and lake houses.

43:30 Obviously I would say a lake house is obviously the preferred as what I am seeing out of those three. Absolutely what lake houses capabilities are sequel analytics endpoint.

43:37 It’s really just the bet to me the best way I’m putting lake houses at preferred, and there’s no question there.

43:46 I agree, and I am debating in my head about the SQL database. That’s the one that I’m

43:52 debating. I was going to ask you

43:54 about that one next. So I think for storage systems, you can store things in SharePoint.

44:02 Acceptable is a preferred no. I’d rather you get it right to the lake house.

44:09 The lake house has been getting better at storing files and things. SharePoint, everyone uses it everywhere.

44:17 It’s one of these, it’s going to be all over the place. And now with SharePoint integration to OneLake with mirroring, it’s gotten a lot better, right?

44:24 So the whole pattern of using it is pretty decent. I’m not opposed to it, right?

44:32 So it’s,, I’ll put SharePoint in an accepted. Discouraged would be making a bunch of files on your computer and trying to gateway your way into your computer.

44:40 Maybe that’s even unacceptable, right? That might be to the level of like, hey, you can’t be putting files on your local machine and using a Power BI gateway, personal gateway to synchronize that data with the service.

44:47 That’s very much on the lower end of the spectrum for me. So that is definitely not acceptable.

44:55 SharePoint is on there accepted. Lakehouse is definitely in preferred.

45:03 And I think it depends on what data. Yes.

45:10 Go ahead. If the data is batch, Lakehouse all day long.

45:18 Lakehouse and notebooks love it. Great pattern.

45:25 Seems to be reasonably cheap to get moving. I like it.

45:33 It’s easy to work with as a developer. AI and agents are getting much better at building on top of that pattern.

45:41 happy when it comes to SQL databases. I still think you need them, but when you’re talking about like the expansion into fabric apps now and real time data, and I want to have a form and I need something that’s going back and forth in real time, it doesn’t really make a lot of sense to now have a lake house.

45:48 It becomes a lot more data management. I think where a traditional database is, in this case, T-SQL database, that’s from the Fabric SQL database, makes a lot of sense.

45:56 Now, I will say there is a trade-off for using these two approaches. The trade-off is the SQL database does come with a higher capacity unit cost than using, I think, notebooks and lake houses to do batching.

46:04 If your data is really large, you’re going to consume more CUs. But in large data with big notebooks…

46:11 you would assume big data, lots of compute, you need it. The parallel, the more you use, the more you pay.

46:19 There seems to be a minimum threshold for the SQL database that if you’re even on an F2, just running the SQL database will just take half of your capacity immediately. It’ll just eat it right away.

46:26 So even on the F2 level, the SQL database will consume some of that.

46:35 Now,

46:35 interesting, Tommy, there is now a pay-as-you-go, they call it on-demand billing, I haven’t seen the numbers yet. There is no blog post around this.

46:41 I do not know. But I have to imagine that the Fabric SQL database will be on that F0 or the non, I guess it’s an F0.

46:47 I don’t know. I think that’s what they’re calling it.

46:54 It’s going to be a pay-as-you-go licensing for Fabric.

47:00 Yeah.

47:01 Which that does make sense to me because now my SQL database doesn’t have to eat my normal capacity. And if the SQL database gets hit harder or not as hard in certain situations, I still get to use it and I just get paid for what I use inside the database.

47:13 So that to me makes a lot more sense to have that actually on-demand bill me when I use the database because I may have a lot of them.

47:25 I’m okay if they call it F0 as long as they end in X at the end of

47:30 it. I’m going to get the Fox subscription.

47:33 No, no, no. Do you not remember the game? F zero X.

47:38 , I remember the, I remember the Nintendo game. Yeah.

47:41 Yeah. Anyway, I love it.

47:43 It was called. I was better than it did.

47:46 Sorry. I was worried about it. I

47:48 was actually looking at the word F O X thinking Fox, like Fox is in the woods.

47:53 Yeah, I think it was

47:54 F-Zero. Star Fox?

47:55 Star Fox was also a fun one. No, this was a racing game.

48:05 Anyway, we’ll put that in the parking lot. I want to go back to the main…

48:15 I should have landed better. The bigger point that you were saying, so I want to back up when you’re talking about the databases.

48:25 You were making a very important concept change here that I think is very important for those listening. You cannot put these tooling in an isolated category.

48:35 Or rather, a better way of saying that is it’s really important, be it lake houses or databases, you can’t say the preferred, accepted, discouraged, isolated from the work stream it’s going to be in. Because this is really important, Mike, where a database, if I am doing application development and I need that application to talk to other systems, The preferred is the way to go for a SQL database or SQL database is preferred.

48:45 But if I’m doing analytics at this point in Fabric and Power BI, then it’s probably accepted. It’s not the preferred way.

48:55 And I think there’s a lot of these different toolings, though, that you can’t just say, oh, is a lake house preferred or accepted? It’s like, well, what are you using it for?

49:05 What’s the workflow of this that’s also going to dictate where does it actually land? I don’t know, actually, if any.

49:15 There’s only a few things on the list that by itself you could put in a category. Would you agree or disagree with that?

49:27 You mean without, like, a bunch of caveats? Like, if you’re doing this and this and this, then it’s

49:31 this. Yeah,, this is…

49:34 The database was a perfect example.

49:37 Yes, I think this is really around, like, use cases. Because you can’t…

49:46 there’s so many different use cases of what you’re trying to do. Like it’s the three Vs of data, right?

49:54 If you don’t evaluate the three Vs of data, that does change what tool and technology that you do, because not one tool does everything exactly right. And what we’re seeing, I think a lot in the Spark engine, the Spark engine is, it came out,, when I look at the progression of things here, Spark came out, We were learning languages to run it.

50:03 We were learning Scala. We were trying to use that to make it very fast and proficient.

50:12 But then someone at Facebook said, hey, Spark should just run on SQL. And so they made Spark SQL.

50:21 And so there’s a language that’s something we already know. And we brought all these developers in who know SQL.

50:29 It can also write things in Spark. Slightly different, but yet mostly the same language.

50:38 And so as we kept doing this, we keep going back to this like SQL language on top of different technologies. So while the technologies are shifting, they’re good in their own right in different ways.

50:47 they all have that their strengths and that you wouldn’t get many languages of databases and data queries if one just did everything perfectly. People are continually evolving technology to do data storage different ways because of the way they need to access the information or how to get it in or how to get it out, how you optimize it, how you make it fast.

50:55 Those are consistently the problems. And so people are trying to design systems around them to make it easier to use.

51:04 Well, then I think that brings up a good point, though, because you can’t expect, obviously, you’d want everything to be preferred for all the languages, SQL, PySpark, DAX, Git, Power Query, if we’re still doing that at this point. All those different languages, obviously, we’d want preferred, but I don’t think that’s a realistic expectation for a single person.

51:17 Now, going off of that, Mike, if you even look at, let me ask about Git integration, for example. That’s when most developers are still getting into source control, right?

51:30 I would agree. So where would that land for you for Git integration and deployment pipelines?

51:47 And this is the,, we’re talking about Padu stuff, right? So we’re saying, like, where does Git fall?

51:52 Exactly. I would probably maybe even go further than that.

51:58 I would say the topic is continuous integration, continuous deployment, CICD. And one of the topics of this is, so where do we fit on these things, right?

52:03 Deployment pipelines. Where do those fit?

52:09 This new thing called the deployment pipeline scheduler. Is that what it is?

52:14 Yeah.

52:15 Planner. Deployment pipeline planner, right?

52:25 So how do you deploy that? Where does that fit?

52:35 I need a little bit more time with that before I can really maybe give a really good analogy on that one, but I like where it’s going. And then you also argued around Git.

52:45 So my preference, I think, Tommy, in this one is I live on GitHub now. So it has become, GitHub has become my social media platform for better or worse.

52:55 When agents started writing their own README files and having really good descriptions of what a project is and images and like, we could build projects before, but like any project that had like three sentences and like, this is my project. And then a sentence that this does database.

53:05 like there’s zero information. I don’t understand why this project exists, how to get started with it, what code I should run.

53:15 Like none of that stuff exists, but you, now it’s a one line command, update the read me for this project. And it goes through and it goes, and it writes,, hundreds of lines of information so that another agent or another person can just show up and understand why it’s there.

53:25 Right. So to me,, get and get hub is, are essential for me anymore.

53:35 So I’m going to put that under like that for me, I’m going to definitely put that in my preferred area, fully realizing that not every team understands how to properly use Git and branching and making all the code changes happen. This is a language that comes from the app development world.

53:45 that comes from software development. And I think many data teams, even in the SQL database world, we would use Git.

53:55 So let’s say you’re coming from more of a legacy SQL world. We have a lot of really smart people who’ve been doing SQL for many, many years, and they maybe conceptually understand Git, but they don’t actually use it in a continuous integration, continuous deployment way.

54:05 We’re not actually automating, unless you really get pushed, your team in the data world is not fully automating I make a change here in the code. I commit to dev, and then everything builds, and the code and or the data just gets built in the next system.

54:15 So I think we’re seeing a lot more of this integration across companies now moving forward. So I would probably put in accepted would be the deployment pipelines that already exist.

54:25 There are some weaknesses there. And then I’d probably also put in accepted is this new deployment pipeline.

54:35 The deployment planning, right? So I think that’s also one that I would put in accepted.

54:45 One of those may move to the top depending on how good they work. It’s going to really depend on does it have the features I need to properly deploy things?

54:55 And my biggest hangup right now is how do I move the data from dev to test to prod? How do I preload or not preload or clean up or not clean up data throughout this pipeline?

55:05 And I think that deployment plan will help with that.

55:15 Well, it’s interesting, though, too, because what would be the layer underneath it, then? If Git would be preferred, right?

55:22 Or at least in Git, that’s what we would rather you do for an enterprise. It gives you the most

55:29 flexibility. But what would be underneath it?

55:32 but what would be accepted if you that’s preferred then what’s under what’s accepted

55:37 accepted is deployment pipelines and deployment plans right and then what i would this is going to debate for me whether it’s discouraged or unacceptable i think discouraged is making is publishing anything into prod so you have like a dev qa prod type environments

55:52 wow what world we live in today

55:55 Well, it’s a process, Tommy. What people are doing, and they’re breaking things.

56:09 If you use a deployment pipeline and then you go publish a new report into production without using the deployment pipeline, you break the connection between the semantic model and the report, potentially. That’s totally unacceptable.

56:23 That’s taking things down. So letting people publish across different environments…

56:38 discouraged right and i’d maybe even argue like not having a proper segmentation of which users can access not having a deployment manager right there’s there’s there is an individual a persona here that is looking at okay we need to get changes into qa and into production we need a deployment person that says i have the ability to move things through the environments but you’re blocking many of your developers your developers should live in dev only They should have one workspace that they’re looking at.

56:52 And if they can’t fix it there, they can look at, they can observe, run things from the other environments, but they shouldn’t be able to deploy to them. And so I really think these production environments, when we’re talking about databases, really need to be in lockdown. So it’s crazy that you’re

57:06 right. Yeah.

57:08 it’s not crazy. This is logic. I’m not saying you’re crazy.

57:13 You’re crazy that it’s right where three years ago. Yeah.

57:19 Now,, wasn’t the case and how quickly you’re going to not only say that’s acceptable and good, but it’s like, no, if you’re doing what you did three years ago, you’re absolutely wrong. Like that’s discouraged.

57:24 That’s crazy to me. That’s how quickly things change.

57:31 All right. Yeah.

57:37 Add to that mix here. Also, Tommy and another one here too.

57:43 I’ll, I’ll throw in this mix, which I think is important. Add to this the additional agent space, right?

57:49 Now we’re hearing everyone talk about, look at me build a report with an agent. Look at me build my model with an MCP server.

57:55 Look at me. Awesome.

58:02 Great. but agents can greatly screw up your stuff.

58:08 And the idea is it can screw up your stuff so fast. And so for me, it’s a speed thing.

58:14 It’s the speed in which it can build good things really fast, but it also can break things really fast. And so when I’m in that break things really fast stage, like if you’re building something with an agent, it may delete an entire table for you.

58:20 Oh, yeah.

58:21 Whoops. If you’re working on a live XMLA endpoint inside the service and the agent says, oh, I don’t think I need this table anymore.

58:27 And you’re too slow to say, stop, wait, don’t do that. And the agent deletes a fact table that you’ve loaded.

58:32 Potentially. Those model changes are live.

58:37 It saves the changes as soon as the model’s going. So what are you saying

58:43 about where does that fit then? Do you discourage to build a report from an agentic tooling?

58:49 No, no. I’m saying this is adding to the CICD part of things, right?

58:57 So I’m adding to this the idea of if you’re not doing some version control, agents just add so much more potential for big failures and big mistakes. knowing that you have to have some Git.

59:06 And again, at this point, it’s not rocket science. There’s a very easy place to get started with Git.

59:14 You can even do Git locally on your computer and not even connect it to GitHub or Azure DevOps. So you don’t need to go full bore on the whole system, but even having a local version on understanding what does it mean to build and edit files with a Git system in place, like that’s paramount to me.

59:23 Like you have to know this anymore. 100%.

59:31 Mike- What I want to do, I’m going to be- Sorry, I’m

59:34 getting really passionate about this. No, no, I

59:35 love it. We’re both excited.

59:42 What I’m going to do is I’m going to be a complete hypocrite because at the beginning of the Padu explanation, and I mentioned that, you can’t take the artifacts in fabric and isolate them. And that’s exactly what we’re going to do.

59:50 Mike, what I would like to do with you is a lightning round. I am going to list an artifact or tool or feature, and we’re going to quickly give our area where does it fit in the Padu scale.

59:57 Okay. So I think we’ll have a little fun with this.

60:05 Obviously, again, the caveat is what the environment is, but just by itself. So first what I’m going to do is export to Excel.

60:16 Analyze in Excel or export to Excel? Export to Excel.

60:18 Two different things. Discouraged.

60:22 Discouraged for me as well. Analyze in Excel.

60:25 Encouraged. Accepted.

60:28 Accepted. Accepted, yes.

60:32 I would agree with that. Paginated reports in Power BI Fabric.

60:38 Accepted. It’s almost preferred to me.

60:45 The UI is not good enough to get exactly what you want. But it’s definitely in the accepted range for me.

60:52 Paginated reports are very powerful. The UI that Microsoft gives you is not powerful enough to wield what is really happening inside paginated reports.

60:59 So I think it’s definitely for me in the accepted range. It could be preferred if there is a better UI to make it easier to build paginated reports. It’s just not.

61:06 I’m the other way. I’m discouraged unless of unique circumstances for paginated.

61:14 I think unless you are sending invoices or,, you’re using that for email or automation, then I’m putting a discourage. All right, Mike data flows. Gen two.

61:25 Accepted. Accepted?

61:29 They’re killing Gen 1. I get it. Gen 1’s not

61:32 on my list, so don’t worry.

61:34 , Tommy, I’m leaning on the hope that agents will be able to take a Dataflow’s Gen 2 and rip it out and put it into a notebook at some point. right?

61:39 I’m fine. Business build, build what you want.

61:44 It’s accepted. It’s, it’s a good solution.

61:49 It’s solid. It’s powerful.

61:55 People know and understand it. It’s, it’s the biggest, you’re not going to push.

62:00 You can push for it to go into notebooks and do other things, but it’s by far the most versatile tool that most people know in the power BI world. Everyone will be comfortable going to data flows, gen two from power BI.

62:05 And you have to take that as a, as a stepping stone into something that’s better designed.

62:11 I could not disagree more with that. I love the data flow to experience, but until the cost goes down, again, if that’s one of our elements, capacity,, what are we doing here?

62:17 It’s discouraged. It’s not unacceptable.

62:24 If they are using it, we’re going to talk to them and say, what can we do here? I believe I could be

62:30 swayed to be discouraged on that one. It is slower and more expensive than I think the other tools that are out there.

62:36 but I, there’s just so much of people that need it. I’m going to say accepted.

62:43 Okay. All

62:44 right. I’m very, I’m very much on the line on that one.

62:46 You, if we, if we talked about this for a whole episode,

62:49 it’s way to be

62:50 discouraged on this one.

62:51 Give me two and a half minutes. And I think I can make you discouraged.

62:54 Yeah. All right.

62:57 Having environments in your workspace. Environments. You

63:01 mean the spark environments? Shared libraries.

63:05 Yeah., I think that’s encouraged for me.

63:09 Encouraged? Preferred. Sorry,

63:10 preferred. Sorry, preferred.

63:15 Okay. Preferred.

63:20 I think you’re going to need to have some of those established and set up. I think you’re going to want to have environments, particularly when you’re talking about environments for Spark.

63:26 Those are things that you’re going to want to have inside there. I think it’s a better way to control what’s happening inside your workspace by having some environments show up.

63:32 I would completely agree with that. We really prefer that you’re not just using the default data agents.

63:41 Ooh, data agents. I don’t think the lemon is worth the squeeze on this one.

63:45 I’m going to say that’s discouraged right now.

63:48 Wow. Okay.

63:51 Wow. I think data agents is consuming too much to you.

63:57 I think data agents is only wrapping around some of the API layer that is actually already there, which you could use for free. You don’t need a data agent to go talk to it.

64:03 Data agents does give you the MCP server experience, but if you look at what… Kurt Buehler and Maxim are building.

64:09 There are CLIs, there’s other, like at this point anymore, you can build the CLI or things that you need to go make the stuff that you want, right? So like, I’m also like, if you don’t have, if there’s not something Microsoft’s providing you, there’s a whole bunch of APIs, just go build what you need.

64:15 Like spend a little time with the agent. Hey, I need an agent that does this, this, and this.

64:21 How do I build this with just API calls? Then there’s no MCP.

64:27 Then you don’t need an agent. You can just build whatever.

64:33 And this is, we’ve talked about this before in the podcast, Tommy. There’s this idea or concept around a custom harness.

64:39 You’re going to use a large language model. You need it to do something.

64:45 You build a process and the process and the combination of your custom process and the LLM is your custom harness. It’s a CLI.

64:51 It’s a tool. It’s a UI.

64:57 It’s now a fabric app, right? Build a fabric app that does the things you need.

65:03 It can talk to all the APIs for you. you don’t need to go grab something else that’s more expensive.

65:09 So I’m going to say discouraged.

65:16 I, I would say, unless it’s to be, if you’re using a data. Yeah.

65:19 I, what? I want to say it’s accepted.

65:22 It’s fine. thing, but because I think it can be used the right way.

65:26 All right. User data functions.

65:32 While there won’t be many people that know how to use them, I do think this is an accepted way. I think especially when you’re talking about Fabric apps, definitely preferred.

65:44 Again, talking about some items there, right? I think I’m going to push Fabric user-defined functions as preferred just because you need to have them in Fabric apps.

65:57 really oh yeah yeah because i’m thinking of that more in terms of for trans analytical task flows but fabric apps are you going to put that as preferred as well because again heck yes okay and i think that’s all i got here so what i’m gonna give you can i give you one more time please please

66:20 metric ontology item

66:25 , as of today, October 8th, where a lot of these features are still waiting to come out, I’m going to put these at discouraged. Close to unacceptable.

66:38 in terms of not a lot of companies have really started using them and even consultants and then applying them at scale on what they’re used for. I know this is going to change, but I’m saying as of right now, I don’t want a bunch of these ontologies that are ill defined out there.

66:50 So this is one of, I’m going to put my governance hat on and say, yep, not yet. We are waiting.

67:04 So you’re saying discouraged.

67:06 I’m saying unacceptable. Until now, because again, I do not want this until we have a good grasp on its capabilities and how we’re going to do it.

67:17 We need a plan for this. No one’s going in.

67:28 No one should be going in. You’re going to get me on my soapbox.

67:40 No one should be going in randomly and marketing going, I’m going to create an ontology today and just starting creating whatever they want. ontologies are going to be this core component of how analytics and ai are flowing through your organization so this is not a managed self-service type of feature if we’re just just willy-nilly creating these then it’s unacceptable that being said i know it’s going to be preferred i just don’t think we’re there yet we don’t have the proper i think framework in in place yet

67:53 Ooh, I have thoughts on this. We’ll have to save this for another day.

68:01 I would probably put it in discouraged or I’m not sure if I would put it in unacceptable, but I think I would put it in the discouraged bucket for now. I don’t see the value for the ontology piece at this point that the message from Microsoft has not been clear.

68:10 I don’t understand how it clearly, I don’t understand how the ontology easily relates to an agent and how the agent is using it to get better answers or help you build better things, right? Right.

68:19 The agent should use the ontology. And I would argue it’s not an ontology.

68:27 That’s not what we’re talking about. We’re talking about a knowledge graph.

68:35 That’s the word that we should be using here because it’s it’s a mix of business logic and physical items that exist in fabric or tables that exist or concepts that exist in fabric. That’s what we’re talking about that those things need to be a knowledge graph, which I think is more comprehensive than just the ontology.

68:44 And I also want everyone to understand, too, Microsoft, if you’re listening, I’m not saying ontologies forever are going to be unacceptable. I’m saying organizations, when they start creating them, they better do it right from the onset.

68:51 That’s why I’m saying unacceptable as of right now.

68:59 Yes, agreed. All right, Tommy.

69:01 That being said, let’s give me a wrap up here. Let’s go ahead.

69:03 And where else can you find the podcast?

69:06 Oh, my goodness. You can find it everywhere.

69:09 Apple, Spotify, wherever your podcast. Make sure to subscribe and leave a rating.

69:13 It helps us out a ton. Do you have a question, idea, or topic that you want us to talk about in a future episode?

69:16 Head over to PowerBeat. tips slash podcast.

69:19 Leave your name and a great question. And finally, join us live every Tuesday and Thursday, 730 a.

69:23 m. Central on all PowerBeat. tips social media channels.

69:27 Awesome. Thank you all so much for listening and participating in this conversation.

69:30 This has been super useful. Hope you found this valuable as well.

69:33 And as we’ve been doing here recently, here’s your treat of the day.

69:56 An agent draws a chart before I take a breath Tailored to the prompt like he can guess what’s next Sounds like freedom when the old walls fall away Then the open field shows a different day When the sandbox has no lid Useful needs a user and a job it did If

72:31 it

72:31 doesn’t serve them better It has no right to stay Freedom without purpose is a pretty lie Keep a little railing when you’re handing the keys Design systems matter more than infinite degrees Choose a slice that serves and leave the rest unsaid The fence was never prison, it was focus instead Standing in the desert with the whole sky lit I still need a compass for the choice I make

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.