Self Service with AI Part 2 – Ep.540

Episode 539 only got halfway through Anthropic’s self-service analytics article. This one picks up at the two layers that were left: skills and validation. It also produces the sharpest position either host has taken on governance in a while, plus a direct appeal to Microsoft about two features that would make agent-driven analytics actually work on Power BI.

News & Announcements

• Replit + Microsoft partnership — Replit’s partner page for the Microsoft integration, which is heading into public beta. The pitch is starting a Replit project that wires directly into Fabric’s security, backend, and functions, so the app knows how to build against your data estate. Mike was asked to share the sign-up several times — if agentic app development on Fabric is interesting to your organization, this is the way in.

• How Anthropic enables self-service data analytics with Claude — The same article the hosts started in Ep. 539. This episode covers the skills layer, where markdown docs act as routers that load domain detail on demand, and the validation layer, which splits into offline evaluations against static snapshots and online evaluations against live data.

Main Discussion

Topic: Skills, validation, and the governance prerequisite nobody wants to hear

The through-line is that both hosts have stopped building tools and started building the things that build tools. Applied to analytics, that means skills attached to models, documentation the agent can actually read, and evaluations that tell you whether any of it works.

• Tommy’s hot take: no agentic rollout without data governance. He’s explicit and invites the argument — he will not deploy agentic solutions on a company’s Fabric data unless governance exists first, because it will fail. The article was the final stamp on a position he’d been circling.

• Build the thing that builds the thing. Mike’s framing for how his own work has changed: the durable value isn’t the artifact, it’s the system that produces artifacts. Tommy compares the adjustment to learning Power Query — you slow down first, then move much faster.

• Ambiguity inside one model is already fatal. Tommy demonstrated a single model with seven measures scoring 90% similarity to each other. If a human can’t tell which to use, the agent has no chance — and it will still pick one.

• Perspectives are Power BI’s most untapped feature. Both hosts land hard here. Perspectives could scope a model down to what a given audience needs, but there’s no UI in Desktop to build them — Tabular Editor is the only real tooling, and you can’t select one in Fabric’s exploration view.

• Microsoft, we need skills for models. Mike’s direct appeal, delivered to camera: “prep for data is not enough.” Skills per model, plus perspectives, is the combination he wants. Anthropic appears to run skills per domain or per model, which is the pattern Power BI has no story for.

• Put the grain and the scope in the model. Mike admits he under-documents this: the grain of a table, its scope, any exclusions on a measure. That context is what agents need — and what users needed all along.

• Skills go stale as fast as your data changes. The article notes skill docs describe models that change daily, so without active maintenance they rot. Tommy’s conclusion is that companies must budget time for building and maintaining skills, or the whole thing quietly stops working.

• Offline and online evaluations both matter. Offline runs against a static snapshot so results are comparable across runs; online tests against live data. It’s the closest thing to the measurement problem raised back in Ep. 538.

Where does all this context live?

Neither host has a satisfying answer. TMDL can hold model-level detail, Anthropic keeps skills in GitHub repos rather than hardcoding locations, and some of it arguably belongs inside the model itself. But business context also includes roadmaps, Word documents, and decision logs that don’t fit any of those places. Mike points at OSI — the open semantic interchange released by Snowflake — as a promising direction, enough that he’s already added it to the lineage view work.

Looking Forward

Document the grain, scope, and exclusions on your most-used model this week — it’s the cheapest thing on this list and it helps your users as much as your agents.

Episode Transcript

0:25 Hello everyone and good morning. Welcome back to the Explicit Measures podcast with Tommy and Mike. How are you doing, Tommy? Tommy? Good morning, Mike. That was too much of a close-up, but I’m doing great. Awesome. Awesome. Welcome to everyone. We’re going to continue on our conversation around this article from Cloud we found earlier in this week called self-service with AI. It’s really interesting. There’s a lot of things going on there. So we want to expand or expand or look at a bit more of this article and figure out what’s going on and just going to unpack a bit more. We only got about halfway through. We need to get through the skills and a couple other

0:55 through the skills and a couple other governance areas here that I think would be relevant to talk about with our audience. And as we get closer and and move do things more with AI, it seems to be very relevant for us as well. All right. That being That being said, Tommy, where else what other topics you have for us? Anything else that you were looking at or thinking about discussing? Well, a little birdie told me, Mike, that you actually got to talk with a team, I believe, from Replit. Yeah. Yeah. Yeah, we talked with Replit earlier today or yesterday. So late yesterday afternoon, I got a meeting going with

1:26 afternoon, I got a meeting going with Replit and Microsoft Fabric team. So there’s basically the Rayfin team and Replit are coming together. And while I was at the Build conference, I was able to observe and and watch there was an announcement made. Replit and Rayfin are coming together and being able to partner. And so Microsoft is being added as like a first-party partner to the Replit system. So you can go in, you can build a direct integration. We actually went through a demo and a video of how Replit and Rayfin work together. You can go to Replit, make a project, start it,

1:58 Replit, make a project, start it, directly integrate it with all the security and back end and functions and it just knows how to build the pieces you need directly with Microsoft Fabric, which I think is really exciting. Do you think it’s really because of Fabric that this integration started? No, not at all., Replit Replit and and is a lot of like these other companies. Replit, lovable. It’s It’s trying to big bring agentic app development directly into the front line of business workers, right? What can we What What tools can we give directly to

2:28 What What tools can we give directly to business users? What can we hand them to directly use tools that are useful to building apps., that’s what it is. It’s a simpler way. It’s an easier way of building apps without having to go to VS Code, use a command line. It’s It’s trying to simplify some of that experience. I think the things that Replit does is amazing. amazing. And what’s good about this, too, is Replit’s been around pre-AI. It was much more of a developer tool to help host something or an application and means to do so. So, I like seeing these companies

2:58 do so. So, I like seeing these companies who already existed and that are now like all of a sudden they’ve really they they have transformed their business because of AI, but they already had a strong foundation to begin with. So, I really like this tool being brought to Rayfin because now it simplifies a lot of what we need to do, Tommy, to build apps and put things together and and and design them for our applications. So, for that regard, I I really like what they’re doing. I like the approach they’re taking here in this. I think this makes a ton of sense. I’m very excited to to continue diving in. There I do want to make an

3:29 diving in. There I do want to make an announcement around this as well. If you are interested in building apps around Rayfin, Replit, and and if that’s interesting to you, there is a sign-up form. They’re going public beta right now. And so, if you want to go get into that public beta with your organization, Replit requested that I share this multiple times. So, there’s a link in the chat window, also in in of this video as well, Just discussing where you can go find Replit and where you can go find find their partner portal and what you can

3:59 their partner portal and what you can see there. So, the that information is existing in the chat window. Go check that out. Go out to the forum, fill it out, submit your company or you. They’re looking for beta testers on the application today currently. So, if you’re an avid Replit user, you might be able to get in the the program and start testing things out sooner than later. I I feel like we should do a special shout-out to anyone from the podcast. I don’t know if we have like a coupon code. I know it’s a free demo, but I mean, Mike. , Mike. It’d be nice to have some kickback there, like a little URL parameter that says from Power BI Tips

4:30 parameter that says from Power BI Tips or whatever. Yeah. Yeah. But anyways, it there’s not really, I don’t think people are like signing up to Replit. I think they’re really looking for people who are kind really looking for people who are avid Replit users and understand app of avid Replit users and understand app dev and then exploring how they would build those things inside Fabric. The demo that we showed was a very much a report that was being built, but the team was very much in the clear about, well, it’s not just reporting. It’s reporting, it’s a SQL back end, you can shake handshake data between between the, the Replit app built side of things and SQL Server. So, it’s it’s

5:00 of things and SQL Server. So, it’s it’s way more than just, wow, look at this new pretty report I vibe coded with an agent. It’s more, look at this application that has two-way data interchange between the app and then the back end side of things, which I think is incredibly useful and very very helpful for us. I know this is probably a little more difficult for Microsoft to do, but I think one of the beauties of Power BI was the gallery. And I would love to see a way to do like a racing gallery where people can submit and necessarily have a validation or test. You’re on mute, my friend. So, oh. They muted me? No, I’m not on mute. It’s

5:31 They muted me? No, I’m not on mute. It’s Oh. Oh. so, we are building one, Tommy. We have a thing collections. So, Power BI Designer has now added a whole bunch of extra tools. So, it’s a good lead-in. If you have checked it out as a workload, we have built a custom workload. It was all about designing templates for Power BI reports, but we’ve extended that. We now have theming, we now have layout design, we now have lineage, all these other really rich tools that we’re using to help us build and become more effective with Power BI and Fabric. That tool exists today and there’s one tool called collections. A collection is a whole

6:02 collections. A collection is a whole bunch of things, reports. we have a whole bunch of notebooks from Spark that are in there that you can go use and and deploy with one button. we have some user-defined functions we’re looking at adding. We have some goals here short-term, but the next thing we’re going to add is a library of RayFlow projects. So that way you can just find a a single place to go see where people are publishing their projects on the internet and again, you need you can have to pull them down, run the code yourself, but at least you’ll have a single place to go look for all them. So yes, we’re building the

6:33 them. So yes, we’re building the community around RayFlow, we’ll have that hopefully available in the next week or two. So stay tuned. We’ll make the announcement when we get it going. When you do, I I would really like to see in the workload when you open it, it says it’s Power BI Designer and then just in a banner it says it’s a whole bunch of things. All the things. It’s a whole bunch of things. Yeah, all the things. Well, it’s a toolbox., it’s it’s Power Designing Solution ground up. so so We’ve been building a lot of things specifically geared towards like the the reporting side of things, right? How do you make the report better? How do you

7:04 you make the report better? How do you improve the report? What does a theme file look like? we’ve also added recently the ability for you to use agents. So you can go to Foundry, stand up your own model and use said model with within our application. So you with within our application. So, you want to bring in Deep Seek, know, you want to bring in Deep Seek, you want to bring in all these other models that are more efficient to run, you can do so and bring them directly into your application and therefore you just register the model and then it will work with the Power Designer workflows, which I think is really exciting. Awesome. Awesome. Awesome. That being said, I don’t have

7:34 Awesome. That being said, I don’t have anything else for the Replicate conversation. I think Tommy, you have to talk about some deals for Fabric. so this is a little more beef from the street, so I’ve been using this workflow when it comes to using a different harness with notion to create cloud instructions to go to an MCP for Power BI semantic models. It works great. great. Mhm. Mhm. And we’re going to a few projects now that’s more integration notebooks and data engineering on the fabric side. And I’m like I was curious since I can’t directly do that same workflow

8:06 directly do that same workflow in in the MCP at least on my local machine. I know there’s the MCP server, but I wanted to see if I could utilize the make sure I get the right instructions that notion creates because notion would create my instructions for Claude. Could I do that utilizing these skills that already exist? So, what I actually did was I have a skill converter in notion that will take any skill that was designed for Claude with if script space because you got notion doesn’t do

8:36 because you got notion doesn’t do development. It doesn’t actually build applications. But it can actually convert any set of skills into something for notion to where it understands the context of whatever that skill is. So, Mike, I have this really cool workflow now where I have a meeting with someone and they’re explaining their data integration. They’re explaining, “Hey, this is what’s important to us. We’re trying to get X, Y, and Z. The API comes this way. Cool.” Well, now I can just tell my notion agent, which I have a fabric advisor and agent, utilize and

9:07 fabric advisor and agent, utilize and knows to use these fabric skills, the skills that actually came originally from the skills for fabric repo. And it says, “All right, we’re going to connect to this lakehouse. Why don’t you go ahead and update the instructions for this project?” and then then I can now feed that in. And it’s really cool, Mike, that I think we have to understand that skills are interchangeable. And with these harnesses, it is the hardest thing to try to manage the skills across different platforms. Mhm. Mhm. But they can be interchangeable. I’m realizing it’s the thing that creates

9:37 realizing it’s the thing that creates the thing is more important than the thing at this point. And I’ll say that I can say that a different way, but my ability to have my agents create, in a sense, it’s all the other agents or the automation, is what gives me the power. It’s not the one-off artifact that I create, but it’s that long-standing view. So, I focus a lot of my development now on the thing that creates the thing. Rather than the thing itself. Yes. Yes. And it’s been a really powerful part for

10:07 And it’s been a really powerful part for me because I have a seamless flow, Mike, where now I have a notebook or a data engineering agent in Notion to go, “Hey, we just talked to,, the data source team at X, Y, and Z. are there any barriers here? You are there any barriers here?, like, are we going to deal with know, like, are we going to deal with any things that may be pose a problem?” Well, look at the notebook management, notebook creation, the notebook authoring skills. It has

10:30 the notebook authoring skills. It has all this context, and we can have this conversation where I’ll tell you what, Mike, again, if if you’re not using this, and if you if you have the ability to utilize these technologies, and you’re not using it, I you I think it’s just behind. And I know a lot of people, again, are limited by based on the,, infrastructure that they have, but, you’re telling I can tell a company I can save you X amount of time. We can get started right away. We can

11:00 We can get started right away. We can get you up and running in a quarter of the time that it used to take. So, Mhm. Mhm. And also, the I I agree. I I think you’re you’re hitting where I am at also, Tommy, which is the agents are good at built like you have to really rethink what your workflow looks like and what you’re doing on the day-to-day of of work, because you’re actually retooling a lot of your mental model around I don’t build a specific tool anymore. I’m actually really working hard on building a system.

11:30 a system. Right. Right. Agents that build systems for you and that really starts compounding effects of, figuring out a a correct pattern of building a statement of work, but then incorporating that workflow with an agent. Now, people may have been saying this all along, but I think for some reason it feels like Microsoft has finally like absorbed what agents are doing and is actually embracing it and moving quickly to develop on top of it. So, the fact that you can now deploy open claw with with Microsoft, you have these new

12:00 with Microsoft, you have these new sandboxes that you can build containers. preview. I don’t think that’s available yet. yet. Correct. It was announced at build, but like the fact that they’re embracing these things and like Shermie’s, like all these other agents that Microsoft says are effective, they have this new agent called Scout and Scout is going to be like similar in a lot of these ways, but it’s fundamentally changing the workflow of how we get business work done. And I think that’s the shift that’s going to be happening is is we used to be thinking of I had to do this work, I have to move things around. Now, it’s slow down again. It this feels

12:31 Now, it’s slow down again. It this feels a lot like when I started learning Power Query, I had to slow down, learn Power Query so I could build better data loading processes, but it made me more effective long-term. So, I I I feel like that’s where we’re at right now. I felt 100% agree with you, Tommy. That’s a great observation about using the agents to build systems now as opposed to just having the agent do like write me a document, write me a document. document. right? Like I want a thing that can create multiple notebooks and understand the context, so

13:02 the context, so Yes, because it’s becoming so much more capable. Oh, other side news, Yeah. Yeah. I believe Mythos for the or Fable the the Fable Is down? It well, it’s back now, I believe. I believe it’s it’s somewhat come back a little bit. it’s been verified a little bit. So, I I think I saw a post on X this morning talking about Fable returning to GitHub or GitHub Copilot can now use it or I think it’s it’s kind can now use it or I think it’s it’s out out now in the wild. So, anyways, of out out now in the wild. So, anyways, just wanted to point that out additionally letting people know that that model is now back in the

13:32 that model is now back in the space and getting models at that level is really going to change what we can do with things moving forward. You are speaking about something that we have talked about the risk here. I actually one of the people I follow said that there was an issue with GitHub or Claude connecting to GitHub. And you realize so many dependencies there. With that were to go down, when we have services or those connections with models going down, even if it’s not the entire thing itself. Yeah. Yeah. Like you we’re going to deal with a point where like I can’t work. Like I

14:03 point where like I can’t work. Like I know I can’t do my work. What am I going to go back to manual? I’m intrigued if you have a take here on let’s say the MCP server goes down and that’s your workflow. workflow. Mhm. Mhm. Do you say you can’t work or do you go back to manual? Like what what’s going to be the future there when it comes to,, is it like the internet going down where like, all right, I guess everyone go home if,, CP server’s down. What is I think the going to be the future of that allowance? Or what’s yeah, what’s really the

14:34 what’s yeah, what’s really the future like that there? Tommy, we’ve joked about this in the past about like the end of the month something happening and,, everyone stops working and and and no longer does things. I don’t think that’s going to like what I’m finding even in my own team and how we build things. I’m aggressively thinking like that’s not how we want to do work. We can’t have a shortened day or shortened week. So, it now I again, I’m internally, Tommy, I’m struggling with this where you have GitHub Copilot. It’s it the cost has gone up substantially. I can turn off that little switch that says stop billing after a certain amount of usage

15:05 billing after a certain amount of usage or or credits that are being used. But now with these license models, there’s like this idea of the subscription and then there’s the idea of the pay-as-you-go modeling, right? And it feels like a lot of these licenses are you pay for a minimum amount. Like I’m going to pay the 30, the $40 plan, the 100-dollar plan. And you get like that amount of effort out of them. And then where they make their money, right, more or less, is when they say, “Okay, you’ve run out of that threshold or you’re doing things that are more

15:35 that are more token-heavy than what they give you. And then you can pay for above and beyond.” Well, you’re like, “I’ve already built the systems, already built the tools, it’s already here. I’m already in Claude code, I’m already in GitHub Copilot.” And you’re like, “I don’t really want to move systems to something else.” Right? And so I think they get you there a little bit and they want you to pay that extra point of of tokens, but it’s it’s not everyone, Tommy. Like a lot of people could just probably get away with away with a 100-dollar Claude code max plan and call it done. And then, And then,, that’s that’s enough tokens

16:05 , that’s that’s enough tokens for the average user to be using them. Only when it’s like,, Tommy has built out 60 agents and they’re all looping and talking to each other and doing this really cool thing with Notion and coming back, that’s when things get like really intense and a lot of the tokens get used very quickly. So, I look at that going, “Oh, wow, that’s that’s the user I think they’re trying to address with these higher token plans of like, ‘Look, you need a lot, you need to pay more proportionally.’ Cuz it doesn’t make It doesn’t make sense to give Tommy, you It doesn’t make sense to give Tommy,, all full rate access of all the

16:35 know, all full rate access of all the agents for 200 bucks if you can just keep spinning more of them up. The model transference is interesting because because of Claude and its ability with the MCP and its ability with skills, Yeah. Yeah. I use ChatGPT for purely personal things now and all my work, all my daily workflows in Claude. If I have a question about cooking, if it’s about what plug do I need for a TV, it’s ChatGPT. ChatGPT. Mhm. Mhm. So, which is interesting with that that because it doesn’t provide the skills,

17:05 because it doesn’t provide the skills, it doesn’t provide all the harnesses, it doesn’t provide the MCP, which is really disappointing for OpenAI, but it is the same thing with Copilot, right? Like those things are essential to what I do. It’s not the model that’s essential to what I do, The ability of the model, it’s the ability of the harness to be able to work with these other elements of what AI is or agentic solutions are. You’re You’re Yes, agree with you 100% on this one. The harness is so key to, do you have a harness

17:35 to, do you have a harness around the large language model that can be adapted? Can you add skills? You change it for what you need? Can Does it Does it have connections and integrations? Does it have MCP things? Like There’s a whole bunch of things that you need the model to retain or that the the harness to have. And I think of it as like my LEGO set, right? The model is like a core part of the LEGO set and then all we’re doing with the harness is we’re able to bolt on additional LEGO pieces or add different stuff to it and we’re just be able to, with more pieces, you can build bigger things,

18:07 pieces, you can build bigger things, essentially. So that’s how I look at it a little bit. So very cool thoughts there. I think we should move on. I think this lends us to the actual article, which I’ll I’ll re-paste the article here as well. So we’re going to continue talking about this article that came out from Anthropic, which I think is very important. We got about halfway through and just quick summary time. It give us like a quick summary as to to get us where we’re at. Where did this article come from? What’s maybe the first half of the article up to the skills part, Tommy? Give me maybe just a very brief summary of what this means to you and

18:37 summary of what this means to you and then we’ll go further through the skills section. section. So in case you’ve not seen this article, Anthropic recently released on their blog an article for self-service analytics with Claude and how do you really enable it? And what it begins to talk about is how it achieves automation and accurate self-service via Claude for tightly governed multi-layered genetic multi-layered agentic analytics stack. And it talks about why it why typically self-service analytics fails, denormalized tables, inconsistent definitions,

19:08 inconsistent definitions, LLMs directly have a false sense of what to actually look for. And it talks about really building up what the three areas or of errors are. Ambiguity Oh, you’re going to have to say that word for me. Ambiguity? Thank you. Ambiguity, model staleness, and retrieval failure. It talks about also how there’s a foundation that they built or really the stack for analytics, the agentic analytic stack. You and I spoke purely on the that beginning but really data foundations and sources of truth.

19:39 data foundations and sources of truth. Data foundations, they went into semantic models, the ability of with your dim and fact tables, what is important, and making sure that’s set up before you even have the agent check over anything. create canonical data sets, enforcing standards. The sources of truth is that semantic layer. It’s understanding documentation to say how important it is from the user side to make sure that you are documenting and providing that verification. The other side of the coin is what we’ll be talking about today, is the second or the last two layers of the stack, which

20:11 the last two layers of the stack, which are skills and validation. With this, Mike, I want to also really hammer in data governance is never going to be more essential because I think that really ties in everything here. So, that is the idea where we’re at currently, sources of truth, making sure that what is the right fact table, we have the right semantic layers. One of the things I love that you and I talked about was do you build for a model or do you build for humans? Because having a gigantic monolith or I believe the word you like

20:41 monolith or I believe the word you like to use monolith model that has all the facts and all the dimensions is not going to be very helpful for agents. Having very targeted semantic models is going to work better. However, they need to be curated by humans. Go ahead. Yeah, I Yeah, I you’re you’re you’re hitting my point very closely to what I’m trying to make make the point around, which is if humans can’t understand the model,

21:03 humans can’t understand the model, there’s no way on Earth you’re going to get an agent to understand it even more, right? There’s context, there’s synonyms, there’s descriptions of things. things. so one of the items I’ll point out here directly, if you want to go see something really cool, if you want to go play around with some a neat really a really interesting tool, in Power Designer, the the app that we’ve built as a workload, we have released a brand new item called lineage. You giving me preview? I gave Tommy a little bit of a preview. It’s live, it’s out there, it’s in the internet. We’re we’re currently adding features to it and building on top of it.

21:36 Our view of what a lineage looks like gives you the view of one and many semantic models as a single pane of glass. You can have many semantic models. You can do comparisons between measures. measures. it is so powerful, fun to work with, and if you think about,, there’s this concept of like the enterprise. The enterprise has a full view of everything that it’s doing. and what by the the enterprise view of things. It’s it’s like the, you

22:06 view of things. It’s it’s like the, you view of things. It’s it’s like the,, Tommy, you’ve got an organization, know, Tommy, you’ve got an organization,, even now, when I talk to clients, we talk about let’s talk about domain-based modeling. What specific area are you talking about? I don’t want to give the sales team too many tables inside a semantic model. They get overwhelmed, they’re not sure what’s going on. we have issues with, you going on. we have issues with,, handling know, handling the getting that data together for people so they can actually understand what’s happening. So So that mentality

22:36 that mentality needs to be accounted for when you’re building these solutions. I can’t overwhelm them. And so the the biggest model is all the semantic layers, all the tables, all the lakes, all the relationships, all the KPIs and measures. All that’s the business logic you want to have bundled in one place. There’s not a really good place to put all that all that in Fabric today currently. What we get now is a whole bunch of domain-based models that are separate. There may be repeating things, there may not be, but we don’t have a single page or area to go look at all that stuff. And update it or say,

23:08 that stuff. And update it or say, “Tommy, you you have three models, which measures are most similar? Should they be together in the same model? Is there a measure in model A that should be moved to model B? How does this work?” So, there’s a lot of really interesting capabilities we can do when we start looking at the enterprise model. I’m going to call it that. I’m going to call it the enterprise semantics. Because we have semantic models for Power BI reports and data sets. And then we have this bigger version of the whole world. And as I was talking to you, Tommy,

23:38 And as I was talking to you, Tommy, about this one and again, I’d like your reaction to this one as well, Tommy. I kept communicating this terms of like, “Well, it’s just like a perspective.” Yeah, all right. It’s just It’s like we have this really big model and then I want to give a perspective to a user and right now there’s no ability in Power BI Desktop or in Paginated Reports or in any of the building experiences to see the table tree of all of the columns and measures and select a perspective from that. And, you select a perspective from that. And,, just narrow down the list. Give me know, just narrow down the list. Give me a smaller list. If I see a measure in any of the any of the build experiences

24:09 any of the any of the build experiences I see for Power BI, there is no ability for you to say, “What is this measure and what is the lineage of this measure? Where does it rely upon other measures?” That’s very difficult. Our lineage tool helps you with that. It does It goes through each measure and finds dependencies and shows you the tree of them and goes through measure by measure and like walks you down into subsequent measures that are inside the model. So, I I Yeah, I have two comments here and I’m going to first touch on the former, to talk about the lineage view. What you showed me got scary real quick because

24:39 because when you think about the If you listen to our first episode, it’s about helping guide the agent to what’s important and what the source of truth is. Well, you showed me immediately in a single model one measure had nine like seven different measures that had a 90% similarity grade. Which was And that’s something you could do. But, so how is an agent supposed to decide Exactly. Exactly. right there what to actually use if you have all this ambiguousness? And this is again, one of the things that was touched on in the article that Claude

25:09 touched on in the article that Claude recognized or Entropic recognized, excuse me. This is exactly right. And then I’m going to Yeah, go ahead. To your point, Tommy, I had shown you a model that had,, something that was time-bound at the end. The only difference between the measures was like what the time range was. Hey, this is sales 0 to 30 days out. This is sales 0 to 60 days out. This is sales 30 to 90 days out. Like all the measure literally is the same calculation with a very slight change around one line of data that was adjusting maybe a time

25:39 that was adjusting maybe a time range or something like that. And we see this a lot in models. Like we see all the time a lot of time-based calculations that are measures that are applying business filters to say this measure will only resolve data for this period of time. So, if you use that measure in a longer period of time, it doesn’t show anything. It would just be blank cuz like it’s over-filtering the data. you have 17 measures that say member count, but again, this is one of the things Entropic also recognized that there may be different different

26:10 there may be different different definitions in each business and each department, which is okay. But, if you just tell Copilot right now, “How many members do I have?” Which one is it supposed to choose? How does it know? We’ll get into that little with skills, but the other side is perspectives. And I’m going to take what you said and I am going to take that into the deep end, my friend. Okay. Okay. Perspectives right now, if it had the ability to run on models and actually run on self-service analytics, is the most untapped feature

26:40 analytics, is the most untapped feature in Microsoft Fabric. Say that again for me one time. What’s the most untapped feature? It’s the Perspectives. If I could use perspectives to do that Copilot could run on, and I could tell an agent to run on certain perspectives. The fact that we don’t have that right now makes perspectives the most untapped, underutilized feature in Microsoft Fabric. The reason why I’m going to say this and this is let me try to prove my point here. here. I think I agree with you, but what I’m I’m I’m not maybe I’m where

27:12 but what I’m I’m I’m not maybe I’m where I’m having a little bit of hesitation, Tommy, is around the in Fabric part. I’m saying all I’m saying in the Fabric like if you thought of all the features in products in Microsoft Fabric So So is the most untapped. Yeah, I guess I would say there’s a lot for me there’s a lot of power inside pairing down a big view of tables down to something that’s much smaller. Right. Right. Right? And there’s been there’s been in the desktop, right? So who I don’t think we can let me say this way.

27:43 we can let me say this way. There is no standard UI inside Power BI Desktop that helps you build a perspective. The only perspective tool that I’m familiar with is the one that’s comes from Tabular Editor. Yep. Yep. That’s the only tool that I’m aware of that helps you build out a perspective. To create one or manage one, yep. There’s also not many tools that let you look at a semantic model visually and edit parts of that model in a visual editing interface. You can do it in Desktop but Tabular Editor does

28:14 it in Desktop but Tabular Editor does not do it well. And I’m not aware of any other tool that has like a pretty good-looking view of like lineage of met like even in Tabular Editor it’s just like table relationships. It’s not measures, you can’t see the details of the things that are there. So I feel like there’s a lot of things that some tools do well, but there’s really not a good way of looking at this. And I think this is what our lineage view actually solves. Our lineage view I think solves a lot of these problems like looking at a holistic model, seeing it in a pretty way, being able to move items around and actually

28:45 items around and actually edit those items. We’re we’re working on that part next is letting users to with permissions on the workspace to edit things in there. So, that that would also be another useful area. Right. Well, and the fact too, the only way I can consume a perspective is on the personalized visual or in the explore feature right now. Those are the only two places that a perspective can be used. Which is, in terms of from from I don’t think you can even do it in explore. explore. I thought I think you can actually set up I’m up I’m Ooh, I’m I’m This is a steak bet. You might You might I’ll get it.

29:15 I’ll get it. You’re probably talking about a different feature than I am. If you’re using if you’re if you’re making custom visuals in a report, I believe perspectives are used to help you which report. report. not custom visuals. that’s personalized visuals. But, I I thought that you use some perspectives. But, in in the exploration view of an item in Fabric, you don’t get the ability to select a perspective there. Do you want to bet a steak on it? Because I’m about to look it up., you can look it up, but I don’t I think you’re going to owe me again.

29:46 I think you’re going to owe me again. You can use Okay, so you can use an explore Okay. Explore in Power BI does not let you pick a perspective, but you can set but you can set a perspective. So, the the the explorer, yeah. I’m saying that’s not what I want. Like I I want You can’t pick like the drop like there’s not a drop down menu that I would say this is the like the big model is that like that’s what we need though. But, Right. Right. Even further than that, Tommy. even even the perspective if I create a perspective, the only place it can be used is an explorer if I set the

30:18 used is an explorer if I set the perspective that way. What So, Yeah, but like we got to Where do you build them? Like that’s my my bigger complaint is like there’s no UI for them. It’s very difficult to create them. You have to go get third-party tooling to go use it to create perspectives on top of stuff anyway. So, like no one’s going to do that. And to your point,, is is the perspectives the most underused utilized feature?, may maybe. I Debatable. I think it’s pretty high on my list. I would say that. And I would say I think it’s primarily

30:48 And I would say I think it’s primarily due to the fact that we don’t have good tooling in front of us that says these things, tables, measures, or whatever become part of the perspective. So, yeah. So, yeah. So, let’s go on I think to really the other two foundations here because I think that’s from our previous one. I wanted to start with This is at the very end of sources of truth and goes into skills. skills. The common failure pattern, and this is from the blog,, that from one data foundation layer is poor or stale documentation. Claude

31:20 is poor or stale documentation. Claude is exceptionally useful for closing the gap, drafting column descriptions, proposing metric docs for query patterns, flagging undocumented models, but the curation and ownership are

31:33 but the curation and ownership are managed by humans. And Mike, this to me, and they’ve mentioned this multiple times in the article, this to me is going to highlight how never important or how they’re never been more important data governance and skill management are going to be in the future of business intelligence teams and data governance programs around their data. And I want to start with the human side of this before we get into skills. skills. Okay.

32:03 Okay. Mike, we’ve talked about the importance of data governance, but I’m realizing more and more, and this article was really the final stamp for me, I am not rolling out any agentic solutions around fabric data unless a company has data governance in place. Because it’s going to fail, and I know that sounds like a hot take, but try to argue with me otherwise. You cannot roll out an agentic solution using your business intelligence or analytical data

32:33 business intelligence or analytical data without if people don’t know what what things are, what the right definitions are, and data’s always moving. I’m not going to argue on this one. I I’m not going to argue on this one., this is mean, this is There’s no argument. This is exactly the right This is exactly the right point. The The point here is like, if you can’t understand how that measure relates to other measures, or tables, or if I select this measure, which dimensions can be used with it. This is the problem. And And Tommy, I’m seeing this issue across multiple clients of mine. I’m seeing this many people communicating the same thing. This is why I built Lineage View. That’s why you got to go check it out, because

33:03 why you got to go check it out, because this lets you see these things in a holistic view, and you can go look at the measure, see the definition of it. Does it match any other measures? And there’s common questions I want to ask of this measure. That from the designer’s standpoint. I think there’s a a really strong use case for saying this lineage view should be used and and be able to be shared with any number of users that are building any reports, or paginating things, because this actually gives you the proper context. context. Okay. That’s part data governance. By the way, I’m going to start doing a

33:33 the way, I’m going to start doing a drinking game the more you mention Lineage View here. But that’s part data governance, but, I think it’s pretty revolutionary, honestly. cool. I have I have mentioned this You saw it, and you were like, “Whoa, this has got some legs on it.” I Yeah, no, I completely agree. I told you I’m going to use it. That being said, you keep mentioning it, I have to see how many shots there are going to be lined up for me thing. So, but it’s fine. It’s fine. changer. I’ll say it this way. It’s for sure a It’s for sure It’s for sure a game changer.

34:03 It’s for sure a game changer. Okay. what? I I have no argument there. So, put it on the T-shirt, the game changer T-shirt. but that’s If you are just using My argument there is if you are just using Lineage View, and your company does not have a data governance program, you’re still not going to get to the point where you need to go. That is an incredible helpful part for data governance, but you need to actually have accountability and the ownership of data if the lineage is actually going to be as helpful as it

34:33 actually going to be as helpful as it should be. And I think you have to start there. Like, the reason I say this is the I’m not going to help a company with agentic solutions if they have no data governance. Not only because they’re not setting themselves up for success, but as a consultant or as someone in the company, I’m setting myself up for failure, too. It’s not going to work if you don’t have accountability and ownership of the data from a human point of view. I don’t care if you have the best fabric solution. I don’t care if you have bought every data lineage thing

35:04 you have bought every data lineage thing from powerbi. tips and it works great. If people don’t trust it and there’s no process and change management and accountability, what are we doing here? I I think what you’re speaking to, though, is I think you’re speaking to I I think you’re speaking to like the integration between the process and the tool, right? And Yeah, 100%., , I think a lot of the times the you have to consider the tools in part

35:34 you have to consider the tools in part of the process educ- building thing cuz you you could spend a lot of time of Tommy write everything down, where do you put the information, where does that go? Like, there’s there’s part of that. This is just like the supplemental that to that is like lineage view. Like, that’s that’s something that’s supplemental like the process. And I think I agree with you, Tommy. Tommy. That’s a good point. Like,, this is something that’s this is not I’m not saying that is the solution for data governance, but it’s a better way of highlighting what you’ve built, how you highlight things, what users inside your ecosystems. Blueprint schooling for data governance.

36:05 Blueprint schooling for data governance. Correct. And I think there’s a lot of effort effort effort or and or effort or wasted effort around people just not having the conversations, not writing it down, not providing ownership to this. And I I think this really falls back into the idea of let’s go back into the governance and let’s talk about skills and let’s maybe let’s wind it back here to bit to the to the article here a bit but I think if we talk about this this is what the article

36:35 talk about this this is what the article is trying to address a little bit is you is trying to address a little bit is without the output from your know without the output from your process. process. Right. Right. Without the output from your governance model model a lot of what you’re trying to do with agents aren’t going to be as effective because again going back to our I think our first comment of this episode which was if humans are confused about using a model model Right. Right. agents will also be confused. So how do we solve that problem? Well the problem is solved with process it’s solved with the Power BI or the the fabric adoption road map right? Reviewing that content

37:07 road map right? Reviewing that content making sure that we have understand how mature we are in different areas. Right. Right. I can’t tell you how valuable this is. the the organizations that I see winning and doing well and building good stuff and the individuals that we work with with they get promotions and get better and more responsibility when we follow the adoption road map because it it actually is it’s a is it’s a 1000%. 1000%. There’s lots of really big organizations and Microsoft knows this they’re working with the biggest largest customers and they have more time more

37:38 and they have more time more resources more money and they’re showing you the best practices around governance and process building. So evaluate where you are as a company figure out what you you are as a company figure out what how do you measure up next to your know how do you measure up next to your competitors that’s a great bar to put your company against and then see what areas you’re weak in. And I think to your point Tommy it’s it’s almost this idea of like can your organization actually be self-aware and learn what’s going on. This is what people do all the time. Some peo- some people are self-aware of what’s happening and can be taught

38:09 of what’s happening and can be taught hey there’s new ways to think about things and you can educate them and they move forward and they’re like oh great I’ll incorporate that in my life I’ll change my stuff and I’ll do something different. But in the same way you can actually have organizations and people that are stubborn. Well this is the way we’ve always done it. We’re not willing to learn new things. We don’t want to incorporate new tools. I like Crystal Reports. Yeah. I like Crystal Re- I like Business Objects. We’re going to stay here. Tableau is perfect for us. That’s all we ever need. We want to spend all the money on reporting. So, to your point, Tommy, there, like it it if if your

38:39 Tommy, there, like it it if if your organization has a culture of resistance, I think this is something that leadership and the team working on data things has to really have an open conversation around. Can we actually have these conversations? Are we able to actually adjust our process? Do we even know benchmarking, how does our organization align to other companies that have good process? Yeah. Yeah., what does that What does that measurement look like? And I think that’s that’s the ticket here for a lot of what we’re talking about is if you are able to identify

39:11 if you are able to identify areas of weakness and work on them, you will eventually run into better processes, more automation, and that’s also where I think these agent things start picking things up. And the skills here is going to be,, again, going back to to some of these best practices skills here, right? So, some of the best practices that this article notes around skills is creating pairwise skills, right? A knowledge skill that acts on a thin top-level router

39:41 router that allows domain details to load on demand. Right? It’s a skill about your like like telling the agent what your business does, does, how you operate, who your customer is. That’s what I did in my statement of work side of my business. Right. Right. Mhm. Mhm. I gave I gave an agent a skill and said, “Hey, learn these things. Here’s all the stuff about my company, my business.” Right. Right. And now it has context to that when I ask it other questions. Exactly. And just to your point, yeah. Go Here’s a new customer. Here’s their

40:11 Go Here’s a new customer. Here’s their website. Go research them. Come back and evaluate, where does my tooling and where does my skill set of my team fit in lieu of what this company does? How would I have a good conversation with them? So, it’s it’s doing research work for me. for me. 100% and I I think right off the bat, Mike, let’s start with I love so much their first statement statement on skills here when it comes to it in the workflow for analytics. If the sources of truth, we’ve talked about where we come from data governance, is the declarative knowledge, what a metric

40:41 the declarative knowledge, what a metric actually means, then an agent skill is the procedural knowledge, what sources to consult in what order, how to navigate ambiguous data, and what a finished analysis looks like. like. Yeah. Yeah. And you we realize here right now this is the crux of I think two things. This may be a hot take, but so be it. While I think a lot of companies could get by without data governance and this tooling within Power BI, I don’t think

41:11 tooling within Power BI, I don’t think you can get by you trying to have an agentic solution without data governance and the skilling here. Because you cannot just run Copilot on your semantic model and expect every single time to get the accuracy and the desired result. We’ve talked about this in spades here. What they are proposing here is their semantic models, and I’m assuming here, they didn’t explicitly say it, is each of their models, or at least their domains, have its own skills. So, per skill knowledge or

41:44 skills. So, per skill knowledge or model per skill or skilled knowledge. So, it’s like a one-to-one, maybe there’s multiple skills per domain or per per semantic model. And I think this is such a mission-critical part of what we’re talking about today. You and I have realized that skills are transformative into what we do. It’s hard for me to do work without the

42:04 hard for me to do work without the skills that I use. Yeah. Yeah. Just the model that I use. And I think when we’re talking about domains and we’re talking about data, data, if you’re just trying to run co-pilot or an agent off of your general fabric data, you’re going to get a general not correct answer. Yeah, I agree with that. This is Yeah, so I think this is really critical. And what I love here, too, is Well, let me stop there. I’ll pause there. So, what’s your take there in terms of mission critical when it comes to skilling? Do you think we have we’re at a point now where I need a skill per

42:35 at a point now where I need a skill per model or multiple skills per model? In the current form of tooling today, Mhm. Mhm. you probably need to have some skills per model scenario. That’s what a data agent is meant to supply, right? So, there’s this there’s this idea of like I have some instructions that would go along with an agent. Now, I again, I don’t really like how data

43:05 I again, I don’t really like how data agents is deploying this because actually I want a skill that’s like an on-demand thing, like something that’s very small, right? If I’m asking questions around,, this is the skill for sales numbers. Right. Right. Right, which maybe addresses Gosh, Tommy, there’s so much to unpack here at this one with this one comment, right? right? You’re welcome. The enterprise model, again, we’re going to talk about many domains, right? If we’re talking about the entire enterprise, there’s knowledge around each domain. There’s specific skills. There’s specific ways you want to calculate

43:35 specific ways you want to calculate things. There’s additional measures that have distinct filter context in that domain. domain. The enterprise model needs instructions per domain. Now, if you’re building models in a domain-based way right now, then yes, you need skills or instructions or that human knowledge of this measure does this. And one of the things I think that’s interesting here, too, too, in the article they call out treat skill maintenance as a first-class citizen. Oh, yeah. I’ll We need to document that. on the skills. They’re not going to stay put. They’re going to get better over

44:05 put. They’re going to get better over time. You need to You’re going to learn things over time that makes the skill more effective. So, but also with this, I think of this as if you look at your business and stand back, there’s a mix of domain-based information, there’s a mix of enterprise-based information, there’s this holistic view of all relationships, all measures of your enterprise, but then down to the very niche areas, HR, finance, ops teams, they have their own way of doing certain data about their models and what they’re working on. So, you need a system that’s

44:35 working on. So, you need a system that’s going to incorporate all this. And again, going back to the agent piece, right? right? Agents need to be able to discover all this. Agents need to be able to look at all the items and find things across this area. And so, And so, me personally, when I look at the system or what’s the solution here is, this makes a lot more sense now when you step back and say, “Okay, what we’re doing is we’re giving the agent agent we’re offloading reasoning steps that we would have done into the skill.

45:05 into the skill. And those those reasoning elements can be learned, it’s a pattern that’s learned by the agent through the harness. harness. And therefore, the agent becomes it acts more like your analyst would in making the right decision at the right time. And you’ll also note here, Tommy, there’s another comment in there that says, “Create proper reference documents.” Mhm. Mhm. Right? And right now, again, where do you put these reference documents? Do you make an entire markdown file for every single model? No, I don’t think so. But I

45:37 model? No, I don’t think so. But I also I think you go into the model and you add like some of the data should be inside the model. One thing that I don’t do right now a lot of, which I probably should do more of, is of, is, they talk about adding details like the grain of the table, the scope of the table. Are there exclusions on this table or measure, right? That information is incredibly relevant to agents understand what’s going on. Even So, even on an agent, Tommy, like if I had a measure in a model, I should give users the same information. Like, there should be a

46:08 information. Like, there should be a table definition, and it should say, “This table is scoped to this.” Here’s our dim customers table. This is every customer known to man that we have in this table. Hey, here’s another dim customer table, but this one’s scoped to only the customers that exist in this particular fact table. They’re both They’re both still dim customer, but there’s now,, the grain and scope of that table and where it’s supposed to be used. We can have the same definition of customers as tables, and they’re used in different areas, or

46:38 and they’re used in different areas, or used with different data models. So, I think think holistically, you need all this stuff. Like, it you have you have to build this metadata. The trick right now is where do you put it? Where does it get stored? Does it belong in the model, or does it belong in some other tool that’s above the model? Yeah. Let Let’s wait for the end to do that, because that’s I would love to dive in there, because I think this was validation. Mike, I think I have an analogy. I have a cooking analogy, and I want to see I think this will help. pasta or or Italian food.

47:09 pasta or or Italian food. You better believe it. Your model’s like a cannoli. No. So, your agent Tommy likes it. Another Another way to think about this, to me, is if you had an an agent for cooking, cooking, Yes. Yes. your skills are your in a sense your recipes. Like, I had a sauce recipe. The agent Your skill has to tell it, “Hey, when it’s too sweet, add salt. You should always start with these key base ingredients. When this occurs, then, you

47:39 ingredients. When this occurs, then, you ingredients. When this occurs, then,, here’s how you adjust. Never add know, here’s how you adjust. Never add celery. That’s only for northern Italian Italians.”, for sauces kind Italians.”, for sauces thing., if extra sweet of thing., if extra sweet with no sugar, use carrots. So, there’s And that’s what I love about the skill here, where,, what does a What What does every row represent? What’s every filter that needs to happen? You every filter that needs to happen?, what are the what happens when know, what are the what happens when there are default choices? What are the worked patterns? They even say that they have a reference stock in their skill folders of SQL queries, which you’re

48:09 folders of SQL queries, which you’re going to add DAX queries, if, if it’s semantic model, of all the different common patterns that they use, which to me, again, really emphasizes this is a this is a skill per model type of thing. Because I have probably certain DAX queries or evaluations that I would want to use for a model. Most people ask for member counts. This is the default way we do that. If you are looking at members,, for marketing, well, you have to use this date table. And the first-class citizen here, I I I know you mentioned this, but I I do not want to

48:41 mentioned this, but I I do not want to go past this without giving it the proper due. This, Mike, is where I think our job is really changed, and I think where BI really changes, and where companies, if they are not recognizing to give time for people to build and manage skills, things will fail. So, one of the Why am I saying that? It’s not just off the top of my head. The skill docs describe it This is from the article. Skill docs describe data models that changes daily, like all of our data. So, without active maintenance, they’re

49:11 without active maintenance, they’re wrong within weeks. Yes. Anthropic watched their offline accuracy drift from around 95% at launch to at around 65% over a month before they actually treated it as it engineering problem. So, even if you create the skills, if this is not part of daily work, right. And I think this is going to be this is going to be such a conflict in organizations where they’re going to go, “Why is the agent wrong now?” And they’re not going to give people the time or the resources to treat this as

49:42 time or the resources to treat this as just as important as building out Power BI models. And it’s going to happen because they’re going to go, “I don’t understand what’s wrong.” Well, I need time to look at the skill. You need time to look at the skill., and it’s not something that’s know, and it’s not something that’s tangible yet. It’s not a universal concept. So, The data’s like a living organism to some degree, Tommy. And so are skills now. Well, and well, and the skills I think are just adapting to what the model’s doing. So,, this is what they said here is active maintenance, right? With the with the skills. skills. Active maintenance. I’m also thinking about Tommy, your comment earlier around like you’re now

50:13 comment earlier around like you’re now looking at things and saying, “Okay, I’m building I’m not building the tool anymore. I’m building processes around the tools.” segue. segue. Great loop. It’s the same It’s the same concept. The same concept here is you build the skills, you build the model, right? Mhm. Mhm. Think of it this way, the the data coming into the model is regularly shifting and adjusting. It’s There’s new data, there’s more information coming in. certain categories are going up, certain dimensions are going down based on these measure like all that stuff. The model itself is being a bit dynamic. Right. Right. You as a developer, Tommy, you’re adding things to the model. You’re refining

50:44 things to the model. You’re refining things, maybe a measure was wrong, you’re adjusting the model. So, the the model itself is actually a living definition of the semantics of what you’re doing for your business. Okay, so we have two things that are moving, the data portion and the model itself. Well, obviously, if you don’t adjust the skills and have them update themselves based on the changes you’re making at the data level or the model level, it’s not helping you. Right. Right. So, I would argue that in the same way here, here, you have to do the same thing. You have You have to like stand back and say,

51:14 You have to like stand back and say, “Okay, great. All three of these things need to So, where I go to this is what’s the what’s the process? What’s the system you put in place to review these? How How does this mechanically happen day-to-day? And I think that’s where I put my head my hat on next and say, “Okay, well, well, awesome. This exists. How do we do it? Where does it go? And I think for me, I look to process and tooling tooling to help aid with what this would look like. like. Before we do that, I need to talk directly to the team at Microsoft real

51:44 directly to the team at Microsoft real quick. So, I’m going to I don’t know if you have a sound, but I’m going to just do a little effect here. So, let me just actually zoom in super super quick. Microsoft, zoom in. Look at me. Okay? Okay? Okay? Look into my eyes. Look at my eyes. I’m going to actually zoom in a little Can I do more? Perfect. Yeah, go zoom in. Yeah. You need to have skills for models. Prep for data is not enough. Please. Skills for models is essential and add perspectives with it. Those are the two things we want. You’ll change the game. You’ll make a lot of money. I

52:15 the game. You’ll make a lot of money. I don’t care. That’s what you have to do. Million-dollar idea. Cool. We’ll go back. I feel like this is a good phrase to to pull from a great movie. It would be Do you hear the words that are coming out of my mouth? That’s that’s a that’s a old-school reference for us ’90s kids. Yeah. So, we will not quote the rest of it. So,

52:37 so, that was really good there, too. I I agree with that one, Tommy. one thing I want to also bring to people’s attention here as well. I I think there’s also a hidden gem that’s being published in the community, and it’s actually coming from Snowflake at this point. Mhm. Mhm. It’s this thing called It’s this thing called the open semantic interchange. It’s a specification. It defines tables, relationships, calculations, measures, a lot of business knowledge. And it’s it’s held at a generic level. So, if you think

53:07 at a generic level. So, if you think about all the different tools that are trying to build semantics layers on things, there’s Databricks, there’s Snowflake, Power BI has and Fabric has their own as well. They open-source their semantic model, but it’s done in a Timedle format. So, it’s a Timedle format for Power BI. So, all of these tools have somewhat of an open interchange of information and data. I think this OSI open semantic interchange, which was released from Snowflake, is going to be really useful for companies. And so much so, I’ve added it to our lineage view. So, if you want to look at the If you want to go

53:37 want to look at the If you want to go take a semantic model and turn it into an OSI document and shift that around and lift it and move it around different business places for business logic. We’re building integrations with that right now. So, like I think it’s I’m so bought into it, I’m building tooling around it now that I think will be even more useful in the long term. That’s enough. Yeah. Go ahead, Tommy. Take another shot. Microsoft. Also, OSI, great idea. Become a partner. Okay. Okay. Tommy. Tommy. I want to start doing that now. I’m going to have to work with you on

54:07 I’m going to have to work with you on like the zoom in thing. Cuz now Tommy’s going to be like grabbing the camera and zooming it more Taking it. Yeah. I need that feature. I need that feature. I need that feature. Please. Awesome. No, I I think that’s a I think that’s a great feature with to truly though, what OSI is doing with what Snowflake has done and the fact because the biggest thing about that, Mike, it’s not just a generic language, it’s cross model, cross systems. And again, if you’re coming from mirroring data, you want that to work as well. And

54:38 data, you want that to work as well. And I think that’s also a great part to talk about, where do you put all this stuff? Because the last foundation that Anthropic has is validation. How do you actually validate the model using analytics? But I think the problem for us now is we’re at a standstill because Michael, Michael, let’s go to your question. Where do you actually put this information? Mhm. Mhm. I think it’s a mix. I think it’s a mix of So, I don’t This is where I think Tommy things get interesting for me is does this

55:09 interesting for me is does this information live in the OSI? Does it live in the semantic model? I think it actually has to live where people use it, honestly. I think I think you want to push I think you want to push this knowledge downstream as much as you can into the model. Indirectly into the model because the model is what people use day-to-day. So, we almost need, to your point, Tommy, Tommy, I think what you’re asking for, Tommy, is the semantic model, wherever you build it, needs like an additional section section Yeah. the code like you would do like a folder

55:40 like you would do like a folder Like it’s still Yeah. markdown files of get of instructions. This model contain So, I think binded to what the tabular object is. Yes. So, I think I think you want to push the information into the models where they live. Now, you do you do need a two-way interchange to this. Like I want to push all the data into the models people actually use. So, that way if I open the model, skills exist, instructions exist. It’s part of the model. That makes sense to put it there. But you can’t hardcode need to be able to zoom out

56:11 need to be able to zoom out and say, “Okay, let’s look at all my models and see instructions across all models.” And you need to be able to interchange between the enterprise data set semantics into these specific models that are that are there. And it’s interesting you say that, too, because one of the things that Anatropics talks about is they do not hardcode locations of the skills in certain place. They’re using GitHub repos repos for the skills and for those instructions. It’s interesting, Mike, because I think there’s two aspects here. One of the things that

56:43 aspects here. One of the things that they do for validation that I I thought was very interesting was they actually look at looking at business information, not just all the data like, “Hey, if someone says member account for this year, the number’s 10.” Well, they also give it like project roadmaps. They also give it slide decks. So, they give it a lot of soft data, too. So, if a user were to ask a question that was more ambiguous, the our model would know where to look. So, maybe it doesn’t necessarily live in the

57:13 maybe it doesn’t necessarily live in the TML or the the or the the TOM. The TMDL. The TMDL. TMDL TMDL TMDL TMDL the the What Why not? The TMDL is already like a series of markdown files and YAML things. things. Yeah, but at the same time sense. sense. But the thing is you need that to be a I guess Yeah, if you have a repo, then that makes sense because they’re saying now that 90% of their commits or their pull requests are skill changes. So, I need everyone needs to be able to access to this who have an edits to model, which I guess would do well

57:43 to model, which I guess would do well with the get repo. But, you need to give it access to this you have it has it needs to be able to consume the model needs in that needs to be able to consume soft data as well. It needs to be able to understand, you needs to be able to understand,, like is that project roadmaps, know, like is that project roadmaps, maybe some word documents too. It can’t just all be word docs. So, I don’t know if the Tim Dills the best place. I don’t know if we have a perfect location for this. this. Bingo, and that’s exactly my point, which is the OSI handles that specifically and graciously for you. So,

58:13 specifically and graciously for you. So, like like So, you’re winning this time. So, someone made a a really funny comment here, John Kurski, which which great John Kurski, I love the comment. Well, you can have an annotation that’s a mile long. Yes, you could. Right. Yeah, great. 100% you could. So, Worse punishment. you can you can forceably push these things into these areas, maybe not the right way, but you can push them in there temporarily using,, what’s in the model today. But, is there a dedicated skills area in the model? There is not. Should there be? I think there should be. And honestly, I want to store as much metadata and semantics

58:44 store as much metadata and semantics data in the model itself, but I also need an interchange between the domain based model and the enterprise model. I need to be able to juggle the data between the both of them. All right. This is really good conversation. I also think we should just cover off with the last point here, Tommy. I know we’re over almost at time. Let’s talk about the last part of this article so we can close this off and put it to bed. Okay. Okay. validation. Validation. So, there’s a last section here on this article talking about validation. Best practices. Let’s just maybe hit the best practices. I

59:14 I Go ahead. Go ahead. You go. going to say the pair So, one of the best practices that they say is around validation is being able to first have two different types of evaluations. Offline evaluations and online evaluations. They call their offline evaluations that are things that are actually off of static data. We have a static snapshot of this month. We want the model to run, so we know every single time we’re testing at this point, it’s off of data that’s not hasn’t changed. How many members this month?

59:44 changed. How many members this month? What it were the member Where were the members from?, how does this go with our quota? So, the static data is a huge part of here, so there’s a ground truth. truth. and I also one of the biggest things I love, part of data governance, gate launches. A domain owner cannot announce to an agent to the stakeholders until that slice of evaluation clears some threshold of some accuracy. So, say we updated the agent, that has to go through a rigorous process of certified

60:14 through a rigorous process of certified data model or semantic model. So, those are the two that I want to highlight. What did you have? Yeah, I think I would agree with those two, Tommy. Those are the the evaluation standpoint is very unknown in agents. It’s very non-deterministic. Right. Right. are are being built there. It’s difficult to understand what is adding good value and what’s not. And so, I really do think that the having some way to evaluate when you put skills in, when you put things in, if there are specific questions that are being asked, does that actually produce the appropriate results from what’s inside the model or

60:44 results from what’s inside the model or what the agent is supposed to be doing or building on top of it. Again, I don’t want agents to answer questions. Right. Right. An agent should not be giving you what was last month’s sales compared to this month’s sales. That’s not what an agent does. does. Agents should be used to help you build repeatable deterministic visuals, pages, report elements. That’s where agents are really good. That was That is where you will save most of your time and spend a good amount of money. So, the idea is build a deterministic report tool, something

61:15 deterministic report tool, something with the agent using the business context that’s inside the semantic model. model. Right. Right. That’s what you need to have. Once that’s been built, you can then just reuse it over and over and over again. Yeah. Yeah. Awesome, dude. Well, hey, this is great. I’m so happy we actually spent two episodes on this. And if you have not read this article yet, I think this is going to be one of those seminal articles that came out that I think hopefully a lot of organizations use and use and individuals also take as

61:45 individuals also take as changing our workflow and changing what we need to focus on. I would agree with that one, Tommy, wholeheartedly. Awesome. With that being said, thank you very much for listening to this episode. Hopefully you found this article useful. there is awesome skills and information from this article from Anthropic. It’s in the description. It’s also in the chat window as well. Make sure you check it out. and with that being said, Tommy, where else can they find the podcast? You can find us on Apple, Spotify, wherever you get your podcast. Make sure to subscribe and leave a rating. It helps us out a ton. Do you have a question, idea, or topic that you want

62:15 question, idea, or topic that you want us to talk about? Have you taken any of the things from the article? Or do you want us to dive into more? Head over to powerbi. tips/podcast, leave your name and a great question. And finally, join us live every Tuesday and Thursday, a. m. Central on all powerbi. tips social media channels. Awesome. Thank you all so much. We appreciate you in listening to our episode. Please share and like it and give us a thumbs up down below. It helps push the episode out there to more people. Thank you all and we’ll see you next time.

62:46 Explicit Measures pump it up, PA. Tommy and Mike lighting up the sky. Dance to the data laughs in the mix. Every single day I get your fix. Explicit Measures, drop the beat now. Focus skills, feel the crowd. Explicit Measures Drop the

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.