Data Agents Got an Upgrade – Ep.560

Fabric data agents just got easier to wire into Microsoft Foundry and Copilot Studio, and Explicit Measures spends this episode on what that is actually for. Tommy Puglia and Mike Carlo treat the update as a change in role: the data agent is the specialist another agent calls when it needs governed answers from a semantic model, a lakehouse, or a warehouse.

News & Announcements

• Using AI Harnesses to Harness Microsoft Fabric — Thursday, September 24, from 3:00 to 5:00 PM Central, the Chicago Fabric and Power BI user group meets in the afternoon. Tommy is building the session from his four-part PromptingBI series on putting context into your tools, turning that context into prompts, and shaping how an agent behaves from the first encounter through the outcome. The room will pick a project and build it live, with using AI in Fabric and Power BI as the point of the afternoon.

• New CI/CD resources for Microsoft Fabric, from concepts to end-to-end automation — Microsoft published a Fabric blog guide that runs from basic concepts through automating development, testing, and production, and it spells out what belongs in version control. Tommy flags it because that conversation still stops at semantic models and Power BI reports, while a Fabric solution can already include more than 70 artifact types — he has created 10 of them in a single workload — and Mike expects agents to multiply what gets published, on the order of three to five times the reports teams already thought were too many. The path in the guide is understand, plan, practice, implement, and improve, split across REST API, Git integration, variable libraries, Fabric CLI, Terraform, the Fabric CI/CD library, and enterprise architecture, and both hosts want a multi-part series on it, with Mathias as a guest, knowing each team still has to choose the options it actually understands.

• Upgrade Power BI dataflows Gen1 to Fabric dataflows Gen2 with the upgrade wizard — The wizard is a self-service move from a Gen1 dataflow to a Gen2 dataflow, with CI/CD and key features in scope, an evaluation of each dataflow before the upgrade, and an Upgrade to Gen2 action on the Gen1 item itself. Microsoft has said Gen1 will eventually stop being creatable, with no date, and the upgrade stays optional. Mike hears a wizard aimed at more spend every time the pipeline runs — Gen2 does more work and has more knobs for performance — and he would not start a new Gen1 where Fabric is available, since a company without Fabric enabled still has Gen1 as its only dataflow, a pitch he sets apart from the old “upgrade to a datamart” line, while Tommy adds that Gen1 stuck because it feels like Power Query and that the wizard is there to slow new Gen1 work across a very large set of migrations.

• Fabric data agents in Microsoft Foundry: easier to connect, easier to trust — Amir’s update is the article they came to take apart, alongside the Copilot Studio work from the last couple of weeks. Data agents now appear in the OneLake catalog inside Foundry, and one Foundry agent can connect to more than one Fabric data agent, with the same multi-agent support landing in Copilot Studio. The rest of the episode is the argument over what job that leaves the data agent.

Main Discussion

Topic: Where a Fabric data agent sits once Foundry and Copilot Studio can call more than one

A data agent was already the package that turns a plain-language question into the right query language: DAX for a semantic model, Spark or T-SQL for a lakehouse, T-SQL for a SQL database or warehouse, KQL for a Kusto source. Tommy and Mike have covered that on earlier episodes. What changed, in their reading of these updates, is the role. The data agent is being placed underneath a Foundry agent or a Copilot Studio agent, as a subagent with one job, and the new catalog, multi-agent, and trace features are what make that placement usable.

• Purpose is the part Copilot does not carry. Asking Copilot in Power BI or Fabric about a report or a semantic model returns an answer to that question, and it stays general. A data agent is bound to the source and to instructions about why it exists: which columns, relationships, and measures matter, and what the reply should do for the person on the other side. Tommy’s picture is a cycling coach on Strava data. The questions are how the wind has been affecting him, when to ride, and whether next week’s load should come down, and the answers are coaching, written from Markdown instructions. Underneath, a semantic model still means generated DAX, and a lakehouse still means SQL.

• The semantic model is the business logic you want the agent to see. Mike’s framing, the one he credits to Marco Russo, is to give the agent a semantic model with column descriptions, relationships, and measures, and to keep that logic in the model. A redundant way of calculating the same thing confuses a person, and it confuses an agent more. A lakehouse has no column descriptions, no relationships, and no measures, so the instructions have to say that this numeric column gets summed and that the sum is total sales. SQL can hold primary and foreign keys, and the agent still has to go retrieve them. Between a person and a query, the semantic model is the fuller translator.

• One job per data agent was already the practice, and it was hard to deploy. Tommy builds a data agent for a single function. A medical-sales agent that reads quota is the example, and he wants to hear from the chat if people are succeeding with broader ones. That design left organizations with many agents and a clumsy place to use them. Copilot Studio could connect them as agents, and the experience was incoherent. Foundry had them too, behind an ID, and only one Fabric data agent could be attached at a time.

• The catalog removes the ID, and one lead agent can hold several specialists. In Foundry, data agents now show up from the OneLake catalog with a name, a location, and the details you need, so you stop pasting an ID. The Foundry agent Tommy creates can connect to multiple Fabric data agents — sales, supply, and support — and answer across those domains. Copilot Studio treats the data agent as a tool on the agent you build. Mike’s favorite part is distribution. A Foundry agent publishes a set of instructions, can be called, and can be added to Teams. He finds more room to adjust it than in Copilot Studio, including which model it uses, a ChatGPT model or an open model, and what it is allowed to spend. The data team’s work moves out to the rest of the company through that agent.

• Fifteen thousand characters is the reason “several” beats “one.” David in the chat flagged the instruction cap at 15,000 characters. A single data agent for every source in the company runs into that wall, and Foundry used to allow only one of them anyway. Mike would split them: a SQL data agent, a lakehouse data agent, a Kusto agent, a real-time agent, each pointed at its own source. You then tell the Foundry or Copilot Studio agent which specialist to call for a semantic-model question and which to call for the lakehouse. He compares it to coding subagents. A narrower context window does the specific job with less clutter, and Foundry is now able to tell which attached agent the question belongs to. Multi-agent support is the update he ranks first, in Fabric, in Foundry, and in Copilot Studio.

• Write the instructions as if it is a subagent, because that is the slot Microsoft left open. Tommy’s picture of what they did, and what they left undone, is specific. There is still no central place to sit and chat with a Fabric data agent, including inside Copilot in Power BI, and there is no @ reference for one the way other chat tools let you call a tool. He is fine with that gap. The pattern he wants is the one other agent tools already use: you talk to the Copilot Studio agent that is available across the Microsoft environment, or to the Foundry agent you can reach from a terminal or from Cursor, and that lead agent calls the data agent when it needs data. He would put it in the instructions in plain words. You are a subagent of whichever Foundry or Copilot agent contacts you, and when you are called you do one task. Sales and supply, or sales and finance, can check each other. Mike does not hear a new capability inside the data agent. He hears a wider door around the same wrapper: the model is still the one Microsoft chooses, you still cannot pick the LLM, and the skills inside it are already there and still closed to editing.

• Traces tell you what people actually ask, if you can stand to read them. The part of the Foundry article Mike got stuck on is monitoring. Once the agent is running, the trace shows the stages: the agent is called, a tool runs, the agent is called again, another tool runs, with the input, the output, and the milliseconds spent. He wants that rolled up to the theme of the question, the tables involved, and the queries that ran, because a raw log he has to read himself is a log he will ignore. Tommy’s reply is to land the logs in Fabric and point a data agent at them, which is also a new stream of usage analytics for the model, and he is explicit about the meter that implies. The open question he keeps is whether each Fabric data agent gets its own log, or whether one trace shows the Foundry agent moving from the sales data agent to the delivery data agent.

The podium

Tommy listed six updates and asked for bronze, silver, and gold, scored on importance and on how usable they are. They picked the same three, which he noted almost never happens. Gold is multi-agent support, because of the instruction limit and because one agent should not be querying every source. Silver is logging and the traces, framed as the admin feature while so much of this is still unfamiliar. Bronze is publishing from Copilot Studio into Teams, or anywhere else people in the company already are, because an agent you cannot reach from Teams is an agent Mike will not use on a normal day. He left three items off the stand. MCP for data agents in Foundry is Tommy’s usual number one, and he kept it off the podium because he treats MCP as the standard way these tools talk to each other. The OneLake catalog is what makes the agents findable. Fabric IQ MCP in Copilot Studio was on the list too.

What it costs, and the other way in

Mike’s recent test was four questions against a semantic model through a data agent, four turns, about five minutes. It cost 400 CU, and on an F2 that 400 CU is about 45 minutes of capacity. His alternative is the XMLA endpoint and the Power BI modeling MCP: give the agent the model definition, let it authenticate, and let it query with DAX, or let it write Python and use Fabric tools, and spend tokens on that conversation. He put a rough pair of numbers on the choice later, about 150 CU on a data-agent request or about 100,000 tokens with the outside agent. He is unsure the data agent can keep pace with models that already call APIs and build their own tools, and unsure Microsoft will keep the data-agent side level with Anthropic, OpenAI, and xAI.

Tommy’s answer is the company that will not open that door. A modeling MCP inside Anthropic is a non-starter where security policy blocks it, and Foundry or Copilot is the sanctioned place. He also does not trust a thousand employees each querying the semantic model through an outside MCP to land on the same number. ChatGPT had already given him a wrong answer twice the day before, and agreed once he told it to think again. Data agents are narrower, and they are aimed at a consistent answer for the whole organization. They closed on that as uncharted. The community does not have a settled practice, the line can move in three weeks when the next model shows up, and the workable posture is to learn what fits your company and stay willing to change it.

Looking Forward

Treat the Fabric data agent as a subagent with one source and one job, keep the descriptions and measures in the semantic model, and let people reach it through a Foundry or Copilot Studio agent, including from Teams. Run a few real questions and look at the CU before you hand that pattern to a whole department. Tommy and Mike already parked a full episode for the organizations blocking outside AI entirely, because that policy and this architecture are now the same decision.

Episode Transcript

0:01 Dance to this day, laugh to the rhythm. Fabric and AI understand your feelings. Explicit Measures. Let ‘s get the rhythm going now. Pumpkins feel the crowd. Explicit Measures. Hello everyone and welcome back to the Explicit Measures podcast with Tommy and Mike. Tommy, good morning. How are you doing? I’m doing great, great, Mike. How are you? I’m fine. Just busy. We’re in

0:32 busy. We’re in such an exciting time right now, Tommy. I just can’t believe believe how much things have changed and how different my my workload has become since January, well, since December of last year. This radically changed the way I work. work. I have a little interesting introduction for you, but what’s your topic today? So, the main topic today, Tommy, is an article from the Microsoft community. It seems you found it. Nice find, Tommy. This is about Amir

1:02 is about Amir talking about,, and it’s funny that they say “Amir.”, Amir, it says: Microsoft employee Amir. I think it’s the same Amir, the main Amir. I think it’s him. , anyway, Amir says, ” Data Fabric Agents in Microsoft Foundry.” “Easier to connect, easier to trust.” So I really like Foundry. I like the experience of working there. I like that there is an API for a lot of your

1:32 for a lot of your agency stuff. So I like it. This is really interesting. Since we have this article, we will analyze it today. OK. This is the main topic. Sorry, Tommy. Continue. What did you want to say? So, I think it’s part of this amazing new time that we live in. We’ve already talked about cycling, and there’s a lot of a lot of data associated with it. I have a bike computer that knows exactly where I am: latitude, longitude. He, of course, knows the speed, GPS. I have a I have a heart rate monitor that

2:02 heart rate monitor that connects to my bike computer and records beats per minute. And all of this data is sent to an app, or,, a software service called Strava, which tracks all of this, and you see a lot of this data, but they have APIs and MCPs. I was wondering, Mike, you wanted to talk about spontaneity, you asked me on Tuesday how I plan to create new things, and I said I usually write them down. Yes. Yes. Here I became interested, I said: hello,

2:32 , I said: hello, I have a Strava app, there is an API, I would like a dashboard for it in Power BI., and I wish he was like my cycling coach, telling me when to ride. Sounds cool. cool. Yes. And, Mike, thanks to Fabric skills and remote MCP. Not only is this data now being sent to lakehouse, but it’s a bunch of different data coming in. Of course., but they’re kept in the lakehouse. I’m like: well, just use your use your Power BI skills, build a semantic model, give us some

3:02 give us some specification of what I was looking for. Of course. And, Mike, I did n’t have to open Power BI Desktop and create a Power BI report using these skills. The creation, design, and planning of the Power BI report essentially built a very similar theme to what Wahoo looks like, and this is one of the companies that deals with all this computer computer hardware, the look and feel of that. To essentially say: okay, hang in there. Here is your relative effort. Oh, that’s all the headwind there was at that time. So your effort was actually

3:32 was actually less, so you could take a bigger ride and,, go up a bigger hill. So Mike, I did this in probably 30 minutes to get all this data into the lakehouse repository and the report design. , how can you not use this use this now, not only for fast development, but it also looks great. great. Yes. And, and, and so these are most of the major software products that I currently use use MGM.

4:04 And if they don’t, if my software products don’t have easy integration with MGM Agent. I am actively trying to remove this tool tool to find another one. Yes. Find another tool. Take something else. Something else that will be a little more, well,, well,, responsive to my agents and able to communicate with my agents. more, Tommy. Like, if a tool doesn’t allow me to easily plug in any AI I want, I’m not interested in using that tool. And AI, I think, is not that different in each

4:34 different in each system, I feel like there are two are two winners that I see in the market right now. It seems that Anthropic is definitely coming out on top, one of the leaders. One of the tools that, in my opinion, is a bit of a latecomer to the party, but has quickly become one of my favorites., Grok or xAI. We xAI. We talk about this a lot. Yes, I am very impressed with what they are doing., their Grokbot thing is

5:04 , their Grokbot thing is pretty impressive. It does a lot of really useful things, but it’s also it’s also built on top of the very reliable Cursor tools. Cursor has a slightly different approach to working with agents, bots, and bots, and environments, and each agent gets its own virtual machine for its tasks. I for its tasks., he’s mean, he’s quite capable. On a lot of websites, Tommy, you go on them and there’s this… this access restriction effect, right? OK. I will tell you about one case that case that simply annoys me.

5:34 simply annoys me. Oh my God. Not that it’s too bad, but but it’s not that big of a problem. Tommy, you and I work at Notion. So, we both have Notion. You added me as a guest on your board in Notion. Well, whatever it is. So I can So I can review it. No problems. problems. Inside this Notion board., I can log in with my with my credentials. On the board. I can click there. But I want my agent to help me prepare previews, content, and review articles that will be coming out tomorrow. For example, I want the agent to give me a quick overview: “Hey,

6:04 quick overview: “Hey, here’s what I’m getting ready for. ” Help me ” Help me think through my main points. What should it look like?, and MCPs are MCPs are not available for guest guest users in Notion. Like, “Oh, that sucks.” Then the agent tells me, “Well, if you just log into your Notion on my virtual machine, I can navigate the site for you and find everything you need.” I say, ” Okay, let’s do this.” So I go to my little

6:34 to my little bot, I log into Notion there, and now it can go to the page, view things, read the page. I think to myself: if you’re a company trying to restrict user access, I just think about it this way: first of all, I’m a user. user. This is cool. And secondly, I look at it and think: if I’m Notion, then there’s no point in blocking MCP servers or restricting guest access to the API, because I

7:04 guest access to the API, because I can can get around it anyway. This is not even a problem. I just log in and work. So I look at this and I see more and more of these agents becoming so capable that they act as real assistants to people. If I had an employee, I would just say: here, come in here, do this, do that, summarize the information, give it to me, done, it just works. And I… yes, I agree with you about Anthropic, because they are the ones who developed these skills, this

7:34 these skills, this open source. They are also the ones who developed the concept and released the publicly available MCP. I remember when they first released it, I thought: what is this? But the fact is that I am now looking for MCP-enabled apps, regardless of whether I will invest in it personally or professionally. I’m not going to spend money on any tool any tool that doesn’t allow me to use MCP. External access, right? Not just AI in a tool. So yeah, one more, Tommy, I want you to try this one. I think you’ll really like it too. You

8:04 really like it too. You… I think I mentioned this before. Have you heard of pen. dev? pen. dev? I’ve heard of it, but have n’t used it. OK. When you’re working on report design, Tommy, there’s this pattern: you have some data, you tell the agent to come up with a semantic model, you describe what you want, how the data is related,, you guide the agent a little bit on that, on that, but often you go into the reporting area and say, I need a report and you’re not sure what it should look like, or you have some images

8:34 have some images that you like and they look good, but you want a design for the report or just a sketch, sketch, even for even for an application, for that matter. Now I give pen. dev to my agents, and pen. dev allows you to connect your subscriptions to Anthropic, Copilot, or whatever. You just come in. It communicates directly with the API and sends data there for you, and you can even specify how many agents on the page are creating graphic design for you. So you can

9:04 . So you can choose the number: 1, 2, 3, 4, 5, whatever you want. It want. It launches the appropriate number of agents on the page, and they divide the work among themselves. They create the page design. They create graphics— all within this system. And I think to myself: this app is miles ahead of what I can do in Figma. This is so cool. And I think: I want… it’s free. I can use it use it now. I want to delete

9:34 now. I want to delete Figma and use only this pen. dev. Yes, Yes, I love Figma. It has been a staple tool for many years, but it is a paid subscription. This other program is free, and I can use use agents in it. Much better. And I look at myself and think, “Oh my God, if people…”… a few strategic agent-based companies could create incredible software that would really shake up the original original players in these areas.

10:04 players in these areas. Yes, it’s really crazy right now. And I think, as you said, with respect to Cursor, Cursor, whether we’re talking about XAI or Space AI, I don’t know… know… they bought Cursor. So Cursor was a separate IDE, and let’s be honest, most of the things that VS Code released were essentially borrowings —not stealing, but they were taking what Cursor did and implementing it: multitasking, the agent window.

10:35 They were ahead of the curve, so as I said, I mentioned Cursor a lot in this podcast. But I believe these things are necessary, and that brings us back to an argument I make more and more often, as do you. Any tool Any tool should work. It must be able to able to communicate and interact with other tools, tools, for example, in our various environments. This is the only way I think it think it will be truly effective for users. It’s not just “I can only use only use Claude” or “I use Cursor.” It should be, “Oh, it should work with Notion.” Can he

11:05 Notion.” Can he use MCP? use MCP? Does it provide this access and this seamless path? So yeah, this is a huge thing, Mike. But yeah, Strava, thank you so much for being AI-ready AI-ready for all my cycling needs. So Tommy, if it wasn’t ready for AI, you would

11:21 ready for AI, you would look for another bike accessory that had everything you needed and try to upgrade to something else. That is, these… these… these are the realities we are in now. I now. I connected Microsoft Clarity to my MCP. So look, this is another example. Clarity. In fact, In fact, I went to my site the other day and thought, “Oh, I have a lot of old links that are going to the wrong pages or pages or something.” I just contacted an agent, because managing a website is a separate, big job. Now

11:52 big job. Now I can set up a task to task to check the data from Clarity every week. Give me a weekly report on how many 404 errors or old links are pointing to my site that no longer work because I just changed the site to a static web page. Just give me a summary. Give me Give me information. Okay, now add redirects for these links links. For . For these links, add a page with an appropriate message and explain the template. I guess I’m just I’m just stamping new pages right now. And even when there are links to old content, I write new blogs or posts to posts to redirect old

12:22 redirect old web traffic there, explain the changes, and direct direct users to the new place. So I look at it and I think it’s really effective and very useful. Yes, I am. Yes, the level of automation in my work now, Tommy, is simply incredible. I incredible., first of all, mean, first of all, every program I use now has some agent built into it. And secondly, if I do one

12:52 , if I do one action in the program, the other day I was working on thumbnails for a podcast, and there are a bunch of portraits, well,, Tommy and I have a lot of ready-made photos. We have done many portraits over the years. And yes, well, and all those funny funny facial expressions. So they were all separate frames in my Figma document. And I went into this Figma document. I, well, usually Tommy and I are gray in miniature, completely discolored. So I

13:22 discolored. So I reduced the saturation and said, “Okay, agent, on this frame for Tommy number 12, I reduced the saturation. Look what I did there. “and fix all the other images in the same frame.” I said, “Great.” He knew what to do. Boom. Done. Did all of them. Perfectly. Now make the same saturation changes for the other frame, which contains all of Mike’s photos. Boom. Done. Done. So if I had to click on each one, it would take a lot of time— going through 50-60 photos, just

13:52 photos, just changing the saturation to the same level. I now do it once, tell the agent what I did. It sees the code and what happened in the program, and then repeats it for every other object on the page. This, this is the approach, this is what I want to do. I don’t want to go and do something twice. twice. Do you understand what? I’ll share this quickly while we’re sharing curiosities and just say how say how ridiculous we’ve gotten with limits or these are exciting times.

14:22 exciting times. So I’ll share this quickly and we’ll get started. During bike rides, I listen to these things called XLT. And,, basically, there used to be these uploads: uploads: music, performance, and then music again. Well, I don’t want to listen to performances with music. I only want to listen to the musical parts,? So, these were all were all live events that happened. So, first of all, I just went and said that I need to find the entire archive. Look far back. There are about 45 of them. I don’t

14:52 45 of them. I don’t like being on a bike, and when the performance starts,, trying to do something on the bike and fast and fast forward. So I got curious. Yes. Yes. And I like certain artists there. There ‘s this guy named Matt Maher, he’s awesome, awesome, and I thought I just wanted his stuff on a separate album. So, of course, let’s take a look. Claude Code came on Apple Music,, and I’m like, ” Let’s see if we can do this.” I want you to define where the music is and where the performance is, and break it down into separate files and separate albums. Then

15:23 separate albums. Then I want you to identify the singer by his voice, because, for example, Matt Maher is singing in this file. . Of course. Aha. Aha. And I want it to be on his own his own album. album. Well, it took a little while, but what? 45 files, each one and a half hours long. He determined where the music was, where the performance was, cut them up, separated the performances, put them in a file, then took the music at the beginning and at the end, put it together, said: ” Yes, this is Matt Maher.” And when you weren’t sure, hey, I created

15:53 , hey, I created a little HTML file for you with a play button for each one so you could figure out where Matt Maher is. Maher is. I’m not sure what exactly. Wow. And everything was ready. Now I have three albums: “Miscellaneous”, “Matt Maher” and ” Performances”. I’ve wanted to do this for years, Michael. And the fact that whatever you can come up with, as long as you have the right results, well, the solutions are ready, Fable 5. 1 just came out. They have a whole page with the

16:23 page with the structure. Yes. So, and I think the goal, the moral of the story is, Fable 511 comes out, and Claude always has an article or a document on how to create queries for a particular model, and I think it’s really important to read that if you’re going to going to use new use new models to understand how best to instruct them, because that’s a huge part of it. But yeah, I think there’s something something really cool going on, Mike, just a personal opinion, but hey, let’s get to the news. news. Okay, let’s move away from this.

16:54 move away from this. Talking about AI, because everyone literally, we talk about it every day, because it’s so, it’s so revolutionary. We, we, we we, we are in a whole new revolution right now. It’s happening and I, if you’re not there yet, you need to get more involved. You need to need to keep keep diving into it because it gets wild, crazy, and cool. Yes. But at the same time, Tommy, you have a band coming up in Chicago soon. That’s right. And speaking of ” don’t do AI,” what about using using AI tools to rein in Microsoft? Oh, how ironic. But no, honestly, Mike, again, it’s already

17:25 , it’s already September. So if you’re in the Chicago area or planning a visit on September 24th, we’re going to do a full end-to-end infrastructure understanding development using Notion during the meetup. But we’ll just talk about how to build context correctly. This is all based on a four- four- part series on my blog PromptingBI that goes through understanding how to build context into any of your tools, tools, using that to create to create prompts, prompts, customizing their

17:56 customizing their interactions, and how we actually go from encounter to outcome. We’re going to going to interactively discuss during the meeting what we’d like to bring to the project, and we’re going to build right from there. So it’s pretty crazy. I like it. Yes. This is another one of those cases where AI is just everywhere. This is very common these days, but but also how to use Fabric in PowerBI. This, this is our goal. Yes. Yes. You should pay attention to this event. It’s Thursday, September 24th, and it’s afternoon.

18:27 September 24th, and it’s afternoon. This is not, this is not in the evening. So it will be from PM to PM, effectively at the end of your work day. So if you want to learn more about how how to use AI in your Fabric, I think this will be extremely effective. Tommy, you’ve already shared how you build full-fledged models and reports based on Strava data. This is exactly what people need to understand. We must realize that the world we live in live in has changed significantly. 100% of the time. And we’re no longer just slowly clicking buttons. buttons. Yes. Yes. Okay, let’s move on to other news. And, Mike, I think some of them

18:57 I think some of them will become topics for future episodes, because they deserve it. I want it quickly… yes. Well, the first one is a new set of comprehensive CI/CD resources for Microsoft Fabric. It’s on the Fabric blog, and it’s a real guide from basic concepts to automating development, testing, and production environments. It is essentially a set of guidelines for everything you want to do and add to a add to a version control system. And I think, Mike, the reason is that often when

19:28 often when we think about version control, most people focus only on semantic models and Power BI reports. However, in Fabric solutions, Mike, I think there are currently over 70 types of objects or artifacts that can be created. So yes. I know that 10 of them are mine. So I know I created 10 items from that list of 70, and that’s just one workload that you see. And so… you’re

19:58 see. And so… you’re right, Tommy. There are a lot of them… And regarding your point, artificial intelligence systems will only increase this number. You will just create create more things. You used to think you had a lot of reports, right? right? When people got access to Power BI. And now you give them access to Power BI and agents. agents. Yes. Yes. And now there will be three or even five times as many. many. I think you’ll reach 70 by the end of the year, at

20:28 your current pace. We are moving fast. We create really interesting things. That’s right. Yes. So it’s worth going through the recommended workflow. workflow. Understand, plan, plan, practice, practice, implement and improve. And again, all the resources are broken down by tier: REST API, Git integration, deployment variable libraries, which is very important, Fabric CLI, Terraform, Fabric CI/CD library, and enterprise architecture. So, these are the concepts and best practices. Mike, I think we should invite Mathias, we should

21:00 talk to someone about this. I think we should do a multi- multi- part series on this because you and I love it, and I think it think it deserves attention now that we have the almost comprehensive guide that Microsoft has finally provided for CI/ CD in Fabric. Yes. And while I really like it, there is some is some subjectivity here, like what exactly does your team understand? What is your team comfortable working with? And so,, there are definitely some more nuanced points here that you’ll have to make

21:30 make decisions about within your organization., but overall, I think this is definitely the right move. move. It’s just It’s just overwhelming because there are so many options. So, this is a great article. Add her to the chat too. So if you want to want to check it out, it’s in the description below and also here in the chat. What else, Tommy? Tommy? Okay, Mike, real quick. So, do you like you like first first generation (Gen1) data streams, but would like to have second second generation (Gen2) data streams? Well, now there’s an update for,, it’s an update for the update wizard. So, a great name.

22:00 So, a great name. So, what exactly can you do with the Data Flow Update Wizard? This is a self-service tool for migrating any Gen1 data flows to Gen2 for CI/CD with support for key features. Each data stream is evaluated before each update, and again, this is not mandatory. But of course, we know that at some point, Gen1 data streams will, in a sense, a sense, become un- creatable. At this point, although they haven’t said it for sure, they have stated that they plan to do it in do it in the future, but there is no exact date yet.

22:31 no exact date yet. So, all you have to do is simply go to the menu next to the Gen1 data stream, select “upgrade to Gen2”, and the wizard will load. So they . So they really keep pushing it. But I pushing it. But, Mike, do mean, Mike, do you really

22:42 you really need to build Gen1 right now? Probably not. If you’re going to create a create a data flow, let’s say this. And don’t try to dodge this by saying I saying I don’t create data streams at all. If you’re building a Gen1, do you really need this? Well, what I hear at the core of this article is, “Hey, look, we created a wizard to make you spend more money every time you run the pipeline.” That’s what I heard here. , so, the

23:12 Gen2 data stream architecture is significantly different from the original. Eh, it does a lot more extra stuff than it should., there are ways to tune to tune gen 2 data streams. It’s not entirely obvious to me because gen 2 data streams have a lot more parameters that can be tweaked to speed things up and improve improve performance. So Tommy, will I ever create gen 1 data streams? Most likely, no. But it’s also a feature that exists exclusively in Power BI. If your company has not enabled Fabric and

23:42 not enabled Fabric and you do not have the ability to use Fabric in your organization, you are forced forced to use to use gen 1 data streams. Yes. Yes. So it really looks like it’s forcing everyone, I think many are still using using gen 1 data streams. My opinion is this. I think a lot of people created them. They are logical. They are similar to Power Query. It’s just understandable to people. I think many people still use them. And you would n’t release a master if you didn’t have a lot of

24:12 a lot of migration tools, right? You don’t create wizards for multiple migrations that need to be performed. You’re creating a master because you have hundreds of thousands of migrations that need to be done because you’re trying to slow down or stop the development of gen 1 data flows. I gen 1 data flows., that’s why it mean, that’s why it exists, right? This is not the same as “yes, upgrade to a datamart.” So,, I remember those days. Okay, Mike. We’ll save the last article for later because I really want to discuss discuss data agents with you, Mike. Two main features

24:42 Two main features that really get us get us talking about our main episode, our topic today. We’ve already done a few episodes about data agents. However, there are two major things we’ve seen in just the last couple of weeks that show Microsoft is doubling down on data data agent capabilities. Yes. Copilot Studio and Foundry. So let’s get started for those who are listening for the first time or haven’t heard our previous release. Mike, what the hell are we talking about

25:13 we talking about when we say “data agent”? agent”? So, let’s So, let’s figure out what a data agent does. Let me Let me ask you ask you a few questions, Tom. Have you created a data data agent lately? You’re just, okay, simply put, of course., they… that’s how I interpret them. A data agent is a tool that you can use to connect to Fabric data sources, such as Lakehouse tables or semantic models, and add add instructions and information to them. So, there’s

25:45 information to them. So, there’s a lot of information that an agent needs to understand in order to connect to and interact with different data sources, , right? So, if you look at it in general terms? If I interact with a semantic model, I need to I need to communicate in the DAX language. . This is the language I use to use to retrieve tables and join that data. If I’m talking to a data agent and I want to go to Lakehouse, that’s a different computing computing mechanism. To write queries against tables inside Lakehouse, you use Spark. Or TSQL may be used behind the scenes

26:15 Or TSQL may be used behind the scenes . If you add a SQL database, SQL tables, or data warehouse to an agent, you start using TSQL, meaning the agent must communicate with these tables via SQL to return to return results. KQL or Kusto is a different language. Therefore, the agent must be able to perceive questions in natural language. For example: ” Hi Tommy, what are my sales for the year broken down by week by week or month?” The agent

26:46 or month?” The agent must be able to interpret this text, determine what language to use, what syntax to apply, and how to retrieve the data and return the results to the user. It seems to me that Microsoft Microsoft is using some special special approach here. They are trying to make things easier for the data agent. They decided, “Let’s put all of this together— natural language in DAX, natural language in SQL, natural language in Spark—into one package, and that’s what data agents are.”

27:19 At least, that’s how I understand it. Is this what a data agent is to you? I’m going to digress a bit from this explanation. That’s a great explanation, but if you compare it to Copilot in Power BI or in Fabric, I think that’s where data agents have their true purpose. So, if you go if you go to Copilot to ask Power BI or Fabric something about your data, it will be a very general answer; you can ask about a specific semantic model or report and it will give you a query, but it will be very general because it

27:50 general because it just answers your specific question. The difference with a data agent is that you can bind it to more than just a specific specific data source, like this semantic model or this lakehouse. Yes. You can also provide it with a set of instructions on what what the purpose of this data agent is. Yes. This doesn’t mean just plugging into a semantic model where, let’s say, you have a connection to cycling. You are connected to the Strava semantic model, the cycling semantic model. Well

28:21 cycling semantic model. Well, that’s good. But if someone asks a question or tries to give a hint without any instructions, they will just try to make a request every time. Another part that makes a data agent so special is the ability to give it instructions about its purpose: what to look for in this dataset, what are the key relationships, key columns, key metrics needed for this model, and how it should respond or what the output to output to the user should be. What, what is

28:51 the user should be. What, what is the purpose of this agent? This agent is a trip. For example, I want to create a cycling coaching agent whose goal is not just to answer to answer questions, but to coach coach a user who can ask, “Hey, how much has the wind been affecting me lately?”, lately?”, when is the best time to go, considering the wind conditions?, should I reduce my workload workload next week? Based on these instructions, you provide detailed answers, not just numbers. So, this is also fully available

29:22 this is also fully available in markdown format. So, unlike Copilot or other tools that simply plug into a source, it has both the connection and the instructions. To your point, which I’ll quickly answer: depending on the source, it has DAX generative functions, for example, as you said, if it’s lakehouse, it , it will use SQL. If it’s a semantic model, it model, it will generate DAX on the backend, which is another really powerful part. So this is the essence of what a data agent is.

29:52 However, we had limitations, Mike, because this is all great because I get to talk to a data agent. But two problems were the availability of the data agent. agent. We are talking about problems. We are the problem. I want to, let’s get to that in a moment, because that’s what the articles are looking at, for example, were there any difficulties with where to place the data agents? How do we extend them to other parts of our business? I agree. I agree with you, Tommy. But I want to add one more important point. Yes.

30:22 Imagine this mental model: you’re talking to an AI model and giving it a semantic model, right? The agent or AI only sees what you grant access to, what you specify in the prompt that is given to you. So one of the advantages of a data agent is that is that if you give it a semantic model or a few tables from it, I believe the agent will be able to read the relationships, the metrics, and everything that’s inside the semantic model.

30:52 semantic model. That’s where our business logic is, and Michael Russo can talk about it all day. He says: don’t just give agents spreadsheets with the words “figure it out for yourself.” You must provide agents with a semantic model. You provide them with column definitions and descriptions, as you, Tommy, mentioned, right? Do I include a lot of information in the instructions to the semantic model model about about which columns are important, which are valuable and which are not, and how I should proceed? To some extent yes, but I think it’s better to put this directly into the

31:22 directly into the semantic model. If I give someone a semantic model and they can’t can’t figure out how the relationships work, of course, how much more confused is the agent going to be if you have redundant ways of computing the same thing? This doesn’t make sense. sense. So clear descriptions of why why tables exist, where they came from, all of that can be stored in a semantic model along with the description, adding relationships and creating metrics, right? These are all clues for the agent to help him help him understand what is happening. So, the

31:52 happening. So, the semantic model is extremely extremely important here. Oh yes. For comparison: if I give the same agent a lakehouse, there are no are no column descriptions in the lakehouse. You won’t You won’t get connections at the lakehouse. There are no indicators in the lakehouse. These are all things an agent might need to understand. Hey, when you take this sales table, this column, you see that this is a number because it

32:22 number because it comes with the table, but you need to sum it up, and these are the same instructions — sum this column, which now means total sales. So sales. So you have to give the agent more instructions when you work with certain data sets. SQL also has SQL tables. There are SQL definitions of tables. tables. Although the idea with lakehouse is similar if you consider it from the point of view of connections and semantic level. I think in SQL you can even even define primary and foreign keys. So, again, the agent has to

32:52 the agent has to query the SQL server, database, or tables to get these relationship fragments, right? So the semantic model is simply a much more versatile tool that tool that works better as an intermediary between humans and computers. I think this is a much better solution. So, in summary, I look at this and think: yes, I get it. And now we move on to the next part next part of this of this article: okay, now we understand what an agent is, a data agent. What’s the problem, Tommy? Like, okay,

33:22 Tommy? Like, okay, great. That sounds amazing. I have to use this use this everywhere. What was the problem before? Yes. The biggest problem was how to how to use it, right? The problem is that I can have specific agents even for one semantic model. I can have one agent per semantic model. Again, it’s not just about connecting. It is not just a data agent for the source. Data agents work best when they perform a single or very very specialized function. At least, from my experience. I would be happy happy to listen to people in the chat

33:52 to listen to people in the chat if they have seen other approaches, but I want to give instructions on what he should do—not just answer questions based on the model, but, for example, act as a salesperson in the medical field. So, you

34:03 medical field. So, you have to diagnose people’s quotas and understand what’s happening to them. . The problem was that with all these different agents you could create, it was difficult for organizations to use, right? It wasn’t a very comfortable place. There was integration with Copilot in the studio, but it didn’t work very work very well because you had to connect them as agents. There was a bunch of everything, it wasn’t a coherent instrument. And Foundry had it, but you , but you needed an ID. You

34:33 needed an ID. You could could only connect one data agent at a time, because again, let’s think about it. In Foundry, I can create models or agents that, to some extent, connect to other things. So I’m creating an agent in Foundry that can do a lot of things. Now, Mike, we have major updates to the data agents in both the Copilot and Foundry environments. One of the most important things is the ability to easily find data agents in the OneLake catalog in Foundry.

35:04 OneLake catalog in Foundry. You don’t You don’t need to worry about ID anymore. You don’t need to name this ID, agents appear with name, location, and all available information. One of the coolest features is that a is that a single Foundry agent can connect to multiple multiple data agents in Fabric. So I’ll say it again. The agent I create in Foundry can have connections to multiple multiple Data Fabric agents. This allows us allows us to connect multiple Fabric agents to one

35:35 Fabric agents to one, providing deep and deep and multi-domain business analytics. For example, I can connect my my sales, supply, and support data agents to a single Foundry agent. So, it’s not just a one- one- to-one relationship. This is quite important to me, and this is just from an MCP perspective, and Foundry Copilot now allows you to connect to a data agent as a tool. Copilot Studio has the concept of an agent that you create, and it has certain tools.

36:06 certain tools. Mike, all this means is that data agents aren’t just being updated, their purpose is changing. So I’ll stop here. Many updates, many improvements. Let me know what you think or if I missed any major any major updates. Yeah, I think… there are a few points here, right? A lot of this is about Microsoft Foundry. I think the main idea is that you have Foundry, you have data in Fabric, and you need to move

36:37 need to move it to Foundry agents. The Foundry agent is interesting in that it publishes a set of instructions that can then be distributed within an organization. Foundry agents allow you to add them to Teams, you can call them, you can create, for example, a Foundry agent for sales. And this Foundry Foundry sales agent can access data or documentation coming coming directly from your semantic models. So, you create an agent in

37:07 create an agent in Foundry. You create everything you need there. I think this can also be done in Copilot Studio. But, in my opinion, Foundry gives a little more control. There I have a few more parameters that I can tweak can tweak and adjust. You can choose which model you want to use. to use. Want to Want to use one use one of ChatGPT’s models? Or do you want you want to use an to use an open- open- scale model? You can also choose this option. You can adjust your spending depending on what you do. So, all of this is available to you. And you’re essentially saying:

37:37 you’re essentially saying: I have a lot of work that work that my data team has already done. . I am now providing this to my agent and it can be can be distributed throughout your organization. I think this is the most interesting part. That’s what makes the most sense to me: how you take your data and distribute it throughout the company. Previously, you could could only add one data agent. Here’s someone in the chat, I think it’s David, who noticed that the data agent has a limit on

38:07 data agent has a limit on the amount of instructions. You can can use 15, 000 characters in instructions. So, if you want to create one monolithic data agent M-m. for everything in the world. Previously, you could could only use only use one agent one agent inside your Foundry. But Foundry could be smarter. You can create a SQL Data Agent. You can create a create a data agent for lakehouse. You can create a Custo- agent or a

38:37 agent or a real-time data agent. You can create multiple agents, each specializing in its own data source. And what’s interesting about Foundry now is that it’s become smart enough to detect which agent you’re using, using, because you can now add multiple agents to Foundry. And I think that frees your hands a little bit. Yes. Because now you are not limited to just one agent and only 15, 000 characters in the instructions. You now create

39:07 create specialized agents that are customized for a specific type of tool or tool or dataset. dataset. That is why, in my opinion, this is an interesting direction of development. And I think one of the main points that we talked about at the very beginning, Mike—and if I were to rank the four main features, I would put this at the top—is the idea of ​​using Fabric Data Agents through the Model Context Protocol (MCP), which we mentioned earlier. So when we talk about

39:37 when we talk about MCP, from a corporate perspective it’s really important because it’s the best way for these tools to interact with with each other. Mike, I want to ask you this, as you see how they support data agents. I’ll try to paint you a picture of what Microsoft has done and what they haven’t done in terms of the direction I think they’re going with data agents. They created data agents to support the larger

40:07 support the larger agent. So far, they have n’t created an interface, a centralized place where people can go, a go, a central path to connect directly to the data agent. I can’t go into Anthropic and talk to my data agent, can I? I can’t go to another tool and have a conversation with a data agent there. There really isn’t a convenient central place, even in Copilot in PowerBI, to chat with a data agent. I can’t

40:37 data agent. I can’t reference reference it like I can with a with a slash or the ”@” symbol in any other chat tool like tool like ChatGPT. I can’t do this in Copilot and reference that data agent in PowerBI. However, this is something they didn’t do, and that’s okay. What they do, and how I think they’re positioning data agents now, is the concept of subagents, which is what most most tools use. To

41:10 tools use. To me, the data agent now now looks more and more like a sub-agent rather than the primary way of communication between the user and any agent any agent environment. This is an agent subprocess, not the main communication channel. And I think this is an evolution for Microsoft, because at the beginning the idea was: create a data agent and communicate with it. It was wonderful. However, they did not see this process working. But this approach seems

41:40 approach seems much more logical to me: I communicate with my agent created in Copilot Studio, which is available everywhere in my my Microsoft environment, or in Foundry, which can be can be used used anywhere, for example, in terminal chats or Cursor, and my main agents launch the data agent when needed. This, in my opinion, is the better approach. So, Mike, So, Mike, at least that’s how I see things unfolding. What is your opinion on this?

42:11 Well, I wasn’t exactly moving in that direction. I just . I just got stuck on a part of the article that I found very interesting. And maybe this is a little more subtle, because you can plug in an agent and make it work. It is useful to understand how your users users interact with the data agent and what information they request. request. This is a completely different part. Yes. Another unknown, because before, people just connected a data agent. I didn’t get a lot of details or information about it.

42:41 information about it. So now there’s this interesting aspect: connecting an agent is one thing. They talk about tracing and logging in the Foundry monitoring system. So, when the agent is connected and running, these traces and logs of conversations that you have with the agent are very interesting. They have a small graphical interface. They have different stages of dialogue. Okay, you call the agent. The tool is being executed executed. The agent is called again. Another tool launch is in progress.

43:11 . He provides you with everything. It shows what input data was data was generated. It shows the result you got. And then it reports how much time was spent in the chat module, how many milliseconds were used. So, we have a very detailed diagnosis of all this information. One thing I don’t understand here, Tommy, is that this is great. Azure Foundry has logs for this, and you can see what agents and users and users are talking about with the agent. Steeply. As

43:42 agent. Steeply. As developers of AI systems, we need this. We should be able to able to review this, but there will be will be so much new data generated. We need a way to bring this down to the bottom line of conversations. For example, when Tommy is talking to his model, this is what he asks, right? Thematically, right? Which tables are involved, which queries are executed. This gives an idea of ​​what is important to Tommy when he communicates with my model. So it’s great

44:12 model. So it’s great that we have the ability to log. I to log. I like it. Very happy with this. My next thought or next step in my thinking—how do I put all this together? How do you start to put this together to analyze? analyze? Because simply collecting logs doesn’t help. And I personally am not going to going to dig through the logs myself. If I can’t point an agent at the logs and say, ” analyze,” it’s not going to be useful to me. I don’t care. It’s that simple, Mike. Just collect logs,

44:42 Just collect logs, send them to Fabric, create a create a data agent to work with those logs. And then I’ll have to have a have a data agent to analyze a analyze a data agent that analyzes a data agent. It’s like… … How do people even ask about logs? And then… Meanwhile, I see Microsoft’s eyes shining with money: “ding- ding, ding-ding.” They just see the money flowing. Yes. Aha. Yes. Aha. Laughing on the way to the bank. bank. You bring up the issue of logs when I think that… That deserves some

45:12 … That deserves some credibility. But, Mike, for me, this is a feature update. I will object a little, because I said and emphasize: I think Microsoft

45:24 Microsoft changed changed the purpose of the data agent in these two articles, and not just added features. So, the role of the data agent in an organization has now changed in terms of terms of usability, and, in my opinion, how Microsoft perceives them. They’re not the primary communication tool, at least that’s the case, and I’m wondering: from an observability perspective, if I just if I just connect and have multiple subagents, does each Fabric

45:54 each Fabric data agent get its own log, or is it a big picture… so does it know that, hey, when I was talking to this Foundry agent, it launched or talked to the sales data agent, and then talked to the delivery team data agent? So I would be interested to know about that. But for . But for me, Mike, the big picture is that we’re changing the approach to why you should build a data agent for an organization. True? So if someone came to me and said, “Hey Tommy, I

46:24 said, “Hey Tommy, I want you or Mike, I want to I want to use AI with my semantic models. I’ve heard about this thing with data agents. So can you create a create a data agent for me? "" At first, you would probably say, “Yes, I will create a data agent.” ” But now I But now I hear more and more often: ” Well, we’re going to do this: we’re going to build a Foundry agent or a Copilot agent and, essentially, this data agent. “I’m not going to tell the client all this, or it will be in the technical specifications, but we are launching these subagents, which are actually data agents. So, Mike,

46:54 agents. So, Mike, does this change your perception of what a data agent is? No, I don’t think so. I think it just expands my understanding of what it already is. I don’t think this changes anything here.,, it’s not like there’s anything revolutionary here. They do not add new add new capabilities to the data agent. Data agents do not gain new skills. The instructions are still, well, there are a lot of things that I think the data agent could do better., and, someone here is commenting in the chat,, it’s really sad that we can’t add semantic models as a source in

47:25 models as a source in Copilot Studio, otherwise we would be choosing LLMs ourselves and configuring them for business logic and context in Copilot Studio. No, but you create a data agent directly in Copilot Studio. He says you can’t directly directly add a semantic model. That’s the point. You cannot speak to her directly in Copilot Studio. Create a Fabric data agent in Copilot Studio. This is not a problem. This, in fact, is the point. No, no. He says he doesn’t want to use a use a data agent. The data agent does not allow you to choose a model. A data agent is a wrapper

47:55 data agent is a wrapper around a model and some data, right? It works with the AI ​​that Microsoft provides. You don’t get to choose what agent it is. You can’t choose a large language model. Of course. Good. Of course. So, you lose it. But I guess why do data agents even exist, right? They exist so that you get you get natural language for DAX or SQL. Microsoft implements certain things and you have to have to use the use the model they impose on you,

48:25 impose on you, which I don’t really like. This doesn’t make much sense. I understand,, I understand why Microsoft does this. However, I don’t really like it, and I’m a big fan of having more control, right? Data agents seem somewhat limited., I recently took a test. I ran four queries against the

48:55 four queries against the semantic model, and I think I only asked four questions, made four iterations between me and the model, interacting with this data agent. It cost me 400 CU for just 5 minutes of talk time, and 400 CU on the F2 SKU will drain 45 minutes of capacity in about 5 minutes. So it’s expensive to use these things. They are not cheap to run as they are large

49:25 as they are large language models and the CU consumption is quite high when you query data from agents. agents. So an alternative to this solution is to use the XMLA endpoint. Provide the semantic model definition directly to your data agent. Tell your agent to access the API, authenticate, and use API use API calls. There are many other ways to get the answer, but I believe this is an attempt to make it easier to access the data that

49:55 access the data that resides resides inside these semantic models. I directly used Power BI, Power BI semantic modeling. Currently, semantic modeling has tools for tools for editing models, but you can also use use MCP Server for Power BI modeling. No data agent. You get full access to read all the details of the model. You can You can query the model using DAX. So, the

50:26 using DAX. So, the MCP modeling server does exactly what the agent does, but only for the semantic model. Yes. But many organizations, Mike, probably won’t allow such access in their Anthropic unless they already have a corporate security system set up. Got it. So, that’s what Copilot, and especially Foundry, is for. If your company doesn’t allow it allow it, , it’s holding your people back. This is a conversation… Okay, I’m a

50:56 This is a conversation… Okay, I’m a different conversation. Say it again. So what will this topic be, because I will add it to our backlog. So, So, because it’s actually true: if your organization limits employees’ access to AI, you are directly directly inhibiting their productivity. For example, the amount of added value from training the team on how on how to use AI, how to apply it, and how to use it in use it in conjunction with things like Power BI. To your words, Tommy, you prepared a bunch of data and a report in 30 minutes, and it was just like a side project. Imagine

51:27 project. Imagine providing that level of skills and capabilities to the rest of your team. Mike, I understand. Yes. So, I So, I understand you. I’m just talking about organizations that are outright saying it’s dangerous. we can’t do this. Dude, you are obsolete, you will be bypassed and replaced by other organizations that have made decisions, chosen safe tools for themselves, and they will do what you do not allow your your company, your employees to do; you will simply be outdone. Before you trap me in talking about this until the end of

51:57 about this until the end of the episode, I’ll put it on the backlog. We will dedicate a whole hour to this. This will… I want to make arguments so badly just to just to annoy you, but I won’t do that right now. So we will have guests coming to our podcast. Maybe this would be a good topic for one of the guests to explore as well. Like Like Yes. Yes. Yes. There ‘s a management issue here, but there’s also, you also,, if you know, if you really want to increase the productivity of your people 10 times, what’s the template for that? How do we really ignite them and make them truly

52:27 truly professional in Power BI and Fabric? Apparently, turn on AI and teach them to use it effectively. OK. So, Mike, for the sake of the data agent episode, I’m going to do something I’ve been doing doing with my kids lately when we’re discussing the weekend or what we did. And I’ll give you, we’ll call it a call it a pedestal. I’m going to give you a list of features or updates that have come out here, and I need you to rank them by bronze, silver, and gold in

52:58 silver, and gold in order of importance. Some of this list won’t make it to the podium because I have six of these updates. I need you to tell me what you have bronze, what silver, and what gold. We rate this based on importance and ease of ease of use. OK. Are you ready? Yes. So, the first one is MCP and foundry, the ability to use use data agents through MCP and foundry. This is the first.

53:28 The second is that we have a one lake directory in the foundry service. Third, and if you need to write it down, let me know. The third is support for multiple data agents, available in both fabric, foundry, and co-pilot. The fourth is that the IQMCP fabric is available in copilot. Fifth, this is the logging and observability feature that you talked about, the fact that all these traces, logs, and requests are actually

53:58 requests are actually sent. And the sixth feature that could potentially make it to the podium is that with copilot studio I can publish this to Microsoft teams or anywhere in my organization for people to access. So, once again: first—MCP and foundry, second— second— catalog availability. Yes. Number three— multiple multiple agent support. Number four

54:30 agent support. Number four is that fabric IQMCP is available in Copilot Studio. Fifth—the logging function, and sixth— access to teams. I think I can Yes, I think I, well, since you, I think I understand everything. I think I’m ready. OK. I will probably

55:01 rely heavily on I think multi-agency is a big win. Especially when you look at the limitations on what can be done, because there are only a certain number of instructions that can be given to a single data agent. Data agents don’t have skills, right? You can’t give them much additional context. So if you are trying to build a lakehouse and a semantic model, I don’t recommend making one data agent for all of that. I think my advantage would be to

55:33 say to a foundry agent or co-pilot studio, “Hey, I have two data agents.” If you are asking a question about the semantic model that contains this information, go here. If you are using this information from lakehouse, go here. Is n’t that right? I think it’s more logical logical to use to use multi-agent experience. This way, one agent will not make requests to different sources. This is the same effect as using using subagents in your programming solutions, right? Subagents Subagents have a new context window. They are smaller

56:03 context window. They are smaller. They are more efficient in terms of characteristics. So the more specific the task you give them, the better their performance will be. The system is not cluttered with a bunch of other other information and confusion. What was that? was that? Is this your bronze, silver, or gold level? level? I guess I’d say this is my gold, my golden pedestal. The next point is probably more for administrators. Many of these agent things are still little known. So I’ll say that logs are probably my silver lining. This is quite logical. And I do

56:34 This is quite logical. And I do n’t even know what my bronze medal will be here. Maybe bronze is a message in Teams, right? Because what good is an agent if you can’t

56:45 if you can’t communicate with them? Where to use it to use it? I’m constantly on Teams every day, so why not make it available in Teams or in channels for many colleagues in my company, huh? Ask him questions and get answers directly there. No, Mike, we really agree here . So I’ll add something, but I have the same pedestal as you. It can’t be. We came to the same conclusion. Although, I think everything is pretty obvious here.

57:15 pretty obvious here. This never happens. happens. This is very rare. So if you’re listening to us, you’ll have to wait another 560 episodes. But regarding multi-agent support, Mike, as we were talking, I was already thinking about how to write instructions for data agents. One of the things I’ll probably do is point out, “You’re a subagent. ” This is a typical practice in prompt engineering: define roles and define existence.

57:48 Why not write in the instructions: “You are a subagent of any Foundry or Copilot agent who contacts you. When you are called upon, you perform a specific task,” but to call a spade a spade. This is a subagent. This is what a data agent should be in Fabric. And the fact that I can have several of them. I can have agents for sales and sales and supply chains. I can make sales and finance,, check each other to some extent. Log, Mike. So, let’s So, let’s put this in Fabric and then understand what questions people are asking and what

58:18 and what those interactions look like. Let’s make our data better. This is great. Look, this is another way to get usage analytics for what we do. And again, about the commands, I didn’t add MCP here, Mike, because I feel like it’s already a standard, not just a feature. And regarding your comment: where will all this be placed? So, Mike, I think we understand everything correctly. So, cheers, as we as we come to come to a close, what is your view on where where data agents actually exist? And I’ll

58:50 data agents actually exist? And I’ll probably probably end with this. For me, I really think that the idea of ​​a Fabric agent as a subagent would be the best way to use them. I think we haven’t seen data agents really really succeed and be widely adopted yet because there hasn’t been a sense of: is this the main agent? Is this the main means of communication? Do where I’m supposed to access it? I probably never need to need to directly directly interact with a data agent as a

59:20 data agent as a user. I user. I will let my agents, the main lead agents who are already doing other tasks, do this. And to me, we’re finally seeing a path and a direction where data agents really exist. Your comment makes me think, me think, Tommy. Have we gone beyond what a data agent is actually capable of? I think we What

59:51 is that our agents and the tools that we are given today For example, I want to go to Anthropic, use Claude and just talk to the data, the semantic model, right? This is what I want to do. Do the tools and capabilities that other providers, other than Microsoft, provide us with today surpass the value of a data agent and do they outpace the ability to require the use of a use of a data agent in this process? So,,

60:24 ? So,, data agents were this ability ability to turn natural language into DAX, natural language into SQL. It’s a wrapper wrapper around everything that directly interacts with your agents. The models have become extremely extremely powerful. Models are becoming truly intelligent. Models work great with APIs, and they become much more much more efficient. And models are now creating their own tools. So, do I want to pay more money for a data agent to run CUS, or am I better off spending a handful of tokens, five

60:55 a handful of tokens, five dollars a day, to build tools in Python to directly access the model without any data agent. Isn’t that right? So, there is a decision that needs to be made. I don’t know the answer to that, Tommy, but I’m just saying that the development of AI tools and models is moving very quickly. I’m not sure the sure the Microsoft team will be able to keep up with the data agent improvements needed to keep up with the overall progress in model

61:25 model and agent capabilities. Does this make sense? sense? Yes. Then I have an additional question for you. What does a data agent need to keep up? Because while you’re talking, I’m thinking about what exactly the data agent is missing. Obviously, it’s the ability to connect to a remote MCP, instantly make a request, use skills, get a response, and either answer my answer my question question directly in a chat interface like Anthropic, or start building something

61:55 building something based on that, right? This is not just a question and answer. I can literally go into Anthropic on my computer and ask, for example, let’s go back to cycling. How have my last few runs been? Here are the results. what? Build a ? Build a control panel based on this and he can start building it. A data agent only has such a narrow purpose, so I think the question comes down to your thesis: first, again, there are organizational security policies, and

62:26 security policies, and that’s another topic we discussed, and second, what would a data agent need to meet meet the requirements to be able to say it say it has these capabilities from the start? I start? I think it’s a skill function and a role that almost replaces MCP for the semantic model. Is n’t that right? So, these skills are technically already built into the data agent. I just can’t change them. Yes? I don’t see this., this is from another article about visualization. So of So of course, is there more

62:56 more customization needed, what do you expect from a data agent to be on par with the capabilities of Anthropic or other similar systems? And maybe maybe I am, I don’t know. This is still a relatively unexplored area. I’m just saying that that I’m not sure if people really want to use a to use a data agent. I think people just want to have a have a PowerBI MCP modeling server, talk talk directly to the model, and have the agent determine what’s in the

63:27 determine what’s in the model and find the right things. So I think the ability for an agent to understand what I need and then create my own Python scripts or use or use Fabric tools to get answers from the model is what I need. I don’t know if I want to do this. So either I spend 150 units of units of computing computing power power on each request to the data agent, or I spend 100, 000 tokens

63:58 spend 100, 000 tokens on a dialogue with the agent. OK. So, yes. To get what I want. want. But then I create repetitive processes, and this is where it gets a little more complicated, more technical. It is less deterministic. So how do you… I think this is a new area. I’m just saying that this is a new blurred area: how do you choose the right tool and what do people really want? And I think the speed at which Anthropic, OpenAI, and xAI

64:28 at which Anthropic, OpenAI, and xAI are creating these environments and tools like Cursor is outpacing the development of data agents. Of course, that’s true. I know we’re getting a little out of time, but let me just say this quickly: Mike, I imagined this scenario and immediately started laughing. Because imagine, a thousand people have access to this Fabric PowerBI MCP and can query the query the semantic model instead of using a data agent, right? A thousand employees in an organization want

64:58 organization want to know how sales are going. Can we trust Anthropic or other AI tools without proper instructions and thousands of different queries to get the right answer every time? ChatGPT gave me the wrong wrong answer twice yesterday, and I had to say, ” Think again,” to which he replied, “Yes, you’re right.” right.” You are right. So data agents, while they may lag behind in terms of functionality, do aim to ensure that the

65:28 to ensure that the entire entire organization receives consistently correct and accurate answers to its queries. I can’t guarantee this with Anthropic, but I know we’re already past our time. We’ll put that aside for later. But Mike, Mike, great conversation. But again, that’s what we’re we’re talking about here, it’s like I agree with you, Tommy. This is very uncharted territory. We need to figure this out. This will be an evolution. This will be constantly changing. . We don’t know. The community has not yet decided and

65:58 decided and developed best practices for how this works. So we’re still trying to trying to figure all this out and understand what makes the most sense here. And again, we’ll answer that today. In three weeks, the answer might be completely different, because new new technologies appear, new capabilities, a new model,, model,, Fable 6. 1 comes out, which eats up,, a billion tokens every half minute. Who knows? Who knows what will come of this. Download Download FCP agent data, like that. Yes. So, there can be

66:28 can be so much going on here, which is exciting, but it also means you’re trying to trying to hit a moving target. You won’t hit. You just have to go with the flow and say, “Listen, this is going to be a constant learning process. We’re in a new world. Everything is different. You need to figure out what works for you and your company. And move in that direction, and be flexible. Don’t approach this with rigid rigid beliefs, because everything is moving too fast to say, ‘This is the only way we do it,’ because there’s so

66:58 so much change happening right now that you can’t keep up with it. With that in mind, thank you very much for listening to today’s podcast. I thought it was a pretty good topic., it’s like it’s like, we’re , we’re always talking about a billion things about AI. So here’s another episode. Add it to the pile. So, Tommy, where else can you find the podcast? You can find us on Apple, Spotify, wherever you are, don’t forget to subscribe and leave a rating. It helps us a lot. Do you have a question, an idea, or a topic that you want us to cover in the next episode? Come on over to

67:28 Come on over to powerbi. tips/mpodcast. Leave your name and a great question. And finally, join us live every Tuesday and Thursday at AM Central Time on all on all PowerBI tips social media channels. Okay, that’s it, thank you all so much, see you see you next time. Tommy lights up the sky. Dance to the laughter in the mix and I’ll give you a drive. Clear steps. Hit the beat now.

68:01 Feel the crowd. Clear steps. Clear steps.

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.