Data Modeling in Event Driven Architectures – Ep. 471

When your data arrives as a stream of events rather than a snapshot of state, everything about modeling changes. Mike and Tommy discuss the architectural shift to event-driven systems and what BI developers need to understand about modeling on top of them.

News & Announcements

• Introducing Improvements to the Report Copilot in Power BI — Continued enhancements to the Copilot experience for report creation and editing.

• Introducing Power BI Controller: Streamlining Storytelling with Bulk Operations (Preview) — A new tool for managing report operations at scale, enabling bulk updates across multiple reports.

• Copy Data Across Tenants Using Copy Job in Fabric Data Factory — Cross-tenant data copy capabilities in Fabric Data Factory, simplifying multi-tenant data integration.

• SSMS 22 Meets Fabric Data Warehouse — SQL Server Management Studio 22 adds native support for Fabric Data Warehouse, bringing familiar tooling to the new platform.

Main Discussion: Event-Driven Data Modeling

State vs. Events

Traditional BI models are built on state: what does the database look like right now? Event-driven systems capture what happened: every change, action, and occurrence as an immutable record.

• State-based: Customer table has current address, current balance
• Event-based: Address changed event, payment received event, order placed event

Why It Matters for BI

When your lakehouse ingests event streams:

• Fact tables become event logs rather than transaction summaries
• Slowly changing dimensions are naturally captured (every change is an event)
• Time becomes central — every fact has a precise timestamp
• Aggregation patterns change — you’re often counting events, calculating durations between events, or deriving state from event sequences

Modeling Challenges

• Deriving current state from events — You need to “replay” or aggregate events to get the latest snapshot
• Volume — Event streams generate far more data than state snapshots
• Schema evolution — Event schemas change over time; your model needs to handle that
• Deduplication — Events can be delivered more than once

Practical Patterns

• Use materialized views or lakehouse tables to pre-aggregate events into state
• Build both: an event-level fact table for detailed analysis and a derived state table for standard reporting
• Partition by time — event data naturally partitions by date/hour
• Star schema still works — but your facts are events and your dimensions may need point-in-time awareness

Reference

• Event-Driven Architecture Style — Azure Architecture Center

Looking Forward

As more organizations adopt streaming data patterns (IoT, microservices, real-time apps), BI developers will increasingly model on top of event-driven sources. Understanding the difference between state and events—and knowing when to materialize one from the other—becomes a core skill.

Episode Transcript

Full verbatim transcript — click any timestamp to jump to that moment:

0:00 Hello everyone and welcome back to the explicit measures podcast with Tommy and Mike. Good morning everybody.

0:33 How are you doing my friend? It is good to see your face. We are back. So last two episodes we did were a pre-recorded sessions that were on out on the last week’s two episodes. This week we’re jumping right in. We’re are back live again. Tommy’s back from his conference. And before we get into the news, Tommy’s probably got some fun stories to tell us about the data dynamics summit conference recap. Let’s jump into what our main topic today is. Our main topic will be talking about how do you you

1:06 Leverage data modeling in an eventdriven type architecture when when events are part of your ecosystem. how do you do the data modeling? And again, we’re thinking my initial concept here is semantic models, but there may be something different. we should we should maybe expand our definition of what data modeling looks like when we’re talking about event driven architectures. So, we’ll jump into that here in just a moment. That being said, all right, Tommy, how was the conference? Oh, yeah. So, it was a honestly a great conference. So, I went to the Dynamics Summit Conference in or Orlando,

1:40 Florida. And Mike, we used to go to these all the time. world tour dynamics communities used to be a pretty large organization that was basically helping put on user groups and they did a ton of PowerBI world tours like seven every year and then the big dynamics summit we didn’t go out I think you and I went to the last one together when it was in Houston because it was all just dynamics business objects business central not a lot of power platform powerbi

2:12 I actually talked talked to the guy who I believe took over the company and they are doing a lot around fabric and data. So, it was a great conference. 6,000 people at the Gaylord Palms. Wow, that’s a lot of people. Yeah, it’s too hot for me. 80 75 80 is too hot for me. So, you your blood has thickened up since you’ve been living up here in the north, Tommy. My wife I always ask my wife, “How did we live in Florida?” She’s like, “You barely did because you complained every

2:43 Day we were married.” So maybe I [laughter] think but I did two sessions. I did one on data agents in the semantic model and I did one on promoting and adopting PowerBI in your organization. Funny enough, I mentioned this on the podcast before my initial session when I set the speak was on metric sets. I data agents, they wanted me to do the metric sets one. Well, we know in the beginning of October, PowerBI and Microsoft said

3:16 Metrics, it’s not a thing. No more. So, I had we had to change the website for the conference. However, at the per inerson my room where it says what’s the next session and on the planner app it was still saying metric set. So, obviously naturally no one really wanted to show up and then they would walk in. I’m going I’m not gaslighting you. We’re actually talking about data agents. And the reason why is metric sets is not worth your time. So, it’s not going to be around much longer.

3:48 And bas basically, it’s it’s out now. Like, it should be done. It’s it’s you cannot create metrics that I believe as of if you try today on Tuesday the 28th, you can’t. They’re gone. So, then they’ll be gone from your tenant in the middle of November. Anyways, my data agents session which I was able to do on Wednesday where it was all promoted and all that. Dude, Mike, I have to give it to data agents. I thought it was a dumb down version of AI, but with custom instructions in a semantic model. , it was one of I and

4:22 I don’t gloat or u blow my own smoke, but it was one of the best demos in terms of working and impact for people that I’ve done. Really that good? [clears throat and cough] Yeah. Well, this is this is the this is what I was talking I think we’ve talked about this before, Tommy, which is like, hey, we need to look at like what features come out from Microsoft and when the feature wows you, right? If it if it really does something that you’re like, wow, with without a lot of effort, you get a first you only get like a couple tries at a feature before people are like, is this

4:55 Useful? Is it not useful? If you don’t get that right, people walk away from it and they won’t revisit the feature for months, right? they just won’t touch it. So you you have this like very short window of like can you make this work really well and if you can’t then you’re probably going to be losing a bunch of people and they’re not going to revisit it. And this might have been like the story of metric sets here a little bit. People tried it and were like interesting but I don’t really find a use case for it and they just never came back. Nothing was ever added to it. Never gotten additional investment to make it more polished.

5:26 That was and again the initial one I forgot what it was called. It was it wasn’t data agency. It was like but when the first iteration of being able to create like the AI and connect to your own data but what it’s on now what they’ve really invested in is the data source implementation integration great but the custom instructions is integral here Mike. Yeah. Yeah. That’s what you you needed the extra prompting the the context the business context. Yeah. So I went through no prompt, a

5:59 Basic prompt, and then a full advanced prompt. And we saw how the agent literally behaved differently and and and got the answers right. And semantic models, I didn’t have to, , like basic things of the prompt was only use explicit measures that are already in the data set. Yeah. don’t rely on da da da thing. And then one of the things was if someone asks you for this particular measure but they don’t ask for a group or field please ask them to

6:32 Clarify one. So don’t give an answer all the time because that’s usually what happens with AI is you you give a broad question and rather than the AI saying can you clarify that actually having that conversation it just does a basically a search. Sure. And that worked incredibly well. , but no man, I I have to say I’m impressed with data agents and I I’m on the disciplehip right now of telling people there’s copilot and standalone.

7:04 Yep. There’s copilot and report and there’s copilot studio which is on PowerBI and there’s dating agents and none of these are the same. No, they they are different experiences. Yeah. Oh, good. I’m glad you’re finding some value from that one. So, that brings us basically to our first announcement here. , since we’re talking like the news items here, I think your your first one here, Tommy, is talking about introducing improvements to the co-pilot in PowerBI. Did you have any notes there around that one? So, this is I think like I told the attendees is right out of your book,

7:39 They asked like, should we invest our time in this? It’s like if you want to know where Microsoft is investing their products, look at the blog. Yep. Look at what updates are happening. That’s where their money is. that’s where their priorities are. And we’re seeing a lot with Copilot now where really you can you can even edit existing reports. It can now select the most appropriate visuals and go through better instructions. So this is pretty cool. So it will actually look for visual recommendations in your

8:11 Based on your data automatically. it’s live now in the service. It’s coming to desktop. It’s not desktop yet. But this is we’re seeing some pretty significant updates there. You can add a visuals, change or replace visuals. So this is all about building reports. Mhm. This makes more sense to me. And this is this is like the agent agents for reporting I guess it would be where I would maybe classify this one. Agent allows it to do things or edit things on your report page. I think

8:44 This is going to be extremely effective and this is I think the direction it should go. It should go more this direction. this is a lot of it’s going to make decisions or best practices or trying to make things that are useful here. There’s still always going to be this, , gap between like, hey, I just need a quick report getting some charts on a page and then there’s a polished report, right? So I I don’t see co-pilot right now being able to make a fully polished report. and it’s going to give you like a good starting point to begin, but you’re going to need to continue to

9:16 Like there’s going to have to still be individuals or there will be still individuals who are going to make these highly polished app data like app experiences as well. So, I don’t think those are going to fit the co-pilot like experience yet. but definitely nice to see that they’re going that direction. All right, what other news or articles that you have out here for us, Tommy? Oh, Mike, right down your alley. , introducing the PowerBI controller, which your first thought very broad, streamlining storytelling with bulk

9:48 Operations. Michael, when you read that title, even if you read the article, what what do you think that it’s going to be? Like the title doesn’t make sense. , control, I don’t even understand what it’s talking about. , you really have to go in and like understand like the the naming of this is very weird. You really have to read the article to understand like what is it trying to accomplish here. So yeah. So go ahead Tommy. This is all about PowerPoint. What I know thing power controllers sounds so cool like it

10:20 Something in the service that you could do but it it’s like a like an AI agent or I don’t know but it anyways it’s a PowerPoint addition or enhancement to the original plugin which simply allows you to serve as it’s it’s a pain it’s a pain it’s a PowerPoint pain for all your existing slides. So basically the idea here is you have to a lot of users have complained that they have to go through and replace images every month because they’re doing the same PowerBI embedded

10:54 Reports and every month they create a new presentation with the latest data yada yada yada. So what the controller comes in so you can switch all the add-ins from the current image to the live one and revert all the add-ins and give the finalized presentation making sure the data is up to date. So it simply is bulk operations multiple embedded reports and yeah I I it’s it’s a pain. So when I whenever I hear PowerBI inside

11:28 PowerPoint, there’s a couple key requests or features that I’m seeing that people need to have inside the PowerPoint slide deck. One of the things that I I see a lot of times or at least I’ve I’ve heard a lot of times I don’t use PowerPoint. My my opinion is don’t use PowerPoint for PowerBI. It’s not the right tool for it. Push people to the actual website and you’ll get the information there. So that that’s my initial opinion, but I realize slide decks need to be maintained. people need to continue to push in information. So this controller is really the way of letting you have more control about what is being presented in

12:00 The report. Do I iframe the report the PowerBI report is it interactive inside the power Power PowerPoint presentation or do I want a bunch of static images? So there is a feature here they call it’s called switch to snapshot where you’re able to present a snapshot [clears throat] mode of the report and I think this is going to be highly effective because a [clears throat] lot of times people will say I’m going to build the slide deck but I need the images of the report to be in the slide deck but they’re going to like put it away somewhere on a SharePoint drive or

12:32 They’re going to print it out and they don’t want the data changing from when they present it. There needs to be a static look or static lens of the data at that time when that present presentation was made. And this I understand, right? This is the whole point around PowerPoint presentations, right? This is my executive slide deck and we’re going to go through this every quarter or every month and we’re going to review where we’re at businesswise. You don’t want those slides changing over time. And so PowerBI doesn’t necessarily have this without a lot of additional effort and design in your semantic models. doesn’t really have a way of for you getting snapshots in time

13:05 Of data and saying this is the status of this report as of this date. But I think this is a really good solution for people who want snapshots of data and it feels like this controller is giving users these kinds of additional capabilities right specific features only for PowerPoint when you’re using the PowerBI plugin basically or the addin directly for PowerPoint. I think this makes sense. So what do you think? It makes sense. I It don’t get It’s a good name. It could have been saved in something else. And

13:37 Honestly though, too, if you are using the PowerBI plugin, you’re probably not using that on a one-off basis. Like, what ? That’s that addin is definitely something that is you’re not going to do oneoff. You’re the only person that knows it. Odds are that your company is saying, “Hey, if you’re going to show PowerBI, use the plugin.” So, that’s probably why there’s a lot of dependencies there. I guess I’m I’m I’m going to argue like if you’re if you’re already using PowerPoint today and taking screenshots of PowerBI reports and putting them into PowerPoint, this is definitely the tool

14:10 For you to go take a look at. I think it will simplify a lot of what you’re doing and potentially give you a lot more control around what you want it to do, what you want PowerB to do inside the PowerPoint presentation. So, if you use PowerPoint a lot for this stuff, then highly recommend this tool to go with your workflow. I think it’ll fit very well. So, I do like this one for those who actually need it. I do not need it 100%. All right, we got a few more. We’ll we’ll run through it. Fabric updates. Well, simplifying data ingestion with copy job, copy data

14:43 Across tenants with the fabric in fabric data factory. So, you can cross tenant data movement fabric to fabric, fabric to Azure, Azure to fabric, which is pretty cool. You can use ser service principle authentication and multiple types of delivery styles. You can bulk copy, incremental copy, even CDC replication. So pretty cool, Mike. And I think for a lot of organizations, there is a lot of multi-tenant usage where it’s not just also all fabric. There’s a

15:17 Lot of organizations I’m talking to today where especially government too. A lot of the government licensing is not available with all things in fabric yet. So they got to use Azure but they want to do fabric. So this is makes things a lot easier. so just making it easier and easier. You got to love it. What’s your thoughts, Mike? Yeah, I I think this is a really interesting feature here. just because what they’re doing here a little bit differently when you look through the instructions on this particular article and I’ll make sure I get this

15:48 One over here. Let me copy the link here and make sure I put this inside the the chat window here. This is a links in the chat. Now it’s also in the description down below. One of the main features here is they’re using a service principle an app registration that is being leveraged to connect to the data. So this is essentially allowing you to create an app registration, but during the copy job process, you’re able to go to that app registration and get access to that data on behalf of a different organization or a different Microsoft

16:21 Tenant. There’s becoming more and more of these use cases or at least the request I’m hearing on my side is tenantto tenant sharing of data is becoming more and more of a of a real pattern and use case here. So how do we do that? So this is a really interesting feature in that regard. I think it’s more technical in the nature of like the setup. The techn you have to like build something in Azure. There’s a service principle. You’re not using the workspace. This is a very in this example here they’re going from blob storage through to a lakehouse in some other tenant. So good use case. I think

16:55 This is useful. I think this people are going to ask for this. It needs to happen. and I as I look at this feature here I think okay it’s just like table stakes for me a little bit. This is something that I would expect needs to exist as part of the base feature set for those who are more into the data engineering, more in the advanced side of things. So anyways, I like this feature. I think it’s good. I again I’m still going to ri a little bit here on copy job a little bit. , I still experience copy job looks like it’s still a bit more expensive

17:27 Than it does when I run like a a pipeline or data flows gen 2 seems to have some cost implications as well. So I think that the pricing is getting more competitive against the different products or the different compute engines but still right now I’m I’m looking at this going my fan and favorite right now is still Spark. Spark still seems to be like the most efficient use of my compute unit so far. So, I like this feature. I think it’s useful. , I’m not sure how many customers or listeners that we’re

17:59 Looking we’re listening to right now are actually going to need cross tenant sharing. It seems like a very a little bit of a niche point, but again, you should be able to support it. Like this is this is like a table stakes thing in my opinion. It should just work. Yeah. Awesome. Let me ask you a question, Tommy, real quick about this one. So with this idea of a app service principle or app registration are you in your workflows Tommy are you exploring the workspace managed identity have you been building anything with that recently? what I actually just started

18:32 Dabbling with that? I talk about being thrown off when fabric API first came out. It was tough, man. But what they’re doing now, I’m like seeing a lot automation opportunities. But Mike, what what have you been doing that’s blown your mind? Not really blowing my mind per se. It’s just more of a it feels like a new best practice is starting to evolve, right? , if you are connecting to a SQL server, if you’re connecting to some

19:04 Other object inside the lakehouse, if you’re connecting between semantic models and lakehouses, you can now start providing the identity of when you create the connection between different items, you can start using the workspace identity as opposed to an individual users identity. And not all connections are covered yet. So it it’s not fully I guess would say rolled out to everything which is annoying cuz you would expect the workspace identity could be used with any connection type across anywhere inside fabric doesn’t

19:37 Feel like that’s the case. I’m trying to remember what we were trying to connect to. It wasn’t it wasn’t being covered and it was just it was weird that it wasn’t it didn’t exist. but it only gave me the OOTH 2.0 identity to sign in to to connect to the data source. So, it doesn’t feel like everything is yet being able to use be able to be used to to connect to things yet, but I really do like this idea of Tommy, your identity is no longer attached to all the connections. I I think that’s a much better way to handle things in the

20:10 Future here. So, dude, I am I love it. I love it. But Mike, I know we got one more and I don’t know how exciting this is, but let’s just go through it. Well, for you Tommy, I think this is a really exciting one. Eagle Server Management Studio 2020 or 22 meets Fabric Data Warehouse. Microsoft has put a strategic investment in for SQL developers and fabric and SQL server management studios SMS. tool used by millions of data based

20:45 Professionals. You can now have workspace names can show friendly names instead of that cryptic like I’m not sure if I’m connecting yeah connecting to something hackable code or what what it is from schema based object grouping tables views and store procedures organized based on schema especially if you’re using the one in lakehouse you can do warehouse snapshots contextware menus for cleaner menus and some of the things that are coming is context menu actions, extract the data

21:18 Pack, refresh the snapshot, fabric script generations and warehouse life cycle all ins SMS for Windows. cool. I think this is a straight pull for any of the SQL developers who’ve been building things in SQL in other places inside Microsoft. This is a straight pull to go into SQL Server Management Studio, right, for fabric. , this makes sense. If you’re going to put SQL server and data warehousing inside fabric, you’ve got to have the tools support it 100%. I think this is a table

21:51 Stakes for me. I I think this is useful. I think the SQL community needs this so they feel comfortable in fabric and have the existing SMS experiences that they’re used to using but directly inside fabric. So I think this makes sense. No, 100%. So, just wanted to shout that out there because honestly, Mike, a lot of people I’m talking to and like a lot of people at the conference. Oh, some people at the conference, they love us. They make their kids listen to it that are in high school on their way to work. So, a few people came asked about the podcast, but

22:24 , at least at Dynamics, man, a lot of database side like heavy usage. There were even some baseball teams or representatives from the major league baseball teams at this conference because they used Dynamics. Interesting, Tommy. Yeah. So, I did not get a job with the Yankees, which I was very upset about. Well, they weren’t they weren’t there because they they run their own they’re on AWS. I guarantee it. [snorts] Are they on AWS? I guarantee they’re on they’re on the coast. So most most of I would say most observationally like when you look at the coasts, east coast and the west

22:56 Coast for whatever reason they’re earlier to adopt technology stacks or whatever that means. So like it feels like the coasts are much more AWS. The Midwest where Tommy and I are is much more Azure based. We feel like there’s a lot I feel there’s a lot more Azure experience andor companies using the Azure in the in the Midwest area. So just I don’t know just maybe it’s a cultural thing, maybe it’s just where it started getting adopted. So my guess would be is Yankees are using AWS as their data store. They were there. Apparently the Yankees guy was there.

23:28 What? So yeah. So I was like, I need a job. I need with [laughter] the Yankees. Oh, work for you for free. Yeah, exactly. Just get me tickets thing. But database man. And so a lot of people are like ask about what capabilities with the normal tool. So I think this is these are updates that I like to see that are I feel on direction on where we’re going. So this makes me very happy. Well, Tommy, maybe next year we should you and I should really take a serious look at the community summit for fabric

24:01 And PowerBI and do some talks around that at the community summit. from our side of things just because I’m me personally as I’m as I’m growing my business I’m getting to the point where like I think I really do need to start using some substantial system to put all the information in. One of the things I’m looking at is Dynamics. I think Dynamics is going to be useful. I’m a Microsoft shop already. So I’m already looking at okay what what products can I pull out from that system? how can I start how can I start leveraging that to store data that I care about and and build

24:32 Little mini automations. I was just talking with Armando on a different podcast about all these automations we do. Anyways, dude, it runs the world. Just so , dynamics runs the world. It’s a big system. It’s a big system. All right, let’s jump into our actual topic today. So, , the topic today is going to be around data modeling and event driven architectures. I’d like to bring in our guest again. So, , welcome Chris. We appreciate you being back again on the podcast. Sorry for a long intro today. It’s been, , we Tommy had a lot to catch up on. So, we’re jumping in today and welcome back.

25:06 Yeah, thanks for having me again. It was a newsworthy week. So, no problem at all. It was all right. Jumping in for our main topic, data modeling in the in the eventdriven architecture. Let’s talk about this. Let’s unpack this a little bit. Chris has also generously supplied us with a learn link here for guidance around eventdriven architectures in general. So, I’ll pull that up as well here and I’ll put this in the chat window. just be some of our talking points for today about like what does this look like? and how we unpack let’s just maybe define an eventdriven architecture maybe for our

25:40 Audience. Maybe Chris start us there and give us your definition of what an event driven architecture looks like and then we’ll take it from there. Yeah. So if you remember back from our last couple of episodes, we’ve been talking a lot about real-time intelligence and fabric and really acting on data in motion and acting on things as they’re occurring. Right. In its simplest form, you can think of an event- driven architecture as really allowing you to unlock this. If you’re familiar with Lambda architectures and how Lambda has worked, think of it like the hot path of a Lambda architecture, right? I’ve got my two paths in Lambda.

26:13 I’ve got hot where my data is coming in consistently and it’s always being generated and it’s something that I need to do something with like in the moment. It’s only powerful in that, , five, 15, 20 minutes after it’s generated. and then it just becomes aggregated and and sent on its merry way. So event- driven architectures allow us to take advantage of that data in the moment when those things occur. I I came up with a a new analogy a couple of months ago and I think it’s, , really fitting. So a great way to think about it is if you think about it like

26:47 You’re driving a car and if you’re listening to it while you’re driving a car right now, right? So imagine if you didn’t know how fast you were going until you got to wherever you were going and then you got a report that said, “Hey, while you were driving, this is how fast you were going.” That would be weird. That would be weird, right? And I in many ways we can think of data in much the same way, right? But it’s very reactive. We get a lot of these different pieces. And so event- driven architectures really allow us to in its simplest form take these things that are occurring in the moment and knowing

27:19 About what those pieces are. Your your car on that dashboard we’re just talking about telling you how fast you’re going is an event driven architecture. Like there’s something that’s reading from the engine. It’s saying here’s how fast we’re currently going and then it’s going to go show it on a digital display. This you can go see it. So that’s that’s not useful telling you what happened five minutes ago. It’s useful to know like in this second as I’m driving down the road what my current speed is, , my my RPMs and anything else, my engine oil temperature, all these other pieces. So, being able to get to these

27:51 Pieces allows us to act on our data much quicker like we’ve talked about, right? So, real-time intelligence really allows us to go there. But when we move to that, we need to start thinking a little bit differently about the way that we’ve been modeling data historically. Right. So when we think about and I’m gonna jump ahead a little bit maybe but yeah go ahead. pre prequest but when we think about a lot of the ways in which we’ve modeled and built data for as long as I’ve been in the industry it was do

28:24 We do we build third normal form and do we go with inman do we go with Kimble and we create star schemas do we create data vaults right and so I I can count on more than one hand the number of conferences where we would have these long drawn out conversations around well I’m going to build a data warehouse should I build my data warehouse on Inman should I build my data warehouse with Kimble Should I build a data vault? What are all the different tools and techniques that are available to me? But when we think about those things, they’re very reactive by their very nature, right? We have to take time. We have to assess. We have to understand

28:55 All the data that’s flowing, where everything’s coming from. And what we know about creating those thesis is they’re useful like u a bag of Legos, right? Of here’s all the Legos that I have and I’m going to go piece them all together. Once people start adding Legos to the set, then you’re like, well, what do I do with this piece? Right now I have a a 2x4 or, , a 1x8. Like, how am I going to fit this into my structure? So, you have to take this assessment and come back and say, okay, how am I going to go remodel my data a lot differently? with event- driven architectures. One of the

29:29 Real key pieces here is that the importance and where you really want to put the the the the pressure or the like the key pillar, right, is we want to be able to act on this data. And so being able to contextualize and being able to create these references and being able to pull these things are really important to be able to say, “Hey, I want to go pull this on this event as it’s occurring. However, I do want to still do that modeling, but maybe I want to do that modeling a little bit later, right? So, it’s like I it’s almost like a deferral, right? Of

30:01 My primary focus on an event- driven architecture is I want to consume the data and I want to keep that data in motion. I might want to contextualize. I might want to enrich it. I might want to do some of these other pieces with it. But when it comes to creating my star schema for my reports or all these other pieces, maybe that’s something that you either generate on the fly with what’s called a materialized view in Event House or it’s something that you just say, hey, I’m really going to focus on using the data in its current form because I really don’t have time to go through that large expensive modeling

30:35 Exercise. I’m going to defer that downstream, right? , and I think that’s a really fundamental piece of event- driven architectures because when we think of processing data in a data lake or in a data warehouse or data lake house or MAR or whichever inventory supply location you want to choose for your your data needs. when it comes to modeling that data, we have to move through all those different phases. So we have to create these stages. We have to curate the data. We have to go do all

31:07 These different refinements. Event- driven architectures are really about consuming that data in the moment. So we create the the the contextualization of the data as we need to and maybe we’re sourcing that from a data warehouse or we’re sourcing that from a data lakehouse or said inventory system of choice and we’re it into our event as that event’s being generated. And so that allows us to really take a lot of those actions on top of it. So there there’s two in my mind there’s two lines of thinking around like when we think about eventdriven architectures I think the default mode and we talked

31:41 About this before which is we go right to like oh it’s an IoT device it’s a manufacturing line it’s something that’s like always streaming data directly to the events it’s it’s a series of events always happening and you’re trying to like find anomalies and look for things that are out of scope but I think that’s what my mind naturally goes to but I think there’s another realm of this that is undertalked about or not communicated as well which is the hey I have a file that shows up in the lakehouse we can be notified when that file appears and then

32:13 Take actions on doing something with that file it’s a it’s an asynchronous action but we do something in a slower pace it’s not as fast it’s not it’s not like I’m streaming data from an IoT device it’s more about things are happening inside a data system and we’re able to listen to particular events. I I liken this. I think we’ve talked about this before, which is your Ring doorbell or your your your video camera. That’s a doorbell, right? A lot of things there’s video all the time 24/7 getting streamed through your camera. I

32:45 Don’t really care about most of the time of what’s happening, but I care when that Amazon driver shows up and drops the package, right? I want to go get the package and bring it in the house. So, that is like a specific event that I’m interested in. And when that occurs, that’s when I want to take action. And so I look at this going there’s there’s two worlds to this. And what is your reaction to that to that, Chris? Is that your impression too? Do most people fall into the IoT realm when we say event driven? And do you think we need to emphasize a bit more of the event driven

33:17 Like the asynchronous messaging that we can do things with? I I completely agree. I call it the Ferrari effect, right? Yeah. when and to your point when we think of event driven and we think of events that are being generated we typically think of manufacturing or your car driving or whatever right there’s tons of tons of use cases for that right I call those Ferraris right those are all things where by their very nature they’re generating data every second sometimes even at the subsecond level and so we want to be able to pull

33:48 This and see these things as those are happening those introduce their own different set of challenges with event- driven architectures because we start to introduce things and one of the things I joke about quite frequently is when you’re building a system in real time intelligence or in another sol system around some type of event- driven architecture that does is handling that you’re processing that data so fast that as a data team you’re actually introducing a problem that you’ve never had before which is you’re creating and absorbing what’s called jitter from

34:22 Those things as they’re being generated. So false readings, right? Or something that’s happening, right? whatever it could be, and it could be lots of different things, but essentially you find yourself in a situation where you physically have to hold the data for five or 10 minutes or whatever the right number is. It’s different for everybody and it’s different for every scenario and each implementation of whatever that system is. But yeah, you have to wait for a few minutes for the data to stabilize enough for that to be actually valuable to a reporting or for an event- driven scenario. Right? It’s not always about like hey I’m

34:55 Generating the data so fast and I just want to consume it within that second there are cases and we have many customers who say I’m pulling this data in real time but I don’t want to report on it in real time I want to hold it in five minutes or I want to hold it for 10 minutes and then I want to go do something with it after the fact right and then you have to to your comment then you have a whole another set of data which is I’m not building a Ferrari my data doesn’t generate that fast right I have a I have a bicycle right? or I’m walking, right? Or whatever that is, right?

35:27 Don’t punish me because I have a bicycle and you have a Ferrari, right? We’re both still valuable and we both still generate what we need to do, right? It’s just, , you run around like a you run around like a Ferrari and I prefer to take a bike every day, right? [laughter] I take my bike as I will also I’ll also argue too the price to buy a Ferrari is much more than a pair of sneakers or even a nice bike is much cheaper too. So I there’s also this is one of the things that I like to communicate with customers and I think Tommy you do the same thing as well is like the more real

35:59 Time you go the more you’re going to need to be comfortable with like look this is going to increase you need to have a machine on a computer needs to be monitoring the the the higher the volume the so there’s the three V’s of data volume velocity and variety we’re probably talking about the volume and velocity of data in this realm the higher those go the more you’re going to need to be capable, the more data you’re going to produce on disk, the larger the number of computers you’re going to need to run to handle all those events coming in. If you’re doing windowing and I think what you’re talking about with

36:31 Like the five minute window or holding on to the events and then making some decisions based on that five minute window of time, the windowing that requires some computer to like store up those events and hang on to them and then aggregate and spit that out. So there is a I think a cost implication here as well and I think a lot of people my opinion here and I’ be curious Tommy your opinion on this one as well people shy away from real time or event driven things because they assume IoT high volume they think they’re always getting a Ferrari but in

37:05 Reality well thought out or thinking about the architecture you could you could still do event driven things but they don’t have to always be at that high price point. It could be a bicycle or it could be your video doorbell camera. It could be all these different pieces. I think my one of my favorite features, if not my favorite feature in all of real-time intelligence and fabric is a feature called continuous ingestion into events. And it’s a it it’s unfortunately I don’t want to call it a secret because I don’t think it should be a secret but essentially what we do

37:38 Is we subscribe to a storage account in Azure. so anything in ADLS and as new data gets added to that we have just subscribed to the storage account. So the data is automatically picked up loaded into the event house no pipelines no notebook there’s nothing I need to do right I just configure and I just say hey I’m subscribing to the storage account and as new stuff shows up in that storage account just automatically load it. a demo I have that I show for for customers and at conferences and things is I build a continuous ingestion

38:10 Pipeline and then I take an update policy on top and then I have a PowerBI report that sits on top of that in a semantic model and then I as before I go into the demo I talk a little bit about okay so imagine you’re a vendor or so and so drops you a file and it’s a JSON file or a CSV file or a pipe separated file or tab or whatever right anything that you want You let your dreams run wild, right? Whatever whatever file you get, right? Yeah. And then imagine all the steps that you go through to process and transform that data. So before you make

38:42 It ready and available in a PowerBI report, right? It could be going to, , storage and then it’s going to another storage account, then you’re applying something else, then you’re moving it downstream and downstream in all these different places. And then I switch over to events and I show continuous ingestion where we are just loading that data into event. And we’ve created this subscription. And so what the event house will do within RTI is as soon as new data gets loaded, it just picks the data up, loads it, adds it into the event house. And so if I have a PowerBI report on top of it, instead of it going through that traditional

39:14 Process, that report can be updated within a few seconds of that file actually getting added to the storage account. So it’s much quicker, but you also have a very powerful engine on top of it because of what the EventHouse engine is. we we recently created a a new endpoint on top of lakehouse called event house endpoint for lakehouse u and if you’re you’re not familiar with it but essentially for the same thing right okay we want to be able to make it so your data isn’t always a Ferrari it doesn’t always have to be that Ferrari it can be these different

39:46 Things but the the key premise is that as my data becomes available I immediately want to go take whatever action that is right. I want whatever downstream system to go kick off. I don’t want to have to worry about, , creating artificial delays in my data by scheduling things. I just want to move it all the way through. So, it’s a it’s an interesting paradigm shift, but as you said, there’s a lot more to it than just Ferraris. There lots of different price points. Go ahead, Tommy.

40:17 Let’s dive in then to the semantic model side of this, too. And [clears throat] I I’m I’m really wanting to get your feedback on this is where I think you’re going to get a lot of the PowerBI only people to be a little more willing to try this out because I think there’s a lot of conversations that I’ve seen is like well if you use real time then there goes your semantic models which is a misconceptions because no matter what a semantic model used to be always

40:52 In a sense more or less cold because it was imported data coming in the real time is they’re completely different data for different different purposes but we know that we can build a semantic model off of those dynamic data feeds. So how does the semantic model come into play with real time with the event driven architecture? Yeah, it’s a great question. So I think that a great place to start is in a conversation I have quite regularly

41:27 Customers who have been trying to implement PowerBI models by essentially pushing data. , so we used to have a feature called streaming data sets in PowerBI if you guys were familiar with it and we recently announced that it was deprecated which by the way I’m extremely disappointed by this feature and I’m going to I’m going to complain here for those who are at Microsoft listening. I have a number of web hooks that I’m trying to consume across various platforms and internet things. There’s nowhere to put them. like you can’t I’m

41:59 Trying to put a web hook into fabric where the web hook just sends data and fabric can consume it. I liked the streaming data sets in PowerBI cuz it was like a web hook. I could just have a URL define a schema and just send data to it. Now I don’t have anything and there’s everything that you want to send data into for fabric requires like authorization and sometimes web hooks don’t give you that. So that’s Chris I understand your point there. I want to just complain here for just a hot second like It’s missing. That’s a great point. I It’s killing me too, Mike.

42:31 We need a web hook that like Okay, Microsoft Teams, I don’t know who’s listening to, , Satia, can you please can you please work with the Microsoft team and give us at least somewhere where we can consume an open endpoint, I don’t know, user data functions some somewhere that is unauthenticated and I can just send data to it. I need it needs to be a URL that I can push a a JSON body to that I can go get into fabric. It’s missing and right now I have to go build. Right now I’m I’m having to go back to Azure to build it which is not wrong. It’s just I don’t want to do that. I’m lazy. It’s annoying. I I totally hear you.

43:03 Okay. Sorry. You might unpause. You might see the huge smile on my face because I’m listening and and I know it’s it’s a conversation that we’ve had many times. Awesome. All right. You’re saying deprecated feature streaming analy streaming to PowerBI data sets is now gone. No, right to streaming to data sets. And when you think about why, like there was a lot of overhead that happened within a semantic model when you’re trying to create a streaming data set into those different pieces. It’s far more efficient to keep those engines and either direct query or whatever. But when we create direct query, there’s a lot of

43:35 Trade-offs and considerations for that. It seems like far too often it’s either, , I I can have direct query or I can have import, but I couldn’t have both, right? It was I either got one or the other. And when you went to a direct query model, it was you put all the pressure back on the the source database. And if the source database isn’t really prepared for that, then you wind up into all kinds of performance challenges and led down a less than ideal path. But now because we have all these different features available to us in fabric and we can stream this data into the right spot

44:07 Within RTI, it’s really easy for us to start to blend those things together. So we can say, hey, I’ve got my streaming data as it’s coming in here and I’m loading this into my event house. I have a semantic model that’s sitting on top that’s maybe pulling and being able to query that stream in real time that I’m just joining on top of. But I’m also maybe sourcing the the import portion of my semantic model from somewhere else. Maybe it’s coming from direct lake or maybe it’s coming from somewhere else where I can really start to bring and bridge these experiences together. you can also try if depending on what your data set is and depending on

44:39 What you’re doing just load the data into event house and just spin direct query on top of it. custoto is a diff much different engine. the only challenge with with custoto and the the caution is be careful of your joints, right? So when we think of a traditional relational system, we have a tendency to create a lot of different joints, right? We want to have our our star schema with our fact and our dimensions all around it and surrounding all these different pieces. when we’re direct querying an engine like a stream or like hot data, we might not want to go through all that modeling

45:13 Exercise, especially if we’re going to go into a direct query against event house or whatever that might look like. We might just want one larger table where we can avoid joins or or do those different pieces. And so there’s there’s trade-offs and there’s different things. The the short answer is you can absolutely use semantic models on top of your streaming data. How you want to implement it is very much up to you and it’s up to your scenario and what you’re looking to go do. Do I want all my data just hot and I don’t want to have to worry about it and I just want to run a PowerBI semantic model in direct query. Do I want to keep

45:46 Maybe three days of data hot and then put everything off into a lakehouse or something in a cold path store where I can blend those different things together? the semantic model is really your glue. It’s your bridge that allows you to bring all these different things together under this umbrella, right? So when and I think going to what you were talking about Tommy last week with your session around data agents. When you take a data agent and you put a data agent on top of a stream of data that might be coming from a ven house and then you might point it to a

46:18 Semantic model or you might point it to a lakehouse or you might be pulling that data from somewhere else. Trying to get the agent to bridge those different things across all those different systems is challenging. Right? There’s not a whole lot of great ways today to say, , , customer record here equals customer record over there and all these different pieces. You have to be extremely explicit in the instructions to the agent to tell it to go do that. But if you can use that semantic model as your bridge, as your glue, then you can bring that semantic layer together, apply that contextualization on top there, and then point your data agent to that. So you

46:50 Still have that entry point. You can still unlock all those fabric goodies, if you will, right? and then create your semantic model that’s really tying all these different artifacts and all these different pieces together. So, what are some of the the considerations though? Because obviously there’s a lot more to plan for like you said how much you want to be hot, how much you want to be cold. what are some of the tradeoffs and what are the and again life of consultant the two favorite words are it depends but on the

47:25 Average on the baseline what would be considered like okay I’m going to start with the smantic model with real time but I have no idea what to do so how much data should I do I don’t want to I don’t want to worry about cost I obviously want to make sure it works but what are some of the baselines for you to back and that generally work for most circumstances. So, I’m going to preface that by saying I’ve been in consulting for a very long time, Tommy. So, for me to give any answer other than it depends, it’s going

47:57 To be really really challenging. [laughter] Take a take a swing. Take a swing. But I think the the consideration and the trade-off and you’re also going to get different answers, right? I think based on who you talk to. I’ll tell you if you you look at event house and depending on what your data volume is if you just want to bring your data into event house and just cache some of it and keep the rest of it retained then maybe all you need is direct query right and just direct query on top of that event house and it’s going to serve your need for what you’re

48:29 Looking for. If you are doing a lot of transformations where you want to bring that data into maybe a cold path store and then join it with your reference data and your dimension data and all the other pieces and go through all your dimensional modeling exercises for what you’re going for. then it makes more sense to go point that to Lakehouse, right? Or bridge those two things together, right? say, I’m I’m going to keep this data hot for only as long as I need it because this is really what I need to go be able to answer my requirements and then I want to create

49:01 My model and my my my proper data model here downstream based on whatever it is that I’m looking piece going back to what we’re talking about before. I think it all comes down to where do you really want that modeling to occur, right? Remember with an event- driven architecture, we’re consumed we’re concerned with consuming the data in motion, right? That’s our goal. And so I think the short answer without trying to get too it depends is the as long as you need it for your specific scenario, right? If you need

49:35 That for if if you need that data hot for 12 hours, then that’s fine. If you need that hot data for a data hot for a week, then that’s okay, too, right? It’s it’s all about those individual pieces. I think some of this talks down to like it’s it’s all about the requirements of the business, right? What is it that’s important for those particular pieces and what the what it is that you’re looking for? I I I think I want to go after some like I think there’s like a when I look at beginner users about talking about event driven or pushing users towards this world of real time experiences. I think what I I want people to clearly

50:10 Understand at least as beginners when I look at it and again Chris Tommy fill me in here what you feel like is also a beginner user challenge. I think it’s just being able to get your head around what is the producer of data and and identifying in your design use case what are we talking about right are we talking about consuming things from blob storage are we talking about a device that’s sending information are we talking about like what what is the thing that is producing the data and then on the other side I use this analogy of like the catcher mittm is going to like the baseball analogy here

50:42 Right so whenever data is being produced something has to catch it like something has to go grab wherever the data is coming from from where whatever system whatever that may be and then in once it’s caught once the data is like been able to be identified then you have to identify okay does all the data coming from that system and this is where I guess maybe the concept of like a channel or like a topic comes into place so you need to define what’s producing it what’s catching it and in that stream of information or stream of data what is

51:15 The topic or channel that you’re you’re you’re using to do the things. And I think Chris, what you’re describing to me right now, I’m having an aha moment here, which is when I think about data and data modeling, I’m thinking dimensions, facts, star schema things, all all the data that’s basically applied into like the model. And what you’re saying in real time here is a a bit of a shift in this is as the data comes in, you may need to append those dimensional properties to that data as it comes in as part of the event stream itself. And so instead of thinking about

51:48 A star model, yes, you still need those things defined, but you’re going to go look up or grab or enrich that data as needed. That way as the data is coming in you can then have the final result table output data whatever you need on the other side of this that you can be it can be actionable right away. Is that a good mental model here? That’s a great way to put it like I’m really glad you said it that way because it’s one of the things I talk about all the time. Okay. So when we think of that we we think of data it runs at all these different

52:19 Speeds right we’re talking about Ferraris and bicycles earlier right? So yeah, if your event data is your Ferrari and it’s coming in at some speed, right? And maybe it’s coming from to your point, maybe it’s coming from an event hub or maybe it’s coming from Kafka or maybe it’s coming from a CSV or maybe it’s coming from JSON or maybe it’s coming from parquet or could be anything, right? Or it’s coming from some database, right? so you’ve got all these different places where you’re building these events and you’re consuming these events. For really a large portion of time, our goal has been, well, how do we

52:51 Process all of our data at the same speed, right? So, how do we make the bicycle go as fast as the Ferrari, right? I need to refresh all of my data, right? And I need to make those events available for the business as fast as I can make them available. So, Chris, start pedaling your bicycle really, really hard because you got to keep up with his story. [laughter] It’s gonna be Oh, no. You better do some training, right? So start start pedaling, right? But to your point, when we look at event-driven architectures, it’s the opposite. It’s about, hey, I have my Ferrari, my

53:26 Data is moving at that speed, or I want to be able to react to that data at that type of speed, right? And maybe it’s not being generated that fast. Maybe it’s coming from a vendor who’s dropping a file in Azure storage every couple of hours or Yes. Or once a day even like that. That’s a common use case. Yeah. or I have a backend system that’s refreshing the SQL database. I need something to say I’m done refreshing the back end. Now when that is complete then do a triggered event to go load something downstream into fabric now. So even then it’s like I don’t want to delay. I don’t

53:59 Want to wait or the scheduling of things gets really off because you don’t want to wait 2 hours, , cuz the the process may take 15 minutes today, it may take two hours tomorrow. Like sometimes the process varies and so having event- driven things, it can just take along without having to like wait. The number of engagements I’ve been pulled into where before I went down the event driven path of it takes me seven hours to refresh my reports every day, right? Yeah. Why? Great. Because it’s waiting then then we go look the the vendor

54:32 Drops the file at 11 p.m. and then okay well it goes to system A at midnight and it goes to system B at 1:00 a.m. and it goes to system C at 2 a.m. Right. Yep. Even though it only takes like 5 minutes or 10 minutes to process the data. Yes. We build all these buffers in because like you said today it takes five minutes, tomorrow it takes 55 minutes or it could be anywhere in between that. So in order to get some type of guarantee of I avoid conflicts I just wind up scheduling with big buffers in the middle. Right? Then we lose, right? Then we lose and as our as organizations have become more and more geographically

55:05 Dispersed, this has become a bigger and bigger problem. Right? Because now we can’t always say, “Oh, well, data is available at 8 a.m. Eastern. It’s fine. Like, you’re good. It’ll be available at 8, right? Because what about all my European colleagues who they start working at 2 am, right? I don’t want to start I don’t want to guarantee that as my SLA, right? or all my colleagues in India or all my colleagues in some other country, right? It could be any country in the world, but I got all these countries and I really need to make this data available to them as these things are occurring, right? So rather than to create and build these built-in latencies and these built-in delays for

55:37 Doing all these schedules, I can get to the point where I can say, hey, I’m generating I’m pulling these events as these events are occurring. Like you said, I’m just triggering the next stage in the process and just saying, hey, I want to keep my data in motion. But to your comment a minute ago, instead of trying to say, I’m going to process all my data and I’m going to try to process it as quick as I possibly can. Hey, I’m going to bring this data through and I’m just going to contextualize that stream. I’m going to take the information from whatever I’ve curated for my reference data or my dimensional data or whatever. I think fabric’s fabric’s really

56:11 Interesting to me because we’ve taken all these different worlds and we’ve really brought them together and now we have three words that all mean the same thing. , so it’s going to take us some time as an industry to be like, well, is it an event again? Is is it an event? Is it a fact? Or is it a transaction? I don’t know. Well, let’s let’s let’s debate. It’s the new [laughter] it’s the new Kimble versus Cinnamon debate. I can see it happening. Yes. Yes. But, , we take all these different pieces and we say, okay, we’re going to contextualize the stream of data as these things are occurring, and this is what I want to react on. this is what I want to have happen in real time

56:44 Or this is what I want to make available to my business as quick as those things are being generated and then [clears throat] I want to go apply and say okay now I’m going to go take all these different things and I’m going to go send it down my cold path I’m going to go do all these different pieces but you’re really able to like you said flip the script a little bit one thing I tell customers quite regularly is it’s always easier to bring cold data hot than it is to try to make your hot data go as as or make your hot data in cold, right? Interesting. So when when we look at something like real-time intelligence, being able to

57:17 Shortcut those things in is like from your lake house or from your warehouse or wherever your mirror database, wherever that data is coming in or however it’s landing in the fabric, right? Yep. And then being able to short that into event house where then you can go do that contextualization, you can do that reference, you can do those joins is much easier than trying to say, okay, I’ve got this stream of data and I’m going to go land it in a data lakeink and then I want to just go process it as fast as I can with all my schedules and everything else that I’ve had to go deal with. Interesting. This is good. I think we’re almost at time here. Tommy, any any kind

57:50 Of wrapping thoughts or thoughts around semantic models, the event stream, bringing in key events and enriching the event stream for actionable data? So, [snorts] at what point does KQL become the new DAX? and in terms of from I just want to know from a developer point of view there’s obviously the K KQL language and if I’m still building semantic models at what point do I need to change careers or does someone listening where they think that they can actually

58:23 Provide the value where there’s and I think we’ve talked about this a lot already with our real time conversations is real time is great but where the best applications of But if I want to get started today, if I’m listening, I’m running, I’m I’m biking, and I’m like, I what, I’m going to begin to get my feet wet with semantic models and the event driven architecture, what language do I need to know? What is some of the those skills that I need to know? And how do I make sure I don’t go off too many rabbit holes in terms of all the other things that you need that

58:56 Go into real time? So, it’s a great question, Tommy. I think first semantic models are very powerful because they allow us to apply all that semantic meaning across our business right so if it’s hey I’ve been using DAX for a really long time I’m really skilled at it I want to continue to do those I I don’t think that’s changing what I think is changing is instead of saying I have a toolbox and the only thing in it is my hammer now I have an entire toolbox that’s available to me

59:29 And so do I need a hammer all the time or can I use a screwdriver or can I use a drill or like what are all the different pieces, right? Do I need a sander? Right? What are all the different things that I might need that allow me to do those pieces? KQL is just another tool within that toolbox that allows me to go act on that data. when it comes to dealing with data streaming and data in motion and things that are happening in real time. That’s where KQL is really powerful because it also allows me to do a lot of these different pieces and look at these make these transformations whether

60:00 I’m creating an update policy or whether I’m trying to go create something. But being able to analyze that data very deeply regardless of my data volume allows me to become an extremely effective analyst whether it’s security data whether it’s business data whatever it is. So I think it’s start by understanding the use case for the thing that you want to solve and then how can KQL help you do those different pieces. There’s a lot of functionality in the KQL language, a ton of functionality. And so, , there is a what’s called the

60:33 Cuso Detective Agency if you really want to get started and dive into KQL as a language, , to understand all those different pieces. , the the custo detective agency and I don’t know if we can grab a link. We could probably put it in the chat place to start. , but that’s always a it’s a gamified experience help you get started and help you understand real-time intelligence and how KQL works. And then on top of that, right, how do you leverage KQL to go accomplish these big data constructs? all these different pieces of hey, I want to go dive into my

61:06 Data really deeply in all these different pieces. But I I never start out with saying like everything we’ve ever done, we should just drop and go start over. I think it’s more of a blend, right? It’s just a how do we start to bring that in and say, okay, like I know I can go do this. something, , I know many customers love to use KQL for is breaking out JSON, for example, because it’s extremely easy in KQL. , so I can use KQL to break out my JSON. And we have customers that’s what they use it for, right? They take a JSON file that gets loaded into a storage account, drop it into events,

61:38 Parse it, and then they go move it into a lakehouse. So it’s it’s like a hot bath because they’re able to go report and act on that data in real time for whatever they need, but they’re also able to use it for that scenario for what it is that’s working for them. But there’s tons of different use cases and tons of scenarios. So rather than starting with everything, I’d say, , start with whatever it is you want to go solve. I think when I when I look at the the custom, custoto is a database. It’s like a storage system and it’s a query system that’s independent of

62:09 Everything else that we’ve been using. I think you if we look at like the landscape of like what what PowerBI is, we have semantic models that write that write DAX. You have the Spark engine that that does its own way of processing data. It uses storage storage accounts basically blob storage behind the scenes. And then you have like custo which is doing its own thing. It’s a storage system and it’s a query language on top of it. I believe, correct me if I’m wrong here, Chris. Gusto was made for PowerBI. It was made for the internal analytics of Microsoft to to build like

62:44 Reporting at scale for all the events that are coming from PowerBI users across the system. I thought that was like where Kusta was invented from. it was actually a reporting system for PowerBI to see what was going on there. Is that true or is that just something I made up in my head? No, you didn’t make it up in your head. I I don’t know if it came specifically for PowerBI alone, but I can tell you that it came from Microsoft as a whole need to analyze all of the telemetry and all the data being generated from all of the services okay

63:18 And when that when custo was first created there was no tool that really allowed us to do that at scale. And so now fast forward, , over 10 years later, every service in Microsoft, PowerBI, M365, Dynamics, , every Azure service, every service logs their their telemetry to custo. And by logging all that telemetry into Custo, then they’re able to go get all these real-time insights for what it is that they’re looking for. You can still create these star schemas. You can create some of these different

63:50 Things, but it really changes and it’s really quite it’s quite powerful with what you can do because the fundamentals of the engine work much different. They’re not it’s not a traditional SQL engine. It’s always the first thing I tell everyone. Not a traditional SQL engine. It’s much different. Very purpose built for that thing. It was purpose-built for that thing, but it also means it’s extremely powerful at what it does. Awesome. Well, that being said, let’s go ahead and quickly wrap it here. Thank you all for listening and hanging out talking to us about real time semantic modeling event driven pieces. I I think maybe my main takeaway here is think

64:24 About event driven architectures not in just a Ferrari all the time but think about the bicycle use cases of reducing lead time to processing and loading your data. I think that’s one that’s a big takeaway for me. All right, we appreciate your time here. We know you could be spending your time on many other things. Thank you for jumping in and listening to this podcast and learning more about event driven architectures. We hope you found something valuable. Chris, thank you so much for joining us today and giving us your expertise as a previous consultant and now all your time in the Microsoft CAT team, telling us what you’re seeing and how you’re

64:57 Communicating this to customers. , this is invaluable to our audience. We appreciate you, Tommy. Where else can you find the podcast? You can find us on Apple, Spotify, or wherever you get your podcast. Make sure to subscribe and leave a rating. It helps us out a ton. and share with a friend since we do this for free. Do you have a question, idea, or a topic that you want us to talk about in a future episode? Well, head over to powerbi.tips/mpodcast. Leave your name and a great question. And finally, join us live every Tuesday and Thursday, 7:30 a.m. Central, and

65:30 Join the conversation on all of PowerBI tips social media channels. Thanks, Chris, and we’ll see youall next time. Thanks. down.

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.