Planning Your Capacity – Ep. 457

Capacity planning is one of the most asked-about topics in Fabric. Mike and Tommy break down the available tools, the capacity estimator, and practical strategies for sizing your Fabric SKU without over-provisioning or getting throttled.

News & Announcements

• Semantic Model Refresh Templates in Power BI (Preview) — New templates for managing semantic model refresh patterns.

• Introducing the Translytical Task Flow Contest — Microsoft launches a community contest around Translytical Task Flows.

• What’s New in Fabric Warehouse — August 2025 — Monthly Fabric Warehouse updates including performance and feature improvements.

Main Discussion: Capacity Planning

Why It Matters

Fabric capacity directly affects:

• Performance — Queries, refreshes, and pipelines compete for the same capacity units
• Cost — Over-provisioning wastes money; under-provisioning causes throttling
• User experience — Slow reports and failed refreshes erode trust

The Tools

Microsoft provides several resources:

• Capacity Planning Overview — The starting point for understanding how capacity works
• Load Assessment Planning Tool — Helps estimate load based on usage patterns
• Plan Capacity — Step-by-step guidance
• Optimize Capacity — Strategies for getting more from your current SKU
• Fabric Capacity Estimator — Interactive calculator for sizing

Practical Approach

Mike and Tommy’s recommendations:

1. Start with the estimator — Get a baseline SKU recommendation
2. Monitor actual usage — Use capacity metrics to understand real consumption
3. Identify peak workloads — Refreshes, queries, and pipelines often peak at different times
4. Optimize before scaling — Incremental refresh, query reduction, and scheduling can reduce capacity needs
5. Plan for growth — Build headroom for new workloads and users

Common Mistakes

• Right-sizing once and forgetting — Usage changes; revisit quarterly
• Ignoring burst patterns — Smoothing can hide throttling during peak hours
• Not separating dev/prod — Development workloads consuming production capacity
• Treating all workloads equally — A lakehouse notebook and a Power BI report consume capacity very differently

Looking Forward

Capacity planning becomes more critical as organizations expand their Fabric footprint beyond BI into data engineering, science, and real-time intelligence. Understanding capacity consumption patterns across workloads is essential for cost management.

Episode Transcript

Full verbatim transcript — click any timestamp to jump to that moment:

0:00 Hello and welcome back to the explicit measures. podcast with Tommy and Mike. Good morning, everyone. Welcome back.

0:33 Good morning, Mike. How you doing? I’m doing well. I’m a little tired these days. I just been what for whatever reason, it’s been busy. Last week was like a 4 day week work week and I just felt like I was just going going. It feels like things are ramping up. Although this is the time in our area, kids are starting to go back to school. So, it’s just a new pattern for life and it just messes with things a little bit. We have Do you have the curriculum night yet? The what? Curriculum night. School night. Oh, yes. Curriculum. Yes, the curriculum night. ,

1:09 Maybe. , my kids are older now. They’re like in high school. I have two in high school now. So, like we don’t really do like curriculum nights as much. I guess it would be more like, , meet your teachers, say hello, , and then you let it go from there. Yeah, we had to do ours and man, I wish I could send a single representative. It would be so much easier. My wife insisted I I came. Yeah. Like one parent give me the information. And I realized too as soon as we sat in

1:41 Class, I wouldn’t be able to survive. Oh, yes. Did did they one of the things that are funny, my wife was a teacher for a number of years and she would look at the parents and then observe the kids and be like, “Oh yeah, yep. This kid is related.” Like I could see this is why this is why this kid acts this way, ? Oh, and I think I honestly I think they could the teacher could go in there, Tommy, when they see you and they see you sit down in that desk and you can’t sit still. You’re just

2:12 Looking around. I you’re like a million miles an hour and they’re like, “Yep, yep. This is Tommy’s child. I can I can The Pulia children, we can see exactly who their dad is. They know right away.” Oh, she the boys, right? Yeah. I I , I was standing in the back and I’m like getting completely antsy. I’m writing in my do daughter’s notebook just scribbles. I’m like, “Oh, yes. This back in the day, my friend.” Amazing. All right, let’s get into our what our topic is today and then

2:44 We’ll do the news real quick. Our topic today is around planning your capacity, which is great timing because I believe an article was just posted recently. Kim Manis just shared it. There’s a whole section around the capacity planning guide, strategic planning, plan your deployment, scale for decentralized analytics, scale for centralized analytics, manage growth and governance. So the timing of this couldn’t be better about talking about capacity things. articles are already dropping and I think this is actually really relevant because organizations

3:16 Need to figure out the best way to organize their their company and I think Alex P told me something recently that I think this is very relevant with is people are trying to what they call minmax their capacity right the maximum amount of performance with the minimum amount of cost right min right meaning the co the price and maximizing the usage And I 100% agree. Like yeah, I don’t see why you wouldn’t want to do this because this is basically maximizing your investment in Microsoft. So this is why

3:49 People are trying loading data different ways, trying to figure out, , do do I use notebooks? Do I use pipelines? Like what’s the most efficient way of getting something done? And I think that’s that’s fair. That’s that’s the idea of a data platform. When you give us a Swiss Army knife, we’re going to figure out which part of the tool is the best one to use. So anyways, that’s the main topic today. Let’s get into some news items. Tommy, tell us about what’s coming up here for live events. Well, the major live event is the Chicago PowerBI Microsoft Fabric user group is back in person on October 2nd at 3:00.

4:26 We’re meeting downtown at the AON building. This is where we normally meet the Microsoft office just north of Millennium Park. And what we’re doing is a crash course on fabric. So for a lot of you, you probably seen the videos, , we’ve talked about it, but you maybe your organization doesn’t have it. Maybe you just want to play in a tenant that the people who are driving don’t care if they break something and maybe you want to try something out for themselves. So we’re going to really take advantage of being in person. me and Mike are going to deliver live in

4:59 Person. going through all the things in fabric, getting questions, trying things out, what people want to see. That’s probably be my tenant. And we’re just going to really allow that time together. There will be food, networking, all that jazz. And also, it’s at 300 p.m. Not our normally 6:00 pm in the evening. Correct. So, just jumping in there about that one. So Tommy, quickly, what what things what parts of fabric do you think we’re going to be touching?

5:30 We’re probably going to be touching the lakehouse, the analytics endpoint. We’ll probably do some things in notebooks. , what other areas, Tommy, would you like to touch? I Yeah, version of this one. So, we probably won’t hit everything, but anything else that you’re thinking of, Tommy? There’s a I have a good feeling people are going to be asking about notebooks in warehouses and also, can you we look at the SQL database? How can we connect to it? SQL database. Yes, that’s another good one. Excellent. Okay, great. We’ll go through those things. The idea is show up, bring your own computer. Tommy will have his PowerBI tenant available

6:03 For you and you’ll play around with some things in there and we’ll start from there. All right, that being said, another event that we would like to push out here is Pittsburgh SQL Saturday. there’ll be another event coming here. I’ll put the link here in the description. So, for both of these for the Chicago event, here is the link inside the description. and it’s also in the description of this video as well. but if you’d also like to v visit Pitchbird SQL Saturday, that event is also coming up and the more details are in the link here in the description as

6:35 Well. All right, Tommy. any other news items that you have? I don’t think we have anything else. No, I don’t have any news items. We talked about the refresh templates and we talked about the August update. So, I don’t have anything else. Let’s move on into our main topic. We have a welcome caller of today. we appreciate you being with us. Stephanie Bruno is with us today. hey, good morning. Thanks for thanks for picking up the call. No problem. Stephanie is the one running the Pittsburgh SQL Saturday

7:07 Event and that will be on October 18th. cor if I’m correct. Is that right? Yes, that’s right. Thank you for that shout out. We would love to meet some new friends there in Pittsburgh. So come join us there. And also, if you’re excited about what Tommy and Mike were just talking about with fabric, we also have a free pre-day for a fabric analyst in a day training. So, if you go to that registration, join us, you can also sign up for that. Wonderful. Very nice. All right, let’s let’s unpack the main topic here. Tommy, I’m going to

7:39 Kick this one over to you like, , where we start, where we take this thing. there is and again want to preface this with capacity planning. This is more around a fabric experience, right? So I want to just maybe level set a little bit here. There is per user licensing. This is something a lot of people get tripped up in what licensing is involved with different fabric experiences or even just PowerBI in general. So there’s per user licensing, pro and premium per user. the number of people you have is how much you pay. But

8:13 At larger companies, that scale doesn’t doesn’t really fit. It’s it’s too expensive for companies to buy single individual licenses for every user. So when you get to larger scale things or you’re trying to do special embedded or ISV related things, there’s different capacities you could buy. And so there’s u a lot of strategy that can go around how do you build that capacity? What does that look like? So today’s conversation will really be around capacity planning. Tommy, take it from there. What other things should we talk about? Yeah, again we’re making this migration from our normal PowerBI

8:46 Licensing your PowerBI premium per user which still exists but PowerBI premium gone more or less unless you still have it you can get grandfathered in but it’s really transforming into your fabric F skw licensing which includes all your the A embedding which would be a and your P premium but the difference here is now any organization who wants to sign up for PowerBI premium gets fabric. Now they don’t have to use anything in

9:18 Fabric. They don’t have to use notebooks. They don’t have to use any lakeouses, , spark, whatever. They can just keep doing semantic models and data flows gen one. Yep. However, and I believe too, even the the capacities themselves are the same sizes as the premium the capacity units if I am I slightly. Yes, Tommy. They’ve aligned the pricing so it matches. It’s within like 50 bucks of whatever the original pricing was, right? So, yeah.

9:50 , I think that the key thing to note here is we’re moving from a per user license, which basically means a user has access with certain limits, right? A prouser can only have a database or a semantic model size of 1 GB and lower, right? If you have if you have smaller to mediumsiz data sets, a prouser might be just fine. , but when you get into larger data set sizes, you have to start thinking about, okay, then upgrade my user to a premium per user, which is a premium capacity. it unlocks some of the threshold for that user. But when we

10:23 Start talking about capacity planning, this is where we move away from per user licensing to dedicated machines or you’re buying dedicated capacity from Microsoft. Again, this is very boring because the two licensing models are very different, right? It’s userbased with limitations, but then you move over to premium and then with premium you get like all the bells and whistles, but you now have different restrictions. Anyways, does that make sense what I’m describing? It’s confusing. It gets confusing. So, and I and I think for a lot starting

10:56 Off is how do you guys see capacity planning being a lot different in this the fabric in terms of I think a lot of organizations show like well we have a lot of data coming through but this is really the first time we actually have to think about the different products that are also going to be a big part of that. So for for you Mike for for you Stephanie have you guys taken a look at how you’re actually planning for capacities or no the F2 is fine or my normal one’s fine but has it changed because of the products and the

11:28 Features. Well I think one thing to also point out is there is no per user licensing for fabric. So when you’re talking per user that’s just PowerBI world. So for a really good point, we do need an actual F skill. And again, we’re going to add more insult to injury here, right? Because if you’re creating things in a workspace that has fabric, like if you’re the content creator, you still need a pro license. So like even though you’re getting fabric as a capacity, if you’re the one showing up to the workspace and

12:01 Using the workspace, unless you’re under an F64 to use powerbi.com, you still need a pro license. So like there’s like a but only for PowerBI items not for fabric items. So you don’t need it if you’re just creating a notebook or a pipeline, right? So the only time those per user licenses come into play is for PowerBI artifacts. And and this is why this conversation is so important because there’s like all these little like gotchas or nuances of things you got to know. You got to know all these little details of like every

12:33 Little thing here to make everything work. Yes, 100% agree. Okay. So, one other caveat I guess I’ll throw in here. What if you’re an organization with an E5? An E5 license is a Microsoft license and every user in your organization gets an automatic PowerBI Pro license with an E5. So, like there’s there’s all these different ways you can get into PowerBI, right? If you have an organization with E5, you get PowerBI Pro by default. It’s just something that comes with your licensing. You can go buy PowerBI Pro license separately or you can go buy fabric capacities that

13:06 Let you do fabric related things, but if you’re going to go work in the workspace, you need an F a license, a pro license to get in there. Anyways, lots of things to think about. We just started it off complicated right from the beginning. We’re trying to give you all the options, right? So, Tommy, you asked about where where do I recommend and where do I start doing things, right? So, I think fabric is incredibly useful for the data engineering space, right? When you’re trying to bring data in from semistructured or structured tables somewhere else, I really like

13:37 Fabric. I’m more my bent is I really like the lakehouse. I think the lakehouse is really good for what we need to do reporting wise. And I think a lot of times I’m not thinking about real time data. I’m thinking about loading like at night and doing a lot of reporting. That’s just traditionally what I’ve been helping companies build and develop. loaded at in the evening in the night and then had that showing up later on. Excuse me. I’m tight. Oh, excuse me. I had to sneeze there. All right. then comes the question of where do you start? and a lot of this

14:11 Comes from people like I, , I think there’s a mindset here of a SQL server, right? When I buy a SQL server, I’m going to go use SQL. I can’t have the machine fall over. I have to have the right size tables. I have to have the right size compute. it has to have the right amount of speed and memory to make sure that that SQL server runs correctly. And I think there’s a lot of mindset of I need to design my capacity for my workloads. And I think sometimes we don’t really know exactly how heavy those workloads will be inside Microsoft Fabric. So, this is going to sound like a copout, but I promise

14:45 You it’s not. A lot of this is just try it, right? Do a do a simple test. Get on a Fabric F2 skew and at the cheapest price point, which is the F2, turn it on, build a little simple workload, get it working, get your proof of concept started, and see how much consumption that takes. Most often, you can get at least an MVP or a load of data working or something worked out so that it just physically works. And then you can step back and say, okay, if we’re doing a load one time per night, here’s how many

15:18 CUS is consumed. And then you can make the decision of do you want to increase the speed of delivery of data? If so, then you can increase the fabric skew and and go to larger sizes. What I find for most organizations is it’s nice to start at the lowest price point, get people comfortable and see the value of the solution, and then we’re more easily able to talk about, well, we’re going to go to an F2 or an F8 to start scaling up to higher levels. Let me just pause there. That’s my impression. Stephanie,

15:51 What have you seen? Is this similar to what you’re doing? Yeah, two things. And one, even cheaper than the F2 is you can get the trial. So, start with that. Then you’re paying zero dollars. , but also, I don’t think it’s a copout at all what you said. And in this new this new series on Microsoft Learn that you mentioned earlier, I’ll link to that one. Yep. Like yesterday or something. , that is what they also say is is the right way to go for part one for planning your deployment. Start with the PC. It’s really really hard to know upfront how much capacity you’re going to you’re

16:24 Going to need without building something. So, I’m going to just back up your your claim there. Okay, good. Tommy, what what about your thoughts? How how do you approach capacity plan? Do you do you do a lot of testing beforehand? Do you just start the free thing and work from there? What do you do? So, here’s the interesting thing with the trial. You don’t get the fabric capacity metrics app. So that’s only on a capacity that you’re actually paying for, I believe. So no, nope. You you still get the fabric capac you can still click the trial in the Yep.

16:54 That’s the So what Tommy’s speaking, let me just fill people in here real quick. When you build a fabric capacity, there is an app in the app store for PowerBI and in there there’s a fabric capacity metrics app which tells you how many CUS, what jobs are running. that analytical data still gets sent to the reporting system and then you just add in your capacity ID and then it works. So that does work but Tommy to your point though you are using an F64 which is a much a much higher version than an F2. So like that was maybe

17:28 Stephanie when you were talking about that one that was maybe one of my caveats of that one yet. Yes, do the trial but also note that that is an F64 level and you’re not starting at the lowest price point. So that’s F64 is a price point with the capacity. Oh, it’s like the yearly pricing. It’s like roughly around like $5,000 a month or so US. 5,000 a month for reserved and reserved capacity. More like eight eight. Yeah, if you have it on all the time, correct? And you can pause things. So yeah, there’s a whole b. We thought we wouldn’t have enough talk

18:00 About here for like an hour and I’m pretty sure we’re going to just like have so many things to talk about and options to like tune and optimize this. All right. Sorry. Go back to your Tommy. Go ahead. And but yeah, to your point, the the trial is fine, but it really doesn’t test or sense put in a lot of pressure tests. You can put pretty much run anything. You’re great. But a lot of organizations when they actually do purchase, they’re probably not going straight to the F-64. A lot of them are probably going to the F2 or the F the

18:33 F4. Mike, I’ve actually seen the if you were to try to compare apples to apples, the premium the lowest premium capacity license that there was, which I think was a P2, versus the F2. I’m seeing a much different throttles and bottlenecks with the fabric because we’re running different things and it’s sometimes a lot very hard to test that. So with premium I could run a ton of my semantic models for entire middle to

19:06 Smalls size company. Mike, I was running an F2 test on just my tenant. Not a ton of data and when one thing was running, I couldn’t even open load PowerBI service. It’s like what is going on? So I think there’s a lot of things that when you do testing and if you obviously start small and then you can always scale up. But the problem is you really also need to understand what one read that capacities app or build your own but that metrics app is

19:39 Vital to making sure okay what data are we testing or bringing in and then that that’s just for a single department. I’m not going to go into coming from an enterprise point of view, but Mike, that goes into conversations you and I have had in spades. Let me a couple reactions to your comments there, Tommy. So, like one of them is being able to observe what’s going on, right? So, again, a lot of this is new information to people, right? working in a notebook, brand new. Working in a lakehouse, brand new. running a pipeline, brand new.

20:13 Like a lot of these things, we haven’t had the experience of running these things inside a PowerBI environment. So I’m thinking from a PowerBI person, right? PowerBI person shows up. We typically connect to like a SQL server, SharePoint, Excel, get our data in, refresh it, we’re good, happy, no problem. The only compute we really cared about was how much compute it took us to reload the semantic model and then serve the semantic model. So very simple levels of compute and it was one package, right? That was like all we had to really worry about. Then we introduce dataf flows gen one which is now a little bit of data

20:46 Engineering in front of the semantic model loading right compute for data loading compute for semantic model refresh and then compute for the report to your point Tommy we now have a million other compute usage artifacts now right every time you turn on a notebook that uses compute every time you run a pipeline that consumes a different level of compute every time I run a copy job that’s still different than a pipeline but still runs at a different rate of compute. So I think Stephanie to your point earlier which

21:19 Was you just need to test some of these things because we’re no longer just introducing data flow semantic model and reports. We’re introducing a whole bunch of other things and I think that’s those are the things we you don’t have any context to how expensive or how many CUS you’re going to use per those things. Does that make sense? Yeah. And and I think also a key point for me that I always have to keep in mind with this Tommy you mentioned that you had their F2 and you were doing some data engineering workload and then you couldn’t view the

21:52 Report right so a really key concept is the that a workspace can only be assigned to one capacity right but one capacity can have multiple workspaces so it’s really important to think about your workspace strategy because maybe in that case Tommy Maybe you could have just had, , another workspace for your PowerBI reports with even some pro licenses. Maybe a pro workspace whereas your engineering is in a different, , so this is to me where it gets really complicated is the mixing and matching

22:24 To to save money like that. So really thinking about what goes in what workspace and what capacity can save money. I I like your comments out here. I do a lot of mixing and matching conversations for customers and so th this is an area where I spent a lot of time helping organizations figure out what’s the right blend of the different capacities. let’s also add in here this idea of the free user right so we’ve talked about prousers premium per users

22:58 And now capacities right buying capacity from Microsoft I want to introduce this concept of a user who does not have a pro license but could or still would want to view reports from powerbay.com or see those reports embedded in other experiences right so hey our team central BI team is building reports we want to put them in teams we want to put them in on SharePoint that’s an that’s an embed for your internal organization or embed using embed with users own data there’s also this idea of I can share my reports with external companies that’s

23:30 Embed for external users so those also have licensing impacts as well and the part that I think is the most difficult for people to understand is at an F64 level when you buy that skew free users ers can start using powerbay.com. There’s a there’s a threshold here and this is the same story. So a an equivalent previous metric was an F64 is equal to a P1 a premium one. In that equivalent world, you now have

24:04 That free user can then show up. They can look at reports. They can build their own stuff using their fabric.com experience. anything below that F2 F2 through F what 32 that free user is not able to go in and again to your point Stephanie was not build PowerBI related things right because they’re still there those users at those lower SKUs could build like a pipeline right yes but also just to be clear yes

24:34 To build a to build and share a PowerBI item you do need a prolic no matter what size skew you’re talking about Those free users are just for viewing for free at F64 and above. So you still do need the pro to to share publish and share. I’m going to challenge you on that one. I think I think the free user can can actually make reports at the F64 level. But I So I would 100% agree with you on on the I had to test this one because this has been a while since I’ve tested this one. But I’m pretty sure to your

25:06 Point a free a free user can go Okay. Thanks, Tommy. A free user can go into an F64, and I 100% agree with you. Look at all the content that’s there. I’m with you 100% on the board. Like, they don’t need a pro license to do that. I agree. But I think at the F64 level, you can actually build reports inside powerbait.com using an a free user. At the F64 level, the authoring capability is turned on. I’m pretty sure about that one, but you can don’t quote me on that one. , chat,

25:39 I’m going to lean on you in this one as well. So, chat, confirm with me. Does a free user have the ability at an F64 level to author reports? I think the answer is yes, but just chat will check me on this one. So, I know someone in chat knows about this. I’ll let them go Google it as well. Okay. , let’s talk about the tools in in the market here a little bit. I’m going to transition just slightly. So, we talked about like the different licensing. We have been talking for like the first 10 minutes of just like different patterns and

26:11 Different licensing options here. Okay. In addition to this article here, I really like what Microsoft is doing. So, the one I just sent out in the chat here talks about playing your Microsoft capacity strategy guide overview. And it talked about in the series, you’re going to be planning your first deployment, scaling for self-s serve analytics, and then scaling for enterprise and managed self-service solutions and managing your capacity and growth and governance. So I I think this article is very useful here about really trying to focus on

26:47 What is your proof of concept, how do you build the pilot, and then how do you get from pilot to production. And I I really like this article that Microsoft gave from like a strategy standpoint because this is literally the conversation I’m constantly having, which is let’s just get something going. Let’s get something started. Let’s just figure out what the usage looks like. So, let me just pause there. What What do you think? What are your thoughts on Okay, so the chat I’m going to come back to my comment earlier. I’m going before I give this over to Stephanie here. Chat is telling me, “Nope, free users cannot edit any content. They can

27:22 Only consume read content.” And so Stephanie, you are correct. PowerBI, they can do they can do non PowerBI in fabric. True. They can do non PowerBI fabricy things, but PowerBI reports and semantic models can’t do it unless they have a pro license. And that’s what the chat is confirming. So if you want to view it, you can do it on an F64. But if you want to edit it and create and do, , make PowerBI related content, that’s a And that’s how it was with the PK SKQ too. Still needed. Yeah. Yeah. So that that remains the same. So yes, clear as mud remains the same.

27:57 Okay, sounds good. All right. , let’s go over to the article then a little bit more. So what are your thoughts Stephanie on the plan your items there? I’m very happy to see it too because I feel like this is what I always say. It’s nice to have this backup from Microsoft as well because I always feel a little bit sheepish about saying, , I’m sorry I can’t predict what you’re going to need. So we really do have to build something before we know. So it’s nice to see this this written out this strategy like yes you you just you pretty much have to do a P. It’s

28:29 It’s really hard to predict the future. So I I do like this a lot. A couple a couple notes here that I really want to point out here is in the key actions in phase one. So okay talking about the proof of concept phase and then in halfway down the article it starts talking about key actions in phase one. I really resonate with keep the scope narrow because sometimes you you get into these situations especially in the consulting space people just try to boil the ocean like hey let’s do everything let’s let’s do real time let’s make a lakehouse let’s do it in a sequel like no no no time out what is the main challenge we’re trying to

29:01 Address right now let’s isolate that particular activity and focus only on that item and then they’re also talking I wanted to touch all the parts I needed to I don’t want to just test a lake house. So if I do need to ingest from, , I do want it to touch all the parts. So I want the lakehouse, I want the notebook, I want the pipeline, the semantic model and the report. I would argue that’s probably the most common pattern that I observe, right? The most common pattern is connect to some data. doesn’t matter what it is.

29:33 But then I connect the data, put it in a lakehouse, use a notebook, transform it a little bit, drop it into and I we’re doing a lot more testing now with direct lake than we are import modes inside cementum. So we’re doing a lot more direct link to reporting all right in the solution and then also we’re doing a lot of testing around where does the call them the artifacts where should they exist even right so when we do I’m going to bring an analogy here and again you guys can correct me if I’m wrong or or redirect me here

30:06 When you do semantic models and reports or thin reports in some situations you may want to have a deployment pipeline and you may want your semantic models to be captured in its own workspace because I’m going to share them. I’m going to govern them differently than the report side. So the report we have a single workspace for all reports. We have a different workspace for all the semantic models. And then we use deployment pipelines to pair the deployments around from dev to test or prod. I feel like this is another thing that we’re testing a lot of is okay now that we have semantic models reports now we have this whole new world of data

30:40 Engineering lakehouses and we’re trying to figure out where does that exist is it a is it one workspace with all the lakehouse pieces in it do we have multiple lakehouses with dev test prod in the same workspace do we need different workspaces for dev test prod like there’s a lot of thinking about who needs to touch the data and who’s allowed to see the different environments as we build them. So a lot of what I’m doing also in this proof of concept phase is really stubbing out the organizational structure and what should be built and

31:13 Which people should have access to what things, right? Because I may need an one extra workspace to help segment permissions. Does that make sense what I’m describing there? It it almost feels like we’re going back to the world of DBA and databases and servers. I remember that back in the day where it’s like hey our development is on this license from Microsoft and we have this on premise so we’re only we can test everything you want on dev but don’t do anything on prod and we’re to me it’s almost going back there

31:46 Where again we’re there’s the mind shift right now Mike because before is like having one single usually premium capacity that is the capacity that you assign to every workspace now people can access your content. Generally speaking, most organizations were, , non-enterprise had were fine with a single capacity. But this is completely changing, especially even when you think about that proof of concept and putting together a trial run. Well, are you

32:21 Going to just do PowerBI? Are you going to try to integrate data? And if that’s the case, , I don’t think that goes in all a single fabric capacity in your development test and prod especially with and this is where the concern is right now. Yeah. So you’re touching Tommy on a point that comes up right next in the article. So scope two or key actions in the development of the pilot phase, right? So now you you move away from pilot to the development side. talk about and this is what I would also

32:54 Echo with is you really want to develop on a separate capacity right so if you have existing things that are already running in like a psq or you have something else that’s in production that’s already ready to go the idea here is you want to separate development work dev and test potentially away from the production capacity and the reason here is there’s and for better or worse it smoothing helps so smoothing inside The fiber capacity is this concept of if you overrun your or overuse your capacities

33:28 Number of compute units CUS Microsoft will let you borrow from the future right so they assume that oh there’s there’s a one job that’s running right now it’s very spiky in nature it has a heavy load we’ll just let you consume all the load you need for a short period of time knowing that you’ll dip back down and then you’ll be able to purchase future compute units and then apply them to that spike that that heavy usage, right? So, this is the smoothing concept here that we’re talking about. And smoothing, I think, throws things off a little bit. It it

34:03 Makes things difficult for people to like see what’s actually being used and like but is is really affecting what’s going on. But that being said, I really do think and believe in the isolation of dev and test capacities away from production capacities. I think those should be separate. And as you become a larger organization, you may also want to think about departmental separations, right? U Tommy runs the HR department. Well, actually Tommy Tom Tommy shouldn’t run the HR department. That would be very bad. Stephanie runs the HR department and Tommy runs the engineering department. But but in the situation, right,

34:38 Stephanie, you may have a lot of heavy demands on whatever data you’re pulling in. Like you’re doing some scraping or you’re using a heavy notebooks or you’re maybe you’re running like machine learning algorithms or things. your department may need a bit more capacity than Tommy’s. And so if we have one capacity for our entire organization, the noisy neighbor applies like Stephanie, you may be, , using more capacity units than Tommy is, and that might not be acceptable. So when we do chargebacks or figuring out who needs to pay for what level of capacity you’re using, again, I’m I’m all for giving everyone fabric, but I just want you to

35:10 Pay fairly for what you’re using, right? Right. So I think that’s another a use case where you can start separating the fabric capacities into department level things. Yes. Any other thoughts? Yes. So many first of all I’m definitely a noisy neighbor a troublemaker but and yes so for some of those like dev and tests another reason to separate those into a different capacity is because you can pause them. , and now, , with this fabric skew, like you were saying, F64 is equivalent to P1. , there

35:43 Was nothing smaller before than the P1, and it was fairly expensive. But now that we can have an F2, it’s a lot easier to have it’s more affordable to have multiple capacities than it was before in PowerBI premium land. So, it’s it gives us a lot more choice, but it’s more complexity, too. I want to unpack that a little bit because I I find the story resonates really well with clients around let’s imagine you’re buying one big F64. When we talk about that, sometimes it

36:17 Makes sense for us to divide that up slightly to get better use out of that 64 capacity, right? So, I could buy one F64. I could buy an F-32 and an F-32. It would also be the same thing. or I could buy like an an F4, an F8 and then like an F-32 and some other piece like so you can piece meal together you’re and the neat thing is if you do this with the capacity planning you can pause this stuff as well. So another thing here to think about is you

36:51 Know if you start thinking about splitting your capacities apart I’m getting a lot of considerations here. Oh gosh, there’s so much to think about this topic. I’m so sorry. This is just like going as we’re talking. I’m literally having recalling like the hundreds of conversations I’ve had around capacity planning. You could even pause things. So like for example, if you have dev and test as a separate fabric capacity, you may say, look, we don’t actually do or need dev and test at nights on weekends. So you can set up a runbook automation. You could turn it off. You could turn it on in the morning. do your dev and testing

37:24 And then pause the skew, turn it off for nights and weekends because again the idea here is you needed people to work on those areas like during the business hours like there’s no reason. So you can save yourself a ton of money if you don’t need to use them or run them over the weekend. They’ll pause them and turn them off. Yeah. And then go leave your production stuff on for more. So there’s all these different patterns that can come out of this. And what’s wild is even when it is when a capacity is paused, you can still access the data that’s in a lakehouse in that paused capacity, for example, by

37:58 Shortcutting to it from another capacity. So it’s just sitting in in storage accounts. Yeah. So you can still access it even if the capacity itself is paused. Thought that was I thought you couldn’t access the lakehouse when it was paused. You can let me give you a link for it. I’m all off on this one. Yeah, I need a link for this one. So, this is one of my favorites. I love this. Yeah, that’s that’s amazing then because now that means so going back to our earlier conversation, right, around the ability to have multiple workspaces

38:32 Like so we were talking earlier about semantic models and thin reports. This is another good use case for saying we should put the data engineering inside the data engineering side of things. Does that make sense? Yes. And I’m so sorry for the dog barking. I’m going to mute myself. Okay. Mike though, it sounds great that we can do this, but this really Okay, let me before I say my any hot take here. Let me ask you a question. everything that we’re talking about is

39:04 This required like this level of detail and attention to detail on turning off turning on the different workloads making sure that we set up not just a single semantic model and get that through the pipeline but how we’re assigning the workspaces and what data we’re putting in there. Is this a requirement for companies running fabric or is this the advanced thing? Is this the and be and you’ll see why I ask.

39:38 Yeah, Tommy, I’m going to this this is I will say when I this is an overwhelming point, right? This is this is people very quickly get overwhelmed with we need help with capacity planning. We need someone to think through it with us. If you don’t know these things, how would you ever learn about them? like how how would you figure out all these different patterns unless you’re like constantly in the tool starting things up and pausing things and then building different divisions and building different workspaces and then again Stephanie thanks for the article I just put it in here. So October 22nd 2024

40:12 Elizabeth says use one lake shortcuts to get data across capacities even when the production capacity is or even when the the producing capacity is paused. Amazing. Like great shortcuts work when the capacity is not running. Love it. Like that’s this makes a ton of sense. So I really like this. So this also means further tuning. Run your run your nightly load on your capacity. Turn it on. Do the lakehouse thing. Run the capacity. Pause it. Shut it off. And there’s a whole bunch of shortcuts you

40:44 Can go use somewhere else. Amazing. So this but to your point Tommy you you made a comment earlier around a SQL DBA. I think this is the same thing. It’s a fabric DBA something something along those lines, right? It’s not a DBA for SQL. It’s a fabric DBA where you’re being the administrator. Like these are all the the tuning and the optimizing of things, right? And I think this is we’re stepping into a place you’re listening to experts talk about ways of optimizing the compute get the best out of it.

41:19 Okay. So I I think new users aren’t going to know this, but take heart. There’s whatever your company needs, there’s probably a design pattern that will work best for your company and you can then suck out costs so that you and again back to what Alex and I were talking about earlier was like this minmaxing, right? There’s this idea of a minmax of fabric. You minimize the cost but you maximize the usage, right? So people are going to be creative. companies aren’t going to be like, “Yeah, let’s just throw all of our

41:50 Money at this really large fabric capacity and yeah, we’re happy. Move on.” Like, they’re only going to want to spend so much money that it makes sense to get the value out of it. And I think there’s a lot of pieces in this because you could have fabric in place. You could have it loading data in, but if you don’t have people on your team who understand how to like properly use PowerBI, properly build semantic models, build fast reports, all of those things can detract from the attractiveness of like this one solution that gives you everything you need. Does that make sense?

42:21 Yeah. And and I pause there. Stephanie, I’m going to let you run with that one. So yeah, I was gonna say Tommy to your point about what if what if you don’t know all of these ins and outs. , just look at our conversation here between the three of us. We’re like, wait, does it work that way or is it this way? Right? And but so I think the answer is if you don’t care about money, just get a big capacity. Then you don’t have to worry about any of this stuff. Then you don’t have to have a that’s the secret. So, and and I think this is the concern

42:54 Is a lot of we’ve already dealt with people running like wild west and somehow breaking PowerBI before and it was just get a capacity and assign it. That was it. And that’s why we would come in because everything was broken or whatever the case was. Now you’re adding all these other things where it’s like, hey, just get a fabric capacity, but that’s really not it. , in to to a degree because yeah, you can have a single F2 capacity and

43:26 Put everything on it, but I think we’re making it sounds very clear in this conversation at least that we’re having is you’re not going to be very successful that way. And there’s from I think this is a big point of what I think we’re not realizing with when it comes to planning more importantly from adoption right and I think for a lot of organizations when they’re looking at more than just hey we’re going to move from PowerBI premium to fabric I don’t think how many organizations are doing that or are they saying we’re going to

43:58 Move our existing data warehouses and lakehouses to fabric or the probably the other scenarios we’re going to test creating a lakehouse with some of our marketing data in fabric. Well, they’re just thinking just buy fabric capacity all the workspaces just like premium. But I think there’s a there’s a thrust point or there there’s a there’s a pressure point here that a lot of organizations are going to face

44:32 When they start thinking I can do fabric everything. I can do all the notebooks and all the lake houses. And I think this is where the concerns coming in where it’s like well I had 10 semantic models before and so I should be able to run a single notebook or the data flow but when you realizing what’s throttling on top of that the fabric capacity is also your capacity units for your semantic model rendering and people viewing it. So there’s a lot more I think this proof of concept

45:05 Becomes very complicated to me. Yes, my dog agrees exactly. , and to your point, Mike, about, , back to school families, that’s exactly what was just happening. Kids leaving for school, dogs freaking out, so apologies about that. No, no, it’s fine. But yeah, like I think like like anything, it’s really optimize or throw money at it, right? I don’t I don’t think this is limited to fabric. It’s limit It’s Yeah, it’s all this could be anything. Yeah. , we over we over buy SQL servers and Azure SQL servers. You over buy on

45:39 On prem servers because you don’t want to come back and have to ask for more funding later on. , I it’s a definitely a political game that you’re you’re playing here to some degree. , where I’d like to maybe move the conversation would be is really love the documentation that Microsoft’s providing, but I also want to start saying like this capacity planning guide article that I’m reading or have has been published literally yesterday is probably one of the most comprehensive places I’ve seen like full thoughts around capacity planning and dev planning and which isolation environment

46:13 Environments. But there are a number of other tools out there to at least help you estimate what you may need to do. And so I want to point out a couple other tools that exist here as well. One of them being Microsoft’s own PowerBI load assessment tool itself. So they actually have a PowerBI capacity load assessment tool that you can go use. , so a load assessment tool is basically a script, a a GitHub workspace that you can go use and you can run the tool directly from there.

46:45 And what it will do is it’ll simulate multiple users running reports on a capacity. So you can stand something up. It has a bunch of sample reports. And so this is an an assessment tool you can use. So this is one area where you could turn on one of these trials, an F64 trial. turn on the assessment tool, do something. Okay, how many usuals do we have? 50. Great. You can you can plug in some numbers here and say we can do a test for 20, 10, and 20 and see how that scales up. because one of the things I think it’s very challenging for

47:18 People here is every semantic model is different and they’re all tuned to various degrees. If you have a really high performant semantic model, you could easily serve thousands of reports to thousands of users on a F2. I do it like we have we have an app that we run right now today that has thousands of users per month hitting a report. But the semantic model is stupid simple. There’s like nothing to it. It’s literally just data that’s in there and it’s a lot of it can cache,

47:49 Right? So a lot of the queries are cached. You don’t need a lot of interactivity. So the report itself is actually really optimized for rendering and sharing the D like showing the information. Again, this is all around theming. one of the tools that we build. That’s not always the case. , Stephanie, you’re in HR and you’ve got all these things. Your cment might have complex DAXs in it or whatever, right? th those things could impact your capacity because this is where it gets so like it depends conversation is because whatever you put into the capacity really is how the knowledge of

48:23 All these things you you have to be good at optimizing more than just one thing that make sense. Let’s pause right there. Yes, it it does. And I’m so glad that you mentioned tools and calculators because I know we’re we’re running out of time and we actually had a bunch of those that we wanted to share. Right. Some of this is trying to plan and some of this is moderate monitoring and we have fortunately tools for both and they’re both important. Correct. But let’s go after the there is a

48:56 Microsoft capacity planning overview which we already went through the article that’s already in the chat as well. you’ll see here the load assessment tool I just put in the chat right now. So that’s for the load assessment. These links are also in the description as well. , I believe data witches here also has a licensing calculator. So, Stephanie, you came up with your own calculator that I’ll also put in the chat here as well that you decided here’s how we here’s how you want to test. You can enter some information in. Walk us through your calculator. How does this work for you? Yeah, so this one I actually built before before fabric. So, it was PowerBI premium just because even then it was a

49:30 Little bit complicated to know should I do pro, should I do ppu, should I do psq? , so I adapted it for fabric. , and I think I I think it’s helpful to use this in conjunction with the other one you’re going to mention, which is the fabric capacity estimator. Because the fabric capacity estimator, that’s not going to tell you, okay, based on your inputs, you actually don’t need a fabric capacity. You could stick with ppu. This is going to tell you a fabric capacity, but you can use our data witches calculator in those cases where you’re thinking, do I need a will a fabric capacity benefit my situation

50:04 Or can I also consider just pro or ppu? So I think these two together can help in that situation. You also don’t need to have as many inputs for the data which is one as for the official Microsoft one. So, I think those two together, yeah, can help with that a lot. , but yeah, mine mine came out of the I mentioned before I have a nonprofit background, so I’m used to trying to do things on a shoestring. So, I was really heavy in that minmax thing. , I don’t want to get more more than I need. So,

50:37 That’s where that one came from. And again, you’re if and when you’re in the nonprofit space, you’re trying to best optimize for, , donors only give so much, right? Their dollars only go so far. So you have to always be as mindful as how can I optimize everything? How can we do everything as efficiently as we possibly can with the tools that we’re given. so love that. I will also echo again this is not a ding against Microsoft but I’ll just maybe make a note here is the fabric capacity estimator. This is the link I just put in the chat right now. This is really to your point I want to

51:09 Double down on this. It is fabric only. It is not trying to tell you and that’s one of my maybe a pet peeves or something like this. If I enter in only things here that are like wait I am only going to enter in like the total size of your data when compressed in gigabytes. They start off with like a 100 gigabytes as as the starting point for the tool. Like who on their right mind is starting with what I should really think about capacity planning. Oh and by the way my semantic model has 100 gigabytes in it. Like you’re like you’re you’re

51:43 Hitting the wrong market bud. like this is we’ve we’re well beyond like if you’re at 100 GB semantic models, you are clearly not in the right zone for tuning and optimizing your capacity. I guarantee you’re you’re already you’re already doing this at some degree. So, , and there’s no way of having like to your to your point Stephanie was what if I have a a 50 megabyte semantic model? I can’t enter that into their tool and the tool doesn’t say look I see what you’re doing here. you’re actually still good to stay at a at a prouser level like so there’s like some

52:16 Other I think there should be some level of like recommending okay you’re fab yes yes this is fabric capacity pricing but it does he should also route you at the beginning of this do you feel like you need fabric tell me what things you want to do and then from there can like give you some suggestions around that but that that’s one of my rubs there in the beginning like if you just selected okay let’s say I have a 1 gigabyte data model and I’m only going to use PowerBI and I have like a minimum amount of viewers. It just ref it just recommends an F2 which in in my mind it should also just say well maybe you should just

52:48 Consider pro or premium per user like there may need to be like a little bit of a caveat of this also might be a workload that’s effective for this other space. So I think it’s a little bit misleading licenses in that case anyway. So your F2 doing anything for you and that’s yes exactly right. So I think there’s there’s a little bit of edge case stuff of what the tool I guess they assume a bit of extra things like hey you’re as soon as you add a data factory and then add some things. Yeah, maybe makes a bit more sense there. So anyways, other thoughts around this fabric capacity tool? Tommy, have

53:20 You played with it a bit? What do you think? Oh man, I I missed the old PowerBI premium calculator. just showing how many people were viewing reports, how many people were building, how many semantic models you had is pretty pretty straightforward. this if you’re coming from PowerBI do what a batch cycle is and can you tell me how many tables you have not just now but are going to go into fabric and probably again I think a this is it seems like what from our conversation we

53:52 Already had about the different way we want to in a sense structure and the logistics of fabric capacities and workspaces. This goes along perfectly with that because this is DBA. This is now we have entered the world of IT my friends. when it comes to database administration, database management because I guarantee you the data size compress batch cycles those are foreign to most people in PowerBI unless you again worked in the DBA side or worked

54:24 In the the like closer up to it. Mhm. There’s three things for PowerBI. You have for the data there. The only thing when it has for the data factories is hours for daily Genflow operations data flow gen two which I thought was hilarious. and yeah, like it’s just I think for a lot of users like how does it matter how many lakeouses I have? What if we’re just getting a bunch of data in raw? Like does how much does

54:57 That c like cost? I think that’s what a lot of people want to know. Yeah, I think we’re probably about just about time here, so it’s probably wise for us to start thinking about our final thoughts on how we want to wrap this conversation up. , there’s a lot of really good tools out there. So, I think we shared like the two main ones I think that are I would spend time on is learning is working on the data witches tool for licensing calculator. I think that’s a really good input model that you can use. , I also think the fabric capacity app that Microsoft provides is is decent. , the one that I’m most

55:31 Impressed with right now is the fabric capacity documentation Microsoft has given because I I think this is a very large point of confusion and the article that’s being presented here , is really well written and gives you a lot of the considerations and recommendations I would be supplying to customers already. So my final my my final thought here I think is going to wrap up is look if you if you’re just hearing about this now and saying wow I really do want to use fabric and I do need some help and support designing the right mix of per user licensing

56:04 And capacity based licensing and what features do I need to use and what do I think I not need to use. This is where I’m going to pitch all of us all of us are in the consulting business. We all have our own consulting firms. We all do this. So if you want help, reach out to us. Let us know. Like find all of our names are in the description below. If you want someone to support you and give and give you like the alternative options or help you walk through the design or the system design, this is where if you don’t know how to do this,

56:37 You probably should recommend I’d recommend going find someone who understands the intricacies of the licenses pieces and can help you design which users, which licenses, how to scale up, how to divide workspaces. , that’s what we do. We help users with that. So, I would say reach out to someone who knows this. go find a consulting firm that has this and you only need to do that once. Once you get through this pattern, you’ll know, right? So that if you don’t know, you’ll learn and then from there, you’ll have the ability to do this on your own and design things. So that’s one area I would recommend is just hire some

57:11 Consultants, go through some ask them what they think and we’ll all help you with that. So that’s where I would I’d lean this one. Stephanie, where do you think final thoughts? yeah, final thoughts. So definitely use those two tools if you’re just trying to figure out where to get started, what sizing you need, etc. , but then of course, like you said, the article is great and I love the idea of, , you really do need to build out a PS Pilot. We didn’t get to cover too in depth today the capacity metrics app, but , that’s a that’s a critical piece

57:44 Of all of this. So as you’re building, , you really need to get some understanding of that app. And I think Mike, you did put a helpful article in there about how to read that app. It’s it’s a little bit old the article that the app has changed a bit, but it’s really really helpful with some of the concepts. , so I think that those go hand inand you’ve got your calculators, your planning, but then the monitoring is key. So monitoring as well. That’s a really good suggestion. I really like the idea of like just do stuff and see what comes out of it. What

58:16 What does the modern need because then you can always adjust later on. What are your final thoughts? No, I think for a lot of us as we start getting ready to jump into fabric or organization has this is not your father’s PowerBI and it’s really the the planning here the proof of concept I think is going to make or break what you do because if your organization just runs into this and saying we’re going to just try everything and everything slows down and everything bogs down and you’re spending the likelihood that people are going to

58:51 Buy in is very low. So have a plan not just for the feature sets but for the back end and what’s going to make sense. I like that one. That’s a good that’s a good thing to keep reminding ourselves is to narrow the scope down to what you only need to build because you can get distracted by a lot of other things and it muddies the water around well how much capacity did I actually use? What is what is really going on here? So that’s that’s a really good point Tommy. Awesome. Well, thank you all so much. I want to thank everyone for attending and listening to the podcast today. We really appreciate

59:23 It. even the dogs are happy. So that’s awesome. We had a couple barks here throughout the episode. That way it’s actually live. Like this is we we were not filtering this out. This is all as it happens in real time. So we’ll see. For those of you who get it on Apple and Spotify, we’ll see if Tommy’s editing is able to cut off some of the dog pieces there. I don’t know. We’ll see what happens. but that being said, thank you all very much for listening. I hope you found this to be informative. I hope you found some links here that were interesting. There’s a ton of links in the description below. Make sure you check out those links. Click on those links and go learn more about licensing, licensing calculators, and how to design your fabric and or

59:57 PowerBI environment so it best works for your organization. One thing I’m just super happy about is we have options. Like it’s it’s not like we have only one tool and one way of doing things. we do have many different options to perform better and to get the right balance of value and price for what we’re doing inside Microsoft Fabric, which I’m very pleased with. That being said, if you enjoyed this episode, please share it with somebody else. Let somebody else know you found value from talking about licensing and capacity planning and organizational structure here as well. I

60:30 Know, riveting, right? everyone’s probably thrilled about hearing that conversation, but we do it as as experts in this space. You have to figure this stuff out. Anyways, that being said, Tommy, where else can you find the podcast? You can find us on Apple, Spotify, wherever you get your podcast. Make sure to subscribe and leave a rating. It helps us out a ton. Do you have a question, idea, or topic that you want us to talk about in future episode power head over to powerbi.tipsodcast. Leave your name in a great question. And finally, join us live every Tuesday and Thursday, 7:30 a.m. Central, and join

61:04 The conversation on all of PowerBI. Tips social media channels. If you’d like to support the channel, please become a member down below. We actually have just recently opened up membership, so you can listen to all the episodes for free with no ads down below as a member. So, if you like this episode, you want to have all the episodes with no advertising, , feel free to go downstairs and downstairs below and the links below and check those things out. Anyways, thank you all. all so much and we’ll see you next time.

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.