Deployment Pipeline Missing Features – Ep. 330

In this episode of the Explicit Measures podcast, Mike discuss Deployment Pipeline Missing Features - Ep.330 - Power BI tips.

News & Announcements

• Tabular Editor 3: June 2024 Release — We are excited to introduce the new version of Tabular Editor 3, which is available now.
• Solved Fabric Community posts are now available in the Fabric Help Pane | Microsoft Fabric Blog | Microsoft Fabric — The Fabric Help Pane provides Fabric users with a fast and efficient way to access self-help content, allowing them to resolve issues independently or direct them to create a support ticket if further help is…
• PowerBI.tips Podcast — Subscribe and listen to the Explicit Measures podcast episodes and related content.
• Power BI Theme Generator — Create and download Power BI report themes using the PowerBI.tips theme generator.

Main Discussion

This episode focuses on the state of deployment pipelines (and deployment process) across Power BI and Fabric. The conversation is less “pipelines are cool” and more “what’s missing, what’s risky, and how do you ship changes without breaking production?”

Key points:

• Deployment isn’t just code promotion—data, permissions, connections, and environment config have to move (or be re-created) safely.
• There are still capability gaps and inconsistencies depending on the Fabric item type, so you need to validate your exact workflow.
• Git integration helps, but it doesn’t automatically solve environment parity or data refresh realities.
• A practical approach is to define a minimal release path (Dev → Test → Prod), then document what is automated vs. manual today.
• The team emphasizes designing for repeatability: templates, parameterization, and clear ownership of the release process.

Looking Forward

Keep an eye on upcoming updates in Fabric and Power BI, and let us know what topics you want us to cover next.

Episode Transcript

0:22 [Music] out out [Music] good morning and welcome back to the explicit mes podcast with Tommy Seth and Mike Feel getting really good at that intro intro now now 330 yeah when you’re good at 3 at you got a you got a pattern that’s working here and if if we’re supposed to do those 10, 000 hours I think what we have another 9, 000 to go oh we got a lot of hours to

0:53 oh we got a lot of hours to go yeah I agree with that one a lot of hours well let’s let’s talk about our main topic today we’ve been talking about this just a little bit here actually a lot of bit maybe more recently it’s another another mailbag topic is coming in today I’ll let Seth read it here in just a moment but the topic will primarily be around deployment pipelines and the Readiness of the P of the fabric platform so that’s going to be our main topic for today if you’re interested in learning about deployment pipelines and or discussing where to use them and

1:23 or discussing where to use them and what they’re good for this will be the discussion for you with that let’s jump into the news I I feel like I really do want to build a little segment here that’s some some audio clips around the news there you go just like that nailed it who needs to build an audio clip I have Seth he is my he is my audio the old school like typewriter don’t you like oh yeah the news the news yeah yeah

1:53 yeah the news the news yeah yeah something written in Cobalt or something like that yeah wow yeah some old school language people things that things that no would know about right like the save icon all of our appliations you mean a keyboard no I appliations you mean a keyboard no a I think I have a music clip on mean a I think I have a music clip on my floppy disc so let me pull that out yeah exactly have you have you ever have you ever given your kids a typewriter or shown them a typewriter they know what do they know what it is own one you don’t own one why would I own one I have we went to a museum one

2:24 own one I have we went to a museum one time and in the museum somewhere in the museum someone had just landed a typewriter and they’re like oh what’s this I’m like this is a typewriter they’re like what do you mean I’m like this is how you would write letters they’re like what are you talking about and I had to like explain there’s this little ribbon and it gets dirty and it’s ink on it these little arms come out and press all the letters they’re like this is cool I’m like well apparently when our parents went to school my parents your grandparents they would take these they’d had electric typewriters and they would take these big bricks of a machine

2:54 would take these big bricks of a machine into school college or whatever and that was what they would type on I’m like yeah it’s a little bit different now with a laptop they’re like w that’s really different anyways sorry I digress you do digress typers you want to start talking about cassette tapes too yes let’s go with at tracks baby all the way back I don’t think I ever SE seen an at I wasn’t an at track Cass I was a Walkman guy Walkman yep that was more like a brand it wasn’t really necessarily like a tape Sharpie cuz you had like a CD cuz

3:26 a tape Sharpie cuz you had like a CD cuz you have a CD player that was a Walkman CD players like almost a rarity now too why would you have one none of my computers I have I have a CD place I would have to buy on Amazon a USB port yeah for a CD slot my desktop came with one and I don’t ever use it I could use it as a coffee holder and it would be better that way by the way when I had to reset my computer I said please insert your CD ROM your Windows CD ROM like that may be a problem we’re gonna have an issue here

3:57 a problem we’re gonna have an issue here that’s that’s hilarious couple items Tommy you picked up some news here from tabular editor on their blog I’ll put that on the chat window here as well what’s going on what’s going on over tabular editor so we got the July release Tabler editor and some would say a few more updates in powerbi this this month Zinger Zinger so I think the the best one if you use table editor 3 you probably use the pivot Grid it’s basically a user interface to create a Dax query in so many words I can add my

4:29 Dax query in so many words I can add my fields and I kind have my metrics kind fields and I kind have my metrics get the numbers and use the Dax of get the numbers and use the Dax debugger well two major improvements here I can now save a pivot grid which I actually really like that feature right off the bat but I think the more important one is a overhaul to the pivot grid it looks a lot more like excels pivot TBL pivot tables it’s what it should look like it was a little clunky it was a little clunky I didn’t understand how to work it I got I got the gist of it and and this is one thing I’ll point out to people I think it’s really important for people to understand pivot grids are very useful

4:59 understand pivot grids are very useful in editor and I think this is only in tabular editor 3 this is not in tabular editor 2 so just FYI this is the paid version of tabular editor but they’re awesome because you can build measures and then render a table for yourself to figure out did your measures actually compute the right numbers that’s like 90% of what I do in desktop when I’m building data models I’m like building measures and then I’m immediately building tables of those measures to make sure that they’re working correctly does this make sense is it working as I expected and I’m just grabbing different dimensions that’s what this does

5:30 dimensions that’s what this does 100% And so yeah I I think the user interface looks great and then the final update is you can export your Dax query to CSV or Exel which it was part of Dax Studio back in the day or still part of Dax Studio but all these features are great are are great to me because I constantly find myself with those types of tooling where I either need to showcase the results somewhere else and I’m usually doing a lot of copy paste so I love the ability to export

6:00 to export but I am very very excited to try the new user interface out that’s a great idea I’m G to actually make sure I go update right away with that one so with that I will also put in the tabular editor 3 link if you want to go download it it is a free software to download you do get there is a license that you need to pay for it so you get a 30-day free trial with tapio editor and then if you want to keep it I think there’s like different price points for if you’re if it’s just you using it if it’s your enter using it and as you go more

6:31 enter using it and as you go more towards the Enterprise you get licenses you can share across different people so and I I I’m finding to a lot of organizations I’m I actually do a training on Tableau editor I think my next one is in August it’s it’s three hours but I find a lot of organizations like yeah we’ll we’ll purchase that for people who are taking these courses so if you find yourself like look I do powerbi once a week you should be able to get this licensing by your organization I know my organization offers it so okay that’s funny yeah

7:03 okay that’s funny yeah yeah joke I know it’s a joke because you’re one you’re one person you’re just one person I’m like you may not get PE very well but sorry I need more coffee awesome and then the second note here is another fabric blog item just came out Tommy Let’s there’s some new information coming to us inside the power. com service you’re right we are doing really well with these Segways so now in Fabric in Microsoft Fabric in the user interface if I’m looking for

7:33 in the user interface if I’m looking for help usually there’ be help documentation I can search there’s a help paint on the right hand side that shows up and no longer brings me out of the experience but now what they’ve Incorporated is the ability to look at solved Community post so we are all huge components if anyone more than anyone here Seth Bower our MVP of MVPs on the community Forum y exactly right on SOL post used used to be what a legacy Never Dies so

8:04 what a legacy Never Dies so right but now you can actually find not only official documentation by Microsoft but it will also filter or you have the ability to filter solved Community post on a relevant topic topic so I I want to test this one so I no I I like it because that that’s why communities exist right to share information and if it’s what one less click away or think about even as we’ve we’ve talked throughout the

8:35 as we’ve we’ve talked throughout the years like so many people didn’t know the community was there right and there just a wealth of knowledge that is going on that is being shared so solve you on that is being shared so solve throwing this into the application know throwing this into the application I think gets more visibility onto that which is great the one question I have which I’ll have to test out is is granted it’s only the solve stuff but is it more curated than that or is it the entirety of the community right now I believe it’s only

9:06 community right now I believe it’s only solved Community posts from from all time if any solution what I’m saying because if you’re because if you’re and here here comes a Nuance right you have a product that has rapidly evolved over seven years and is now in an ecosystem called fabric like are we bringing things from 2000 into here where somebody like I’m doing I’m finding 2020 2022 like I’m I’m seeing some old stuff here so that I

9:37 seeing some old stuff here so that I think may be problematic because that that would potentially confuse people even further right like my my my my my data my data flower my data set is is not refreshing and then something pulls up from super old like data flows flows faces that picture from like I don’t even it’s not even the same application so that that’s the only like criticism I would have around that because like if

10:08 would have around that because like if you go to a community it’s you you go to a community it’s it’s curated for that long of time know it’s curated for that long of time whereas a help and search in a tool should be very relevant to like the tool alone so we’ll see I’ll make the argument that it will be worth those hiccups because a lot of tooling I’ve used I’ve used Confluence and jira and they have the same integration with a forum and I find a lot of my questions I was trying to answer were not available in the normal help documentation yeah

10:39 in the normal help documentation yeah there’s a few like that doesn’t exist

10:41 there’s a few like that doesn’t exist anymore as you look at the answer but I think the relevancy there because it’s a lot more customed towards probably the question I’m trying to answer I get it at the end of the day whatever you’re searching for in the search window search engines do this all the time this is nothing new from a age of answer question right I would imagine there’s going to be like let me let me say this way this is potentially the precursor to co-pilot living in that help window

11:13 co-pilot living in that help window right so first first step would be index all of your Microsoft documentation Second Step would be index everything on the community for that is solved once you have those two things done the next thing I would logically see would be like slap a co-pilot on top of it and then just conversationally talk about your questions and it says here’s all the like so to me this is like co-pilot getting ready to show up in the help window is how I see this that’s where I would see this going honestly I I would agree with you I would or I would say

11:43 agree with you I would or I would say that would be a good place for co-pilot I would say this would that would be a great place right but this is this is the thing I’ve I’ve discussed this with Alex Powers before in the past he’s it feels like every time we want to throw co- pilot at something it’s just a a fancy search engine it’s like let’s throw co-pilot at something like I can’t find it so throw co-pilot at it well you find it so throw co-pilot at it well we already have search engines that know we already have search engines that are good at indexing things is that really the right solution for this but well but but think about it in terms of like even what’s happening on the the browser windows now right when you search for something you get a very

12:15 search for something you get a very clear like response that is sourcing from like p pop article or top articles right or like the first two or something like that yeah yeah so it like if you’re there is value there right you’re curating like I have this question I’m curating your response and here’s the links if you want to go click on them right or it’s taking the you like so it’s it’s summarizing I think and that’s where co-pilot or something could

12:46 where co-pilot or something could could very well en enhance that experience a bit yeah I feel like every frustrated AI engineer Microsoft’s like yeah Mike we’ll just slap a co-pilot on it it’s just that easy oh yeah no problem no problem let’s just slap another one huh thanks I don’t know what they do over there for Co I don’t know it’s just what are you talking about Tommy it’s just it’s one thing yeah it’s just another GPU is going crazy right now so let’s just make me Nvidia make more money so I do see here

13:17 Nvidia make more money so I do see here tenacious data or sry Donald says I want a Kimble co-pilot I’m like yeah yeah I do like put slap a co-pilot on your model view that’s what I want hey this isn’t hey this table has a bidirectional relationship we don’t recommend that can you make that t-shirt got a problem slap a co-pilot slap a co-pilot I can’t actually maybe that doesn’t S no that’s probably not appropriate I don’t think we’re GNA aggressively attack a co-pilot I don’t think I don’t think she’ll like that or he’ll like that I don’t know I don’t

13:47 he’ll like that I don’t know I don’t know what it is but I will say this though I do have I didn’t wear my shirt had I known I would have worn my co-pilot told me to say it shirt so I think that’s that’s actually very relevant cuz I think in the future here we won’t even humans even talk to each other we’ll just talk to the co-pilot which will’ll then talk to another co-pilot would then be read by another human so best podcast episode ever yeah it’s going to be great by the way fun little keyboard shortcut on edge control shift period brings up co-pilot really I know you’re trying that right now I’m trying it right now on my it’s gon it’s

14:17 trying it right now on my it’s gon it’s you’re I’m not going to do it because Tommy’s gonna make my computer shut down with this this shortcut this is something else that he’s trying to trick me they cause erors trying Tommy’s trying to trick me funny it’s the instant blue screen button Windows shift shift period awesome any other news or announcements things that were announced recently feel like there was something else that I was just talking with people about that was relevant and I can’t remember let just check the blog here on the fabric oh I know it reminds me

14:49 the fabric oh I know it reminds me Alex Powers has just announced a new updated release road map powerbi report file so I just saw it I think show up on Twitter recently so I will summon the powers of Alex powers and I will try to snag a copy of the report and we’ll try to get it up on the community site so jam. power. tips and we’ll update our road map because we have a powerbi report that Alex made previously that we just publicly share it’s just available

15:19 just publicly share it’s just available for anyone who wants to use it but we’ll try and put his report up there for you so you can go have a single URL easy to go look at for all of the information news around powerbi actually this is a good plug Tommy I’m still adding a lot of Articles to our j. Community. powerbi tips website actually I need to feed a bunch in there we’ll talk about it after but the search feature there’s gonna be a lot of Articles coming here pretty soon it’s about to be about 1, 200 yeah so

15:49 about to be about 1, 200 yeah so okay well for those who don’t know or who not aware we have jam. par tips it’s a website that we run and one of the the tabs on there is called called bookmarks and so these bookmarks are if I find an article that’s interesting if I find something that’s important I tag it and it creates it creates a groomed list of every single news article blog anything anything we can find or put our hands around that are related to powerbi Tommy and I well I have been Tommy will be we categorize every topic or article

16:20 we categorize every topic or article that comes through it’s like here’s the things that are listed with Dax power query the power. service visualizations administration of fabric things developer so I there is so much content from people being created I have a hard time finding a single news place to go find everything so this has been my go-to is where I store information and Tommy will be storing and and Seth also there too but we’re all trying to collect things that we think are important and last episode we had episode 329 from the podcast and

16:52 episode 329 from the podcast and actually I put the article on there I highlighted a couple lines in the article that were important so this potentially could be your new new news feed if you wanted to to check any of the hot news coming out and we’re going to try and catch all the the relevant articles that are out there so let me I’ll throw that in the window here as well just in case you want to go get it while we do that I’ll talk about the major update coming with that so I also own raindrop since you’re the owner I can’t search it but they’ve just made a major update where they’re adding AI to raindrop because AI everything because

17:22 raindrop because AI everything because co-pilot on everything CU co-pilot and everything copilot so they’re working on their tags or their topics where this automa automa tag for you so I’m going to go through all my articles I’ve saved add that to yours and basically what we’re going to get is the searchability on this feature is probably the reason why you need to bookmark this yourself if not just for the creation of what we have probably now well over 10, 000 articles I I would assume at this point or I know at least 9, 41 is we’re at we’re 9, 41

17:54 at least 9, 41 is we’re at we’re 9, 41 tagged and and C T to be over 10, 000 there’s going to be over 10, 000 short Y and that’s obviously not something you’re going to read chronologically however the search feature will also search the content of the article not just the title so if you’re looking for anything that’s amazing if you’re looking anything for calculation groups not only do we have topics that are hashtags but you can also search the relevant content of that article as well it’s so cool it’s it’s literally like a search it’s like a Search tool just for

18:24 search it’s like a Search tool just for powerbi it’s just yeah again I don’t really go as say find articles I see I see people post things on LinkedIn or or or Facebook or whatever I’ll immediately tag them and my first thing to do is drop a tag save it that way I can have it for later and this will be great when that’s starting to search the entire article because then I’ll be able to like just randomly search everything so anyways it’s a great tool there is a search item already on jam. power. tips so you can go there right now you can use a search icon you can look for

18:54 can use a search icon you can look for keywords now only if we could get that search element to be inside the power. com that would be good that my my one thought we talked about we talked about the the portal getting additional search from Microsoft docs and from Jim yeah I would it would be nice if I could also add Internal Documentation there as well or internal links that would also be referenced because one of the one I would definitely add would be Dax do guide I

19:24 definitely add would be Dax do guide I think I think they should they should let me as an administrator add additional search area and they should go out and index that page and go find all the relevant things on that page and that way I could be like hey go to Dax dog guide and that’s that’s the sqlbi version of Dax documentation it is by far the most robust the most complete the most clear as what’s going on inside Dax I will actually use that over the actual Microsoft documentation for do for Dax because the examples I think are

19:54 for Dax because the examples I think are just much cleaner and more understandable in that in that tool that’s closer being your Secret Santa than it is being a feature in Microsoft that’s maybe true I have to wait a lot a long time until till that one occurs we got we got some worth at least $20 so worth at least $20 all right so I’ll throw that in there for anyone else who’s interested Dax guide if you don’t know about Dax dog guide you do need to learn about that one it is by far the most helpful tool for learning about and writing Dax it’s from the team of

20:25 writing Dax it’s from the team of sqlbi team so awesome with that I think we’ve rambled enough on our intros any other intros or we want to just go into the article or actually the news oh I’ll have a Rand just not today okay right I’ve I’ve got I’ve got some fun experiences with other tools I’ve had to getting engaged with we’ll save it for Tuesday the agenda right now let’s save it for Tuesday I like it all right Seth you want to do us a the favor here and give us the Seth reader voice and

20:55 and give us the Seth reader voice and let’s read this mailbag this mailbag comes from we have no idea what’s the purpose of deployment pipelines and git integration when data flows in parenthesis direct Lake semantic models and reports are not supported I’m struggling to get my head around it and to convince management that fabric should be the data platform of choice I’m reverting back to good old

21:22 of choice I’m reverting back to good old good I’m reverting back to good old golden data sets to have some profit of a test prod pipeline all right well some of this I feel is correct some of that I feel is not correct there’s some correction here maybe maybe interpreting had it is is the tone here yeah I’m done washing my hands of this this trash that isip all right so what what are we frustrated at here fellows what what’s

21:52 frustrated at here fellows what what’s the what’s what are we talking about what feature I I don’t know talking well we’re talking about two Fe features I think yes soy pipelines correct and get integration they’re specifically deployment by pipelines with the direct Lake semantic models okay so that’s one thing I want to qu quantify there because there’s specifics in here yeah because models and reports are supported in deployment pipelines right and they’re they’re supported in just a pro

22:22 they’re they’re supported in just a pro and premium per user experience and in the fabric experience so you can move those through lake houses are also also supported notebooks and pipelines are also also supported it’s it’s when you get into the details of what am I what am I trying to do in Dev testr where I think things start falling apart a little bit for me I think we’re going to get there but again it feels like a lot of this still feels like it’s in preview well big part and I think so the frustration here is sounds like the user here again

22:53 here is sounds like the user here again we’re going to just say the user or a a Avid fan of the podcast is utilizing G integration using Azure devops and deployment pipelines and it’s unable to do so because it’s not supported yet with a direct Lake semantic model other semantic models are supported but utilizing a direct Lake semantic model is not supported in these features and that’s really the Crux of this because we’re getting pushed and I don’t want to say we’re getting pushed but the

23:23 say we’re getting pushed but the suggestion from Microsoft documentation the best practice it sounds like if you’re using a lake house you want to use a direct Lake semantic model rather than import rather than direct query however if I’m trying to do the developer experience or the Allin developer experience I’m unable to it’s not supported I can’t use appointment pip lanes and I cannot use git to do so or use both at the same time and this is interesting because I

23:53 time and this is interesting because I I’m going through this both taking a quick step back I’m thinking of deployment pip lines and I’m thinking of get and I’m like from a almost from a a larger point do I need both because they’re both developer experiences right deployment pipelines and get one’s more developer focused but at the end of the day day they’re both from a developer point of view and do I need both and I’ll pause there with with these features if

24:24 pause there with with these features if I’m using git is deployment pipelines then essential so if you look at the doctor so let’s let’s talk about a couple things here like so let’s talk about what git is supposed to do and then where is it kind supposed to do and then where is it explained from Microsoft how it’s of explained from Microsoft how it’s supposed to run and I think I think this in here is where the story is not quite refined yet we need some more help from Microsoft to finish out the feature in my mind so I believe when you look at the Microsoft documentation everything that you’re saying needs to be checked into git is

24:54 saying needs to be checked into git is you have a workspace that’s attached to the git repo the git repo attaches itself directly to your Dev environment so in Microsoft documentation oh thanks Seth you have a little arle I’ll put this out here as well the the best practices in cicd and this is where this is what I’m referring to because there’s a little image I think in this one is this the one that has the image yeah does not there’s there’s another one that’s up this article there set there’s another one out there that has a

25:24 there’s another one out there that has a an image of like how Microsoft expects you to use git Microsoft does not expect you to use get on dev test and prod so the idea is the development environment and realistically if you think about the people that are doing deployments in your deployment pipeline pieces you’re going to have a de a Dev team your Dev team should only be focusing on the dev environments your Dev workspace or reports your Dev workspace for semantic models that’s all that’s the only you only really need one git integration for that environment once you have published changes into that get repo and you’re done that’s T

25:57 that get repo and you’re done that’s T that’s hanging off of the main branch of your git and then what Microsoft explains is okay now that you have the artifacts inside your development environment now move it using deployment pipelines Downstream to your other environments up to 10 because you can have as many as you want now so if you look at their pattern it makes sense if you look at their pattern as it relates to semantic models and reports it also makes sense because your semantic models can have different connection strings they’re detecting the

26:27 connection strings they’re detecting the connection string in the semantic model and allowing you to adjust it per environment so this this make it like a really technical here for a second but just bear with me the main key Point here is you have core code that’s going to define the semantic model you’re going to have core connection strings that will change per different environment so in any tool that you’re building anytime you have a connection to a source of data those connections are technically

26:57 data those connections are technically variables that are going to be adjusted As you move through the system so a a Lakehouse you need to have a Lakehouse in Dev test and prod that stores where your data is going to go the the infrastructure is the Lakehouse but the guid or the def or the information of the lake housee needs to be adjusted As you move through the different environments so every tool that you’re going to going to use data use data activator deployment pipelines data flows everything you use all of them

27:27 flows everything you use all of them have to be configured in a way that allows you to adjust to adjust the connection string in every single environment CU those are all going to be changing cuz when I move my semantic my direct L semantic model I need to just change the semantic model pointing not to Dev anymore I need to point to test so to me yeah everyone’s saying variables in the it’s all about variables variables are the thing that are supposed to make this work for you as you build your Solutions what’s not really supported very well today is you can’t really do this very well with

27:57 can’t really do this very well with fabric at the point and there’s this missing aspect of the connection strings are not parameters or variabl ased in a way that allows me to move it between the environments so to your first point Tommy how does this going to work right well per per Microsoft documentation they’ve got a very I think simple method I think what people really want is they want the git integration to be the source of truth yes they want the git integration to be okay the git repo will connect to prod to test and to Dev

28:30 will connect to prod to test and to Dev and you deploy directly through git using actions hey I’m done with my code I’m ready to deploy set code into a certain environment and this is actually more of a pattern of how actual software development or website or whatever else app development works is you’re going to have many different branches off of your well not many you’ll have main branches and other branches as well and you’ll either tag things as a development build or you’ll make different branching names and those will all attach the different environments so that’s typically how people are building these

29:00 typically how people are building these things so I don’t think the get branching World 100% alignance to with what we have today in fabric so I can make the argument we we had appointment pipelines because we didn’t have G back in the day like deployment pipelines were created because there was no get it’s the business user you weren’t we weren’t bringing a bunch of developers into powerbi. com yeah and that’s but that’s also part of what that article points out right like the there those

29:31 points out right like the there those two solutions are separate within the ecosystem yes right if you try to use them both at the same time you’re going to run into problems pretty quick and Mark Mark leld wrote a pretty good blog about like what are the scenarios in which you’re trying to use those integrated together but I think it is like those two things are separate for different audiences to try to like segment out and make sure that what you’re you have a test environment you have a prod environment what’s what’s

30:02 have a prod environment what’s what’s interesting like two points are interesting around this if you think about when when we actually got deployment pipelines or now the ability to integrate kit like these are way late in the game right like like the the heavy developer focused tooling you the heavy developer focused tooling for the hardcore Geeks of RBI but know for the hardcore Geeks of RBI but now ever increasing larger implementations of powerbi right are are just getting there

30:34 powerbi right are are just getting there I I would almost say to the listener like like reset your expectations a bit because what we’re what you’re taking is stuff that’s coming late to the game and then the brand new features and saying that they’re not supported like direct like right semantic model like yeah it’s always going to come later for one in all the tooling right like you’re not going to have parody full support for stuff right out of the gate yeah I I gotta push back on this though go ahead

31:04 gotta push back on this though go ahead and and because if you’re going to push these two features you’re going to push direct Lake semantic models and you’re going Microsoft where all their doation sure because because now all of the sudden it’s very very relevant right all of this is and don’t don’t get me wrong like in the in the functional bits like you go into the documentation of all this it’s like there’s a lot of limitations still that I agree have to

31:37 limitations still that I agree have to be removed in order for you to fully leverage the capability of of that tool yeah for sure and we completely agree there they’ve been pushing it since they announced it last year when they said we are moving towards the direct Lake mode of data integration and power btic models that was from build three when we all fell out of our chairs and if you’re going to try to

32:03 chairs and if you’re going to try to support the we’ll call it the gold standard or what the golden calf so to speak what we’re all trying to no not that not that no wrong analogy the golden the trophy the trophy thing we say the first place the prize the prize that’s what I meant thank you thank you the prize that Microsoft’s aiming towards and you’re pushing the git integration and you don’t support that major feature the the the prize that they’re going towards this direct

32:33 that they’re going towards this direct the preferred method of integration in a semantic model we’ll say that’s probably the best way to say it and we don’t support that that’s but Mike okay did didn’t Mike just outline for 15 minutes like the key parts of git that still aren’t there that don’t that make it don’t we can’t fully integrate yet no so we can integrate but it doesn’t support directly semantic models it supports normal semantic mod models understood okay but aren’t there still problems with variables or switching between

33:03 with variables or switching between environments and things yeah that’s fundamental to making this work right so like I I get it right but you can’t like think you’ve lasted this long developing powerp no that’s not true I downloaded a visual studio in 2017 just because all of the other changes right that have happened that Microsoft has pushed his features doesn’t mean that they’re all integrated into get cuz they’re not or all integrated into deployment pip points cuz they’re not so so to have to all of

33:33 cuz they’re not so so to have to all of the sudden be like hey we have direct Lake and smack detic models and then assume that it’s automatically going to get added and supported into these feature like it no it’s not because this is a coming way late I know and that’s a fair argument and I think and I don’t want to be pounding the same conversation we’ve been having over and over because we’ve had a few mailbags this is not our first mailbag about or question from our Avid five listeners about hey there’s a lot I want to do in fabric four now

34:03 lot I want to do in fabric four now because we don’t agree with this so hey what I’ll take four but the we so we’ve had a lot of other mailbags too ones we haven’t even talked about on on our on the podcast yet that have dealt with I really want to dive into fabric but there’s preview features I’m unaware of and there’s features that I I we don’t want to make the mve yet because it’s not all there I don’t want to talk about the same argument I don’t want to keep hammering that but no and but it but

34:33 hammering that but no and but it but it’s it’s the same answer yeah as as the last time which is is deployment pipelines Andor get integration but deployment pipelines more for business users or people using the interface is is production ready for use cases where it’s production ready right like if you are going to leverage an a feature feature that is offered within the fabric ecosystem that is not supported in

35:03 ecosystem that is not supported in deployment pipelines and it doesn’t work for you then don’t push them don’t use them you can’t use them right like something in your it’s the same thing as fabric ready yet that we talked about it depends on what your mus haves are for for your business are you just pushing it because it’s technology or are you pushing the new new latest thing or is it hey it’s core and we can’t use it yet we we’re going to figure out a different way to solve this problem which we have for as long as we’ve been playing with powerbi

35:34 long as we’ve been playing with powerbi so would we encour or I guess with the user who says I’m reverting back to Golden data sets or golden semantic models or maybe he’s going back to data sets going back to Azure analysis Services as a standalone product so I I guess from our conversation what I’m hearing from our dialogue here is that we would encourage him to go back to the gold semantic models no I’d encourage you not to try to push into something that isn’t supported so like is there frustration

36:06 supported so like is there frustration because he needs a feature or wants deployment pipelines or get integration to support direct leg semantic models well of course yes it’s that’s an idea push it like do we agree that it should be in there absolutely it every if I’m using something for code change or deployment strategies within an organization and you’re going through the efforts of training teams how to use that yes I would expect the platform to

36:38 that yes I would expect the platform to support all of the different ways I want to deploy something or manage something in the fabric ecosystem so because it’s there like not there now doesn’t mean it’s not going to be there in the future right but it better be there in the future otherwise what’s what’s the point right so this is that’s an interesting point because I remember when I think it was like data flows one and a half or data flows gen two if I wanted to create that in an existing workspace Microsoft would give an error

37:09 workspace Microsoft would give an error or a notification saying hey if you want to create this newer version of a data flow this workspace can’t have any other old data flows you have to remove them or create a new workspace so it hindered me from trying something without in a sense setting up my environment in the right way it was frustrating but what eventually occurred was my works I wasn’t left with a workspace that had some things that worked and some things that didn’t and that notification really H hindered me

37:39 that notification really H hindered me at first but then made sure ensured that that environment was set up properly right now and I think we’re a lot of the frustration we’re getting from our own listeners and my own personal frustration too with some of the items trying to use git is it says sayy set up git you’re good to go and then immediately see unsupported unsupported unsupported you don’t get that notification it doesn’t say before you actually try a feature or when they say something’s available to use you don’t get that notification without going

38:10 get that notification without going heavy deep into the documentation on just trying it out and then finding yourself yeah and and I do want to I do want to recognize the that there’s shared sentiment here right when when new features come out and the ability for us to deploy things in a better way that is good for our organization the the the way I look at

38:42 organization the the the way I look at that is I want to use these tools they will be awesome and great for the organization I’m very frustrated that I can’t right now because of those two things right like when you identify one of of these net new features that comes out or supported areas there is frustration because it’s like what the hell like I you’re putting this out there I’m I’m trying to like ride this wave with you and now you’re saying like

39:12 wave with you and now you’re saying like I can’t put these two things together so so there is frustration there absolutely I agree with that one and I I’ll keep pointing out here some things here as well in my perspective on this is I’m coming from as your data Factory I’m coming from synapse those were two cloud-based tools that started being built from the ground with Git integration already in place and so to deploy those things so couple

39:42 deploy those things so couple things here that I’m my mind is coming from is when we look at like how the platform has evolved and where it came from right those two azure-based Services were deployed through scripts you could have Json objects that deployed that pattern of Dev test prod environments that was already built into the product from day one so because of that yeah it may have been o it may have been a little bit clunky like doing some doing a synapse Dev test prod is actually quite complex because they do exactly what we

40:13 complex because they do exactly what we need to do to override different variables in the pipeline to say okay this semantic or this this this synapse environment will point to this key volt this synap this synapse environment will point to this storage account and then as you move things through you provide these things called overrides where in that environment it has this is the environment but then you’re able to override portions of Json to change the connection strings As you move through different environments

40:43 move through different environments right this came from a world of we have to support git from day one where I think this is coming from now in fabric we’re getting to the place where more Enterprises want to use it and fabric is trying to be more of this Enterprise grade solution but it’s coming from and I think actually katos bi does a great job explaining this right a lot of what’s built for powerbi is starting at the the Pyramid of users of of fabric the larger user base is individuals and teams so a lot of the features we’re

41:14 teams so a lot of the features we’re seeing have been built for individuals and teams just using powerbi. com what we’re seeing now is we’re seeing that the enrichment or the development of moving more into department and Enterprise grade solutioning and so it’s like this idea of like well we have all this in my mind if I’m a developer right I’m looking at this going wow there’s so much Tech debt around git that isn’t there and what we’re seeing is Microsoft slowly chewing through all of this Tech dep this is why we see the new fabric API showing up

41:45 we see the new fabric API showing up this is why we see items as a new item inside the fabric API where you can go read like I don’t I can edit a report without actually opening the report now I can call an API get the item out observe all the objects make the changes I need and put that information back and have the report automatically updated this is all coming from the pbir and the pbip format that ru’s been working on so we’re heading the right direction it’s just in my mind I see a lot of massive

42:16 just in my mind I see a lot of massive Tech debt that’s getting erased here or having to be reworked so that we can get all the features that we need and you all the features that we need and it’s it’s fabric it’s new I know it’s it’s fabric it’s new I think it’s a bit harder to figure out which features are in quote unquote preview right there’s a lot of things that just don’t work yet because it’s not there yet but I think we’re heading the right direction and when we were at the fabric conference in Las Vegas Amir made a very bold statement on stage he said we are

42:46 bold statement on stage he said we are going to have get integration handled for all artifacts and all assets you for all artifacts and all assets in six months and that was from know in six months and that was from whenever he said so we’re getting like April so yeah we’re getting further down the road here for my birthday running out running so I it’s recorded it’s on YouTube you can you can have you can hear him say that he’s he really wants to push for that portion so I think those so to me that just gives me some affirmation that the the emphasis

43:16 some affirmation that the the emphasis is we’re going to do get we’re going to figure out these supplment pipelines it’s going to happen right now I think we just have so many teams trying to Grapple with this Legacy bit of tech debt and and yeah we’re getting pipelines but that was from Azure data Factory but they don’t work the same and they’re different so they’re rebuilding the code base so a lot of learned things are coming forward that are different so to me I feel like yeah we’re going to get there is it frustrating right now yes yes it is do I stop developing in fabric

43:46 is do I stop developing in fabric because I just can’t do it and it just probably not I’m probably in in this is like what we I felt like we did when we started seeing powerbi to begin with there’s all these like well I can’t get visual to do that well here’s my workaround to make it have rounded edges on a bar and a bar chart like there’s there’s things you can do to tweak it so that it worked it’s not ideal but you can get this the effect that you want by doing some workarounds and I think I’m very excited for this because we’re at a cusp we’re at the top I I think

44:16 cusp we’re at the top I I think if we’re if we’re not at the top of the hill we’re getting close to the top of the hill we’re I think we’re going to really start seeing some momentum picking up here because of ru’s really hard work around the PB and VIP format once some of this starts clicking off we’re going to see John kky like writing 10 blogs a day at some point here because he’s gonna now so like boom it up man ramp it up he’s been way the cicd stuff as it is and so we’re just going to see what we’re

44:46 we’re just going to see what we’re seeing here is I think we’re going to see patterns very quickly emerging here around integration Oppo pipelines cicd what will the patterns look like and so we’re not there yet we’re probably a bit I’m probably a bit too early to the story because I can see the vision of where I want this to go I’ve done this before in other Solutions so I understand this idea of like overrides and how to do things I’ll I’ll fall back to my main point here in software development we probably worry most about infrastructure and the

45:17 worry most about infrastructure and the code that runs on that infrastructure in software development you probably focus a little bit around what is the data in those different environments but sometimes the data is what it is you don’t really do a whole lot of like or checking with the data pieces inside a business intelligence realm though we have three distinct things we care about we care about the infrastructure that the things run on how many semantic models do I have how do I deploy them then we care about the code inside the semantic models or the lake houses or whatever and we care about the data inside each of those environments so one

45:49 inside each of those environments so one thing that has not been very clearly articulated to me is let’s imagine you have Dev testr in a pipeline I may need to have a process that goes into I’m deploying some changes to engineering code it requires me to delete and replace my data there’s no actions in a deployment pipeline that says refresh and reload all data after the artifacts have been deployed that is typically a thing that would show up in Azure devops you would actually build in tests and post

46:20 tests and post production activities that would then do this cleaning or scrubbing or hey I I need to delete the last three months of data and then reload it because I’m doing something new whatever the thing is these are processes that need to exist inside this pipeline so if we’re building deployment pipelines we need to be very cognizant of the infrastructure the code that runs on the infrastructure and we need to be cognizant of the data that’s in the infrastructure and so we need something to help us manage all three of those things yeah I don’t think Microsoft gets the the data side of

46:50 Microsoft gets the the data side of things I don’t think they understand it I don’t I I feel like the conversations I’ve had that they’re everyone asks a lot more questions of like why do you need this I’m like it can’t you just see it like I just need I need a process to let me like clean out and and refresh data if needed in when I’m doing deployments I would on the positive side right and you’re talking about how powerbi devs right pushing for the cicd right is the last group of the pyramid right so it’s been a little slower I but with fabric

47:22 little slower I but with fabric realistically The Clash is coming from the opposite direction now which is probably why it’s getting a lot more love and will be and or air even like voicing that he’s pushing for it yeah is because th this is backend tooling right the these are services that have fully baked Solutions and folks will not move into fabric unless they get the same experience related to that that’s a good point and I think that’s where you do see the acceleration of why it’s going

47:52 see the acceleration of why it’s going to get implemented in fabric faster well and I think it’s important not you you mentioned men this Seth and I think it’s really important to note you’re this podcast and the people who are listening are all more on the nerdier side we’ll say or are testing speak for yourself I’m super cool I’m not nerdy at all go go rewind about 45 seconds architecture I I don’t think there’s been a nerdier thing the last 20

48:26 episodes that’s hilarious dial Tommy style the comment is exactly on and when I went when I went to build I had other people who were like from the powerbi team like Patrick and Adam Alex Powers Patrick from gu Cube they kept saying I I was telling them about things I was learning like oh this is so cool it’s so cool and they’re like Michael you are purely you are 100% a developer like you you are thinking like a developer like we’re not that we’re powerbi we’re not developers I’m like oh

49:01 again another nerdy another nerdy reference that was the that was the from Minecraft villagers that you may hear if your kids are playing Minecraft a lot oh I think we’re having all the evidence we need but oh man I’ve been I’ve been found out I was trying to hide I I was a secret nerd and now I’m not the man who has a QR code on his T-shirt oh my god oh okay yeah yeah you’re right Tommy P back here well but

49:31 you’re right Tommy P back here well but most people who are the the normal business users not going wait gets part of powerbi I can’t wait to learn that they’re probably moving towards the deployment pipelines if that and that’s probably somewhat intimidating for them but I think so that’s that’s a really good point though I think to mention that the majority people are not looking at get in deployment pipelines as they be all end all however let me I’m going to say a statement and I want to see if you agree or how much

50:01 I want to see if you agree or how much you agree with the statement for the the more the developer side of things the more on our end the power users or the power power users because not all features are spored get in deployment pipelines Mo the majority of what we’re doing in fabric should be on the testing and experiment side not production would you agree slightly disagree or not agree at all midle agree

50:31 disagree or not agree at all midle agree I guess I think there are certain things in in fabric that are very much production ready right I think pipelines super solid getting much better to use there’s features that are getting added there like it notebooks pretty solid I’d give it us a good 80% of the way there seems like it runs well it’s getting they’ve been on major improvements for like usability and being able to run things so I think that’s really a solid part of that all the semantic models that you normally have used powerbi reports semantic models all that stuff’s very solid so a

51:02 models all that stuff’s very solid so a lot of things there are very good where I struggle right now I really struggle right now with data flows data flows gen two is just not cutting it for me it it doesn’t there’s some very I think simple features that need to be there that are just not being captured I need a pipeline to send a parameter into a data flow period I need a data flow Gen 2 to be supported in a pipeline period like these are these are to me these are should have been done day one with data flows it’s and to your point

51:33 with data flows it’s and to your point Tommy right or my point earlier other tools were coming from this git world this pipeline deployment world that should have been the starting point of like okay we have data flows gen one let’s start and the first feature should have been we’re going more towards the git and deployments and this is what’s going to happen fix that feature first start with that at the core like take that part of azure data Factory that should have been the very top feature on your backlog get that

52:04 top feature on your backlog get that first then work on parameters and then do all the other enhancements that you’re talking about fast copy all these other things that are in there to me those two priorities are like number one and number two and once you get that done correctly everything else just falls into place and it becomes much easier to use them so I think those to me are like the maybe they’re maybe they’re much harder than I perceive them to be to be I I could totally just be wrong on this but I feel like you already had this working in data in data Factory like why didn’t you just take that bit of code

52:34 didn’t you just take that bit of code over like it was working and now we don’t and now it doesn’t work so to me that’s like things mean that was again that’s not data flows but that’s data Factory but anyways you get my kind data Factory but anyways you get my Point like there feels like there’s of Point like there feels like there’s some core pieces that should have been there so to answer your question directly Tommy do I do I use in production yes I do I use portions of fabric inside production do I steer away from certain things yes I do right so things I steer away from I steer away from data flows for now MH gen two at

53:04 from data flows for now MH gen two at least I’ll take data flows gen one because that still works just fine you can move them through a pipeline you can still make tables across the environments there I’ll take pipelines I’ll take notebooks I’ll take semantic models all those things apps there’s a lot of things that still just work like normal that there’s been some enhancements too do I use deployment pipelines yeah I probably do

53:26 deployment pipelines yeah I probably do probably I like I like them from a ease of migration and movement are they perfect for what I need as a developer no they’re not but are they perfect for me as a business user yeah probably they probably mean a lot of the needs so that’s my perspective on it Seth what do you think like if I’m all in on fabric well are there part are there portions of fabric that you’re like let’s productionize it and ones that you’re like I off on it I that you’re like I off on it all of us are in production mean all of us are in production fabric true very smart Seth very good

53:59 fabric true very smart Seth very good right so yeah we’re just talking about aspects we talk about the G integration yeah well no I’m just saying like from from my perspective like is it is it use do you get more out of the the new features of Hab like the the ecosystem for net new things yeah right could they be production ready well if they work yeah right like it it’s it not everything is super convoluted or complex or whatever like so if you’re leveraging the platform and it works for you awesome am am I personally no not

54:32 you awesome am am I personally no not yet not to any like heavy degree because it’s going to take shifting a team to support internal things in a completely different infrastructure and I have a very large infrastructure on other tooling right other services that that I’m not even close to doing a comparison or like a a net migration yet around things because because no there there are aspects of fabric that aren’t ready yet but it’s I think important to

55:04 yet but it’s I think important to always keep your you always keep your what I’m going to screw up some know what I’m going to screw up some analogy always keep keep focused on the the features and things that are being supported much in the same way that we talked about in episode 327 is fabric ready yet it depends on your business cases and what what the business needs are that you have current Solutions and services that are running to support those those if fabric when fabric offers better Solutions or equivalent Solutions then yeah you you take a a much closer look

55:36 yeah you you take a a much closer look at it and say hey over time like this is the direction we want to go still supports all the business needs still gives us a lot of flexibility but you gives us a lot of flexibility but simplifies this whole experience know simplifies this whole experience for what the internal users or whatever use cases there are and then more and more people move towards it so for the the bank bang for your buck you’re going to get with people using is my my guess is all the net new stuff I’m want to highlight a word that you said because I think it’s a very relevant word that really I think I don’t want to

56:06 word that really I think I don’t want to say sums up but is really integral to understanding fabric right now and I think it’s comparison where if you’re trying to take your current workflows and processes that you’re doing in powerbi and then just compare it exactly in fabric you’re probably going to find yourself in some frustrations because it’s not not a copy paste directly of data Factory it’s not a direct copy paste of data flows or git as we know it it’s getting there but I don’t think

56:37 it’s getting there but I don’t think it’s necessarily even intended to be if you’re trying to compare your current Enterprise or deeply product production based workflows and say okay now we’re just going to immediately move that to fabric you’re probably gonna gonna find some things to that’s come up to the surface that don’t necessarily meet your requirements however I think and I I’ll definitely lean on this as well just because my main production workflows right now don’t have the

57:07 workflows right now don’t have the requirements or don’t have those feature sets in fabric that I would like or to make them work doesn’t mean that I’m completely foregoing fabric if anything I would say a lot of things I am testing out or trying to in a sense migrate into fabric is a lot of my focus right now because because if if it’s not a direct copy paste or duplication of a process there’s a lot that it can solve for just the lak houses alone right now are are have

57:37 lak houses alone right now are are have been huge again Mike and I agree I want to do mention on the data flow side it’s so close it’s such a great user interface to push data in but I I agree Mike it’s just there’s again if I’m trying to directly copy and paste it’s not there yet maybe or directly there yet for all my specifications but man 85% 90% of what I can do I can do in Fabric and I can do very well yeah I think the the thing to

58:07 very well yeah I think the the thing to point out here again is we’re we’re talking about fabric as it evolves like this is Microsoft’s direction for business intelligence across the board and all their data platform so just because we have these frustrations or these current things that are like yeah still blocked can’t get there quite yet right right like give it give it a month give it three months give it six month like they’re all in right so I think inevitably so are we we’re along for the ride right we’re just figuring out when

58:38 ride right we’re just figuring out when is the best time to take the plunge or integrate deeply with it to leverage all of the the the good stuff that it’s offering as a platform already right okay I want to I agree with all these steps here and I while you guys were talking I did do a quick deployment on a I think a directly enabled yeah talk about nerdiest podcast ever here we go I think I just did a test on you were discussing why why did Mike get

59:08 were discussing why why did Mike get quiet all of a sudden well he was doing something on the side all right so doing an appointment pipeline I had a lakeh house with a default semantic model and I would agree the default semantic model doesn’t go along for the ride the lake house is the only thing you can deploy across different environments so a default semantic model will not deploy across environments however if you create a semantic model based on that lake house then you can deploy a direct Lake enabled semantic model just not the default one so I am seeing my

59:41 not the default one so I am seeing my direct Lake enabled semantic model that moves through but again back to my earlier point you don’t get the data so like if you so if you’re if you’re hydrating your lake with tables that are coming from a data flow gen two you don’t get that like you can’t do that stuff like so you’re not going to be able to load the lake so to me it’s like totally pointless if you use a data flow and you’re hydrating your lake with some tables from some location and you don’t have that data flow in the next

60:11 don’t have that data flow in the next pipeline I have to go to the old workspace export the the the flow and then put it back in and then go back and repoint everything to the correct like all this stuff is what it needs to be fixed inside the deit pipelines so at the end of the day it does feature does work again with a lot of these caveats and I think maybe the complaint here in particularly this question is I don’t know all the edge cases like right th this is this is the problem right work around we we see it it it works but like oh it works but I

60:41 it it works but like oh it works but I don’t have this little Fe I don’t have this little thing I don’t there’s all these little like nitpicky things and you’re like yeah it works but do it exactly like this and then it will work and so like I think that’s what I’m I’m I’m also griping about here is like it should just work more un ially across all the artifacts I think we’re going to get there I think the I think the effort from Microsoft is pointing to we’re going to fix it it’s going to get there so I’m very positive about it and honestly I’m loving working in fabric it is very impactful for businesses it

61:12 is very impactful for businesses it does things a lot faster I was just talking with another Microsoft CSP and they’re like we’re seeing companies do things in days and weeks that used to take months because we’re not waiting to procure things and things are just turning on and we’re able to move data around and we’re centralizing and and the idea that the workspaces are sharing things so much easier to PE different across different teams and people like spot on like I’m so I really feel the excitement around Fabric and I do think it’s really making an impact for businesses I think this just means you

61:43 businesses I think this just means you businesses I think this just means we need to be a bit more patient know we need to be a bit more patient and again I said this when it came out in November last year I said you’re going to need to give it some six months to a year or for it to really start picking up some pace and we are it’s it feels a lot not like the very early days of powerbi when every month we got you of powerbi when every month we got tens 20s hundreds of updates every know tens 20s hundreds of updates every single month all all the things that are happening so I’m very encouraged by that it it feels like we’re heading the right direction all right with that we’ve definitely eaten up all your time today

62:13 definitely eaten up all your time today thank you very much chat for being very funny and interesting and having great comments thank you very much chat we really appreciate you if you like this please and as Kratos bi would say subscribe to the most n podcast you can so we would love it if you would hit that subscribe button or or the Bell then when we have new content also if you like this and you felt this was a Val a valuable conversation for other people to hear please share it on social media or people at your business we’d love to have other people engaged in conversation this is our fun

62:44 engaged in conversation this is our fun Community thing that we do every week so we really appreciate it we really thank you for participating and and providing some wonderful feedback Tommy where else can you find the podcast you can find us in apple and Spotify or wherever you your podcast make sure to subscribe and leave a rating it helps us out a ton you have a question an idea or a topic that you want us to talk about a future episode like today head over to powerbi. com

63:25 will be adding I’m gonna updates literally throwing you on the bus right now cuz now you have to do it cuz I’m saying on the podcast it’s Now official it’s in the world you have to do it you have to make sure that that it works ready to get over your jamm to get Jam jam out jammed full jammed full of new articles articles or great ending guys great ending way to go awesome we’re so good at this awesome thank you all very much and we’ll see you next time

64:06 you [Music]

Thank You

Want to catch us live? Join every Tuesday and Thursday at 7:30 AM Central on YouTube and LinkedIn.

Got a question? Head to powerbi.tips/empodcast and submit your topic ideas.

Listen on Spotify, Apple Podcasts, or wherever you get your podcasts.