PowerBI.tips

Category: Articles & Opinions

  • How to Represent Your Power BI Skills

    How to Represent Your Power BI Skills

    This is part 3 of 3 in a series designed to help Power BI users and enthusiasts. The first post focuses on identifying the dizzying array of skills that make up the Power BI ecosystem. It was created to help you take a personal inventory and assess your current skills. The Second post focuses on providing ideas for building learning plans and putting that base assessment to use. Finally, we come to this post where I’m going to focus on some key areas for how you represent your skills when the time comes. This post is for job seekers, for career movers and anyone else that finds themselves in a position where you need to represent your Power BI skills effectively.

    Be Distinctive

    All of us are unique. You have so many different qualities and passions, and your experiential knowledge is one of a kind. The many years it’s taken you to get to this point, the schooling, the challenges you’ve had to overcome, the boss that made your life miserable, the aha moments, and big successes are all part of who you are. There is SO much there…and you have a piece of paper, and possibly 30 to 60 minutes to convey that in an interview. How do you do that?

    The simple answer is, you don’t. You do all that before the interview. You take those experiences. Take those challenges you’ve overcome. You embrace getting outside your comfort zone. You assess your skills, you set a plan of action, and you grow. It is through that commitment to action that you will grow in an area you are passionate about or hold interest in. The old adage, show me who you are by your actions is what sets you apart and will increase your chances of landing your next big job. If you are different, be different first.

    Learning and growing will show well by themselves based on the answers you give. However, you can easily take this to the next level by showing what you’ve done. Build some reports and share the public links in your resume, start a git repository and store your stuff there. You can also add links to Community activities or blogs. As someone who hires people, I know this is the first thing I look for. This gives you an advantage because you show who you are before we even talk.

    You are One in a Sea of Resumes

    Regardless of the position being looked for, the reality is that you are a single candidate in a sea of resumes. I’m not a recruiter or human resources people finder. I’m don’t know all the different techniques you can use to make sure your resume pops up on someone’s screen. Here is what I can tell you based on experience looking at hundreds of resumes over the years. Firstly, there are certainly key words related to tech skills that I look for in order to find candidates. This should make sense. If I need a Power BI developer, I’ll be looking for Power BI in your resume. However, that just gets me to your resume, it doesn’t sell me.

    Is your resume going to stand out? Not in a bizarre way, but have you really thought about how to convey your skills without writing 6 pages? Here are my top recommendations for increasing your chances of going from resume to interview.

    Top Recommendations

    1. Stack rank your skills. It should be abundantly clear somewhere what you are the best at and what you only have limited exposure too.
    2. Don’t put every single program, operating system and application you have ever opened on your resume.
    3. Condense your experience down to the most concise wording. You are putting your experience down to convey your knowledge not describe all your job functions in detail.
    4. Have you blogged? Do you have a community user account where you are actively helping people? Are there any published reports of yours to look at? Do you have anything to show that you are different?
    5. Do you have an ending that outlines what you are currently learning?

    It requires effort to be distinctive. Adopt a learning mentality and let that shine and set you apart in your resume.

    One of the best resumes I’ve seen had a link to encourage me to look at a Power BI report. The candidate built this to represent their skills. It was the longest I’ve ever spent looking at a resume. It showed the candidates skills, technique and understanding of how to put together a well polished report. They followed that resume with a solid interview where the technical skills in the resume aligned with the conversation. Instant hire!

    Do Not Embellish!

    A counter point has to be made right after pushing you to think about how to set yourself apart. I am not suggesting you embellish. One of the absolute worst things you can do is mis-represent yourself on your resume. Land an interview, and then display that you actually don’t have any of the experience that you said you had. Writing it down doesn’t make it reality. I love rating scales, its one way you can easily articulate your technical skills on a resume.

    A business will have different needs, and it isn’t everything. For instance, you might know nothing about Power Query because you work in enterprise areas for data movement and shaping. However, you do have a ton of Modeling/DAX. That could be a perfect fit for an enterprise or more technical role. The converse is that a business unit may only need simple Modeling/DAX because all their issues revolve around connecting to, cleaning, and shaping data in Power BI. Those are two completely different skill paths.

    Why Not Embellish a Little?

    One of the challenges we’ve identified already is trying to convey who you are in a short amount of time. If there are huge differences between the way your resume conveys your technical skills and the way you can represent them in conversation you just took Trust off the table. And that will likely kill your chances of getting hired. All that work to get this far will get instantly flushed.

    Another reason is that you may not know about all the other areas of need a company has. Just because you may not be a good fit for this role, the hiring person may pass your resume around to other parts of the organization. You may have the skillset that a colleague of theirs is looking for.

    Understand the skills the position needs

    Was the job description specific? Did it give you an idea of what skills were needed? Could you figure out whether the job was going to be business facing or more of a development role? If you said no to any of these questions, be sure you bring that up right away in the interview. Clarity around the type of position is really important to understand where the focus of questions should fall.

    Pay attention to the details in the job description and focus on the areas where an organization is placing emphasis. If you don’t have skills in Power Query and the job description stresses that as a main area of expertise, you might want to pass on applying. This goes back to not embellishing. Just because you have focused a lot of your time in Power BI doesn’t mean that you would be able to perform all areas as an expert. The level of job, the requirements they are asking for and the years of experience are good indicators of whether or not it is the “right” Power BI job for you.

    Be Honest

    This is without a doubt the number one make or break thing for you in an interview. Just like embellishing, this will instantly kill your chances of getting a job if you aren’t honest. What do I mean by this? Here is an example that you might not think would qualify, but it does.

                    Question: We’re in need of someone with really good Power Query skills. Are you familiar with Power Query and have you processed data through it on a regular basis?

                    Answer: Yes, I know Power Query very well.

                    Follow up: Great! How can I transform the data type of a column?

                    Answer: Well, actually… I’ve read about Power Query but I do all my transforms in SQL…

    At this point its likely you just flipped the switch. An interview is so much more than just the technical things you know. The interviewer has a limited amount of time to get to know you, and even in a technical interview they are looking for all the key things that they would want to see in a team member. Examples like this erode, or destroy, the trust/honesty element. Right or wrong, an interviewer will take this information and apply it to other scenarios.

    What answers like this represent is there won’t be an open dialogue, and a manager could have a new resource committing that they know everything. This would likely lead to over committing or missing deadlines. Either one is a recipe for conflict and a bad relationship. Be honest when you represent your Power BI skills.

    Be Inquisitive

    There is nothing better than having a dialogue with individuals around topics. As mentioned above, hiring managers are looking for a lot more than just what you can recite. Are you asking clarifying questions? Do you follow up with a question of your own, or talk through your thought process? Did you come prepared with questions on the company? Did you inquire about the team, and the direction that the company is headed. What is the work style, do they work under heavy process and procedure or is it the wild west. What does a day in the life of this job look like for you? All the questions you bring to an interview, show the interviewer that you have a vested interest in the company and the team you would be working on.

    Be Yourself

    The resume opens the door. The interview is the initial meet and greet, and any follow up meetings would be closing the deal. You wouldn’t have the interview if you didn’t appear to have the skills that a company needs. For all intents and purposes you should feel pretty comfortable, provided you have the skills you represented in your resume.

    Bring your personality to the interview, engage as much as you can with the interviewer to let them see the side of you that you would show at work. This is important for a couple reasons. First, you want to show the interviewer a glimpse of the type of person you are. Without that, it can be hard to gauge whether or not you would fit with the team or wouldn’t. The other thing to keep in mind here is that you are interviewing the company, just as much as they are interviewing you! Show a bit of yourself in the interview to make sure that the company is one that you would want to join and you think you could be successful in.

    Extend Thanks, Request feedback

    Wherever possible, follow up with the recruiter or interviewer to extend thanks for their time. Hopefully it was an enjoyable experience for all involved regardless of whether or not it worked out. Extending thanks to people for engaging with you should have gone both ways, but you’ll never lose when extending a bit of closing goodwill. Wherever or whenever possible, if you don’t get the job, request feedback. Knowing the reasons is invaluable to you in your next interview or the job you apply for.

    Do you lack certain skills, did you convey something that you didn’t mean to convey, was there a candidate stronger in a particular area. There is a huge disparity sometimes between how someone reads us vs. what we are trying to show. Getting this type of feedback is constructive because it leads to introspection and tweaking how you present yourself or a skill area you need for that particular job role. Other times, there could be no skill difference but a different candidate presented themselves in such a way to make the interviewer feel it was a better team fit.

    Representing your Power BI Skills

    There are so many roles and jobs that you can apply for now. There are also one’s you can focus on for future goals after you learn more skills, and build more experience. While I can’t make any guarantees, I can say that these tips and recommendations come from interviewing many Power BI candidates. These steps outlined above are key areas that will make your job hunting, your interviews, and your future interactions with your next career move a more positive one.

    This wraps up the 3 part series that I wanted to complete to bring my insights and experience to all of you. This last post was the first one I wanted to write, but I couldn’t bring this forward without the first two. If you missed those, be sure to go check them out (Skills, Learning). This is the last stop, understanding your skills and adopting a learning mindset should be your first focus. Those set the stage for being successful personally, in an interview, or anywhere your career may take you.

    If you like the content from PowerBI.Tips, please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel, and follow us on Twitter where we will post all the announcements for new tutorials and content. Alternatively, you can catch us on LinkedIn (Seth) LinkedIn (Mike) where we will post all the announcements for new tutorials and content.

    As always, you’ll find the coolest PowerBI.tips SWAG in our store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • How to Build Your Power BI Skills

    How to Build Your Power BI Skills

    One of the biggest improvements you can make to your life and career is embracing a learning mentality. If you are here, I assume one of your interest areas is Power BI. This post is the 2nd in the series to help individuals identify where your current skill strengths are, and where you have room for improvement. (The first post introduces the Power BI Skills Matrix and is designed to assess your current state.) After reading that you can dial in on your strengths and weaknesses and your now ready to start creating learning plans to build your Power BI Skills.

    Therefor this post is a series of steps that I have used, still use, and remind myself to use, to stay on track. This isn’t a training manual. It is a compilation of ideas that you can use to formulate a plan of action and stick to it. I’ve carved this into four sections for easier digestion. First, some tips for any level. Second thru Fourth focus on different levels. Beginner, intermediate, and expert.

    For Everyone

    Pick a Time and Schedule It

    The struggle to do this is real! In order to remain consistent, we have to be diligent in carving out time for ourselves to learn new things. Figure out how much time you want to devote to learning and schedule it. Literally, put it on a calendar and make it part of your routine. Not doing this results in large gaps in learning. Losing traction on the things you did learn. Or even worse, regressing instead of progressing.

    Pause – seriously. Take a minute, figure out if its 1 hour or 24 and carve out the time in your schedule. Open your device/notebook/calendar and write it down.

    Don’t Just Read it, Do it

    This relates to almost everything in life. When you actually do something, you understand it infinitely better than just reading about it. I like to separate out my learning time in two ways. The first, starts with reading about concepts or ideas that I can jot down and reference when practicing it later. Visualization is a good example of this. Reading books, blogs, etc can uncover a ton of different methods, theories and approaches to building great visualizations. This requires time to digest these conceptual aspects before taking a direction. Only after we process it, can we start testing something out or re-enforcing our learnings by doing.

    However, the second way is much more applicable to almost all of Power BI, and that is learning while doing.

    • Connecting to Different Data Sources
    • Cleaning & Shaping Data
    • Modeling
    • DAX
    • Visualization & Properties

    All these are best served by getting dirty right away. Have the Power BI Desktop open all the time when you are working on new concepts and learning new skills. Power BI makes this quick discovery so much easier now! You can immediately access and load a sample data set right out of the Power BI Desktop. Click Try a sample dataset

    Select Load sample data

    Load Sample Data

    Choose financials and Click Load

    Don’t just read tutorials, walk through them with the author. Don’t just find books that talk at you. Find those that invite you to follow along, or test out what you are being taught. Always, Always DO IT!

    Share It

    One of the most rewarding aspects of learning new things is sharing those new learned things with other people. Not everyone wants to devote all their time to speaking, blogging, making videos and interacting with the community at large. However, sharing what you’ve gone out of your way to learn with other people is very rewarding. Whether it is on the job or in community forums once in awhile. Not only that, sharing helps others out in ways you will never even know. Sharing what you’ve learned and experienced is the ultimate pay-off for continued learning. Investing the time to learn, then sharing that with others, makes all the struggles worth it!

    Beginners

    Have FUN! Embrace the Experience

    There is a lot to learn, but don’t start the journey of building Power Bi skills thinking you should know everything. Just like you, we all started at the beginning at one point in time. The great thing about you starting now is that there are more people that can teach you things! One of the best ways you can start the journey is by joining the Community. Hop on over to community.powerbi.com and create an account. The forums and linked content in that single space alone will set you up for success.

    I would also highly recommend joining a local User Group. Interacting with other Power BI users that are local to you is a great way to network. Joining the community and starting a Power BI User Group were the two single things I did that had a huge impact on my professional life and career. These communities get you engaged with others like you and open many doors.

    Start with the Basics

    Download Power BI Desktop and open it. Keep it open as you look for new things to learn. Within the tool is a link to guided learning. Click Help and you will see easy links to a multitude of MSFT created material.

    Combine these quick links with the sample datasets we outlined above. You can see it is extremely easy to get started. Now you can test out the things you learn without the need to have your own dataset curated.

    Find Your Learning Style

    One of the best things you can do is figure out what methods of learning work best for you. Is it video? Presentation style or tutorial. Do you love books and will never give up the feeling of paper? Or do you need your tidbits in smaller chunks like posts, blogs or community posts? Try them all out! Once you sort that out, then dive into those sources for information. I could link to many of my favorites, but I don’t want to find your favorites for you. I want you to find them on your own. The only blog I will mention is the Power BI Blog from MSFT. This is where you will find all the latest news, and is the first thing you should subscribe to.

    Use the Skills Matrix to build a plan of action

    Take some time to figure out what learning path you want to head down first with the Skills Matrix. What is the most relevant skillset for your job, or the job you want? What have you struggled the most with? Do you want to know a little about everything, or do you want to focus on one area before others? Most of these questions can only be answered by you. Hopefully the Skills Matrix will help you figure out those paths. You might also find some helpful pointers in this Post by Steve Campbell. He outlines some different roles in a data solution.

    What I cannot stress enough though is this. Understanding the fundamentals in Power Query, Modeling and DAX should be your first goal! It is important to get familiar with those areas before you start diving deep into anything else. Once you understand the basics of those areas. Dive into visuals and the properties and methods used to create good looking reports. There are so many great features that you can use to produce the best end user experiences.

    Find Content Creators you like and Follow them

    Why? Because this is one of the easiest ways to stay up to date on new things. It also serves as a reminder to keep on your learning path. Doing things alone all the time can get boring. You are also more apt to lose focus on your learning goals. Finding someone or some platform that you enjoy will be a subtle reminder to keep it up! Its also really easy to take a break and check your latest YouTube notification or twitter/Instagram feed. People are always posting relevant things you can check out.

    Intermediate

    Refine Your Skills

    Now that you’ve had some time to build your Power BI skills, you likely have an area where you spend more time than others. DAX is likely front and center in your universe, and if its not, put it there. Without a good understanding of DAX. How it filters and shapes data, you will always be battling presenting the right data.

    This is the point where the learning cycles get a bit more involved. You’ll be diving deeper into your interest areas or the areas you have to solve problems for all the time. If you clean and shape your data in Power BI. Maybe diving deeper into M and using the advanced query editor is more valuable. In either case, make sure you spend time refining your craft. Spend time to learn the languages above and beyond what you use daily. Also start to learn the tools that will help you troubleshoot and figure out performance issues in these areas.

    Understand the underlying technologies

    Power BI is the culmination of technologies that MSFT has had for many years. You may have some idea of this already. There has been a ton of work done to make sure Power BI has the full enterprise features from its originating products. Therefor, knowing how all these technologies work together is important. Especially something like Analysis Services. Understanding these concepts and tools is key to understanding how to work and interact with them. What does O365 have to do with Power BI? What is Azure AD? Where do I purchase licenses depending on the business needs? Why are there all these External Tools and how do I use them? All these are questions you should have an idea of how to answer.

    Expand outside your comfort zone

    By this time, you have an area of comfort. It might be visualization, it might be administration, or could be DAX. Push yourself to get into the things that don’t come easy to you. We all fight with the “fear of failure”, and learning new things falls squarely into that “fight or flight” response. Push past that and be sure to take little bites of those challenging areas. The more you embrace, rather than avoid, those challenging areas the better off you’ll be.

    Expert

    Stay Current

    You already know building your Power BI skills is important. However, as you become more and more familiar with all the tools, languages and infrastructure its easy to become relaxed. Conversely, you may be getting overwhelmed with work and outside pressures. In some spaces taking a break might not be a big deal as releases are months if not quarters apart. However, Power BI continues to barrel ahead with change after change every month. It is important to stay plugged in to the community, events, blogs and the latest news that impacts your learning world. There is no end to refining and adjusting solutions and implementations of Power BI within organizations. Here is your reminder (and mine) that its crucial to stick with your learning schedule and stay engaged rather than just reading headlines.

    Find Outlets to Share

    If you are an expert in your field you are likely providing major value to one or many businesses with your expertise. As I mentioned above, one of the most rewarding aspects of accumulating knowledge and experience is being able to share that with others.

    Some easy paths to engagement would be answering questions on the Power BI Community, or other online forums. Start a User Group and/or engage with your local group more. Try to blog a bit. Just because someone wrote about a topic somewhere in the world, doesn’t mean that you shouldn’t write yours. Alternatively, try to hone your skills to the max by biting the bullet and creating a presentation. I guarantee, nothing will lock in your learnings more than having to teach someone else how to do it. This is where your local User Group is important! It’s typically a smaller venue that can be used to hone your speaking skills. Get past your negative inner voice, just get something started and find out what path you like the best.

    Go Deep or Widen your focus

    There will always be something to learn, but you will certainly start to see things in a different way. One of the hardest decisions can be to choose to go really deep on a specific area, or focus your attention on the ecosystem as a whole. The driving forces here may be a career path change, or a future goal you wanted to achieve. In either case, the same principles you used to get you here will take you to the next level, keep it up!

    Start Building Your Power BI Skills

    Wherever you stand in the spectrum of the skills matrix, you will always have something to learn. Carve out the time, figure out a game plan, and execute on it. Building your Power BI skills and embracing a learning mentality will not only help you find a great job, but it will help you grow in your current one and expand into others that you only dream of right now.

    The absolute best thing about owning your own learning path is that you take that knowledge with you wherever you go. Self learning provides you with a deeper understanding of how to solve problems regardless of datasets. This pulls you out of the “I only know what I did at school or in my job” mentality and gives you ownership of those learnings. It allows you to explain them, and talk about broader solutions. This alone will set you apart from a large swath of individuals that don’t have, or embrace, a learning mentality.

    It is my sincerest hope that each of you reading this embrace a learning mentality and make it a part of your life.

    If you like the content from PowerBI.Tips, please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel, and follow us on Twitter where we will post all the announcements for new tutorials and content. Alternatively, you can catch us on LinkedIn (Seth) LinkedIn (Mike) where we will post all the announcements for new tutorials and content.

    As always, you’ll find the coolest PowerBI.tips SWAG in our store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • Power BI Architecture in a Data Solution

    Power BI Architecture in a Data Solution

    This article will focus on Power BI architecture within a data solution.
    In this context, Power BI architecture describes how Power BI can slot in as a piece of this strategy. For instance, this includes not only the reports, but the data retrieval, storage and machine learning involved. Next, it discusses different roles and responsibilities involved. This can expand on Power BI skills, looking at the entire solution.

    In addition, we hope it to provide ideas for current developers looking to expand Power BI skills or change directions in their career. It can provide a look at areas of need in organizations and give thought of learning opportunities available.

    Power BI is Greater than a Report

    In my previous article, I discussed how Power BI should not be thought of as a separate product to ETL, AI/ML or overall data strategy. Rather, organizations need to include Power BI architecture as part of a data culture with all of the products working in union.

    As a recap from the last article, a modern data platform typically has 4 steps:

    • Load and Ingest – extract the data out of the source system and transform it.
    • Store – Land this data somewhere so we can run analysis on it.
    • Process (or transform) – Run analytics on your data and draw out KPIs, AI and predictions.
    • Serve – present this data in an easily way for stakeholders to consume it.

    Medium Size – Power BI Services

    Power BI dataflow Architecture

    It is possible to implement a reporting strategy entirely in Power BI. First, we can load data using dataflows. Next, these can be stored as a dataflow in the service or backed by Data Lake Gen 2 storage. It is good practice to separate our Power Query models and reports.

    Large Size – Azure Services

    Power BI azure synapse architecture

    Sometimes, we want to use more services than just Power BI. This may be due to huge datasets, use of data in other applications, or writing custom machine learning. The above diagram shows an enterprise scale reporting solution. Azure data factory can move and transform the data. Afterwards, we can store in a variety of storage options, depending on the nature of the data. Many options are available to run machine learning on the data. This ranges from custom code to autoML. Lastly, we create data models and we can produce reports off them. This integrates the Power BI architecture into a whole reporting solution. There is also a path for streaming data – through Event hub and Stream analytics.

    Azure has a host of services available. If you are new to these, it can seem a lot to learn. Luckily, Microsoft are rolling out Synapse! This includes a portal that houses many of these services, enabling you to use them all in a single place. If interested, Nicola Ilic I has a great series on Synpase and Power BI.

    Different Roles in a Data Solution

    If we want to design a data culture, we often need more roles and skills than just designing reports. The below list looks to identify some different roles and responsibilities in a data solution. This is not intended to be a fully comprehensive list. Rather, we explore some different and common roles that could be involved within a project.

    It is unlikely a single project will require all roles. Usually, one person may take on two or more of these roles. Instead, the aim is to distinguish different areas of the data strategy. This can help us to view Power BI in the bigger picture, seeing where it fits in.

    We will look at the following architecture – a company that uses Synapse as well as dataflows. For simplicity, we are not looking at any streaming reports. In this diagram the dataflows storage is implied to make the diagram easier to understand.

    Power BI architecture in a data solution

    Each role will look at some common responsibilities and skills. It will also highlight the area in the above architecture diagram that they are responsible for.

    Power BI Developer

    The Power BI Developer is responsible for building and owning data models for KPI and user reports. Therefore they will spend large parts of their time modeling and transforming data in Power Query or dataflows. In addition, a BI developer has good understanding of tabular models and how to write custom business logic in DAX. They may also be required to set up the Power BI architecture.

    Skills

    Expert in Power Query and DAX. Familiarity in tabular editor and DAX Studio. Plus knowledge on database designs and tabular modeling such as implementing good STAR schema. Great Power BI skills and ability on Power BI desktop.

    Roles of a Power BI developer
    Roles of a Power BI developer

    UI/UX Engineer

    In larger projects, report design can benefit from a UI/UX specialist. Power BI developers will often complete this, but specialists can be brought in to help design layouts, flow and brand consistency within the project.  Usually, the Power BI developer will look after the models and logic. However, the UI/UX Engineer helps make sure the final reports are professional looking. In addition, they can be involved in storyboarding the design of reports. A UI/UX specialist is needed for projects with many reports, external facing reports or embedded solutions.

    Skills

    Design skills as in products such as Adobe Illustrator. Dashboard design (such as gestalt principles).

    Roles of a UI/UX developer
    Roles of a UI/UX developer

    Data Engineer

    A Data Engineer is responsible to get the data from the source and load it into Power BI. Smaller reporting projects may use only dataflows or power query, but larger ones might require more steps. The Data Engineer will move the data using tools such as data factory into a SQL database or Synapse storage. This allows larger enterprise solutions with massive volumes of data, or for complex machine learning to be performed on the data. In addition, Data Engineers will transform and clean the data making it suitable for reporting analysis or machine learning. They will then integrate this back into the solution.

    Skills

    Tools such as Azure Data Factory, Stream Analytics, Data Lakes or other data storage. In some projects a Data Engineer might work exclusively with dataflows or Power Query. Databricks, Spark Analytics and SQL are important for prepping and transforming big data. Engineers also can benefit from automating in languages such as PowerShell.  Strong Power BI skills.

    Roles of a Data Engineer
    Roles of a Data Engineer

    Data Architect

    Data Architects are responsible for designing, managing and maintaining the reporting solution.  Architects will suggest the best selection of tools and methods used. They would be the ones to recommend the team involved, the technologies used and the correct approach. Overall, they will set up the Power BI Architecture and machine learning solution.

    Skills

    Architects need to have vast knowledge and experience across all the technologies that can be used. This includes deep knowledge on Azure or Synapse, Power BI and data governance methodologies. They have solid understanding how machine learning can be integrated into a solution.

    Roles of a Data Architect
    Roles of an Data Architect

    AI Engineer

    You may incorporate an AI Engineer in large projects that have a machine learning focus. Data Scientists spend large amounts of time writing custom algorithms. In contrast, AI Engineers use tools such as Azure cognitive services or Azure machine learning studio. On large projects they may work in tandem with Data Scientists helping to merge their code into the reporting system. They often spend time cleaning, transforming and prepping data. In many cases, this job is regularly undertaken by Data Engineers. However, projects with a heavy AI focus may benefit on having an AI Engineer or both a Data and AI Engineer.

    Skills

    While AI Engineers may not write custom algorithms from scratch, they need a solid understanding of machine learning principles. They also need to know how to prepare and clean data ready for machine learning solutions.

    Roles of an AI Engineer
    Roles of an AI Engineer

    Data Scientist

    The data science field has exploded in popularity over the recent years. Typically, a Data Scientist will spend their time cleaning, prepping and analyzing large volumes of data. Next, they write custom algorithms that can detect deeper insights. A Data Scientist often has years of experience and training coming from various backgrounds. Data Scientists write custom code in Synapse, Databricks or Apache Spark notebooks.

    Skills

    Expert in an analytical programming language, typically R or Python. They will have a unique blend of programming skills and statistics. Deep knowledge of designing and implementing different machine learning algorithms. In addition, they will be proficient at cleaning and preparing data.

    Roles of Data Scientist in Power BI
    Roles of a Data Scientist

    Conclusion

    As we can see, there can be many different roles involved in a data solution. Many times, one must wear many hats. Other times, organizations can benefit from having several specialists in different areas.

    Microsoft is looking to unify the Power BI architecture and overall data solution through Synapse. This portal will make these roles easier to be completed by fewer people within the same portal. Still, there will always be a need for specialists. So if you are looking to expand your power bi skills, or finding new areas to expand into, make sure you familiarize yourself with these.

    If you like the content from PowerBI.Tips please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel.  Or follow us on the social channels, Twitter and LinkedIn where we will post all the announcements for new tutorials and content.

    Introducing our PowerBI.tips SWAG store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • Power BI is part of the greater data solution

    Power BI is part of the greater data solution

    Power BI is a powerful reporting tool that has been dominating the market and rapidly evolving. Yet, in many organizations people seem unaware of its true potential or core purpose. As a result, too often it is deployed to simply extract or visualize data points in an ad hoc reporting manner.

    Power BI is greater than a report

    Power BI should not be thought of as a separate product to ETL, AI/ML or overall data strategy. Rather, organizations need to include it as part of a data culture with all of the products working in union.

    To deploy Power BI successfully, do not use it to simply design reports. Instead, design a culture and architecture. This is one that allows business users to understand, interpret and react to rich and powerful data driven insights.

    The many additional products, services and capabilities that come packaged in Power BI are too frequently overlooked. As a result, people see only the top level – visuals in reports and dashboards. But there is a whole host of rich and exciting features below the surface.

    With that, here are some common mistakes I have frequently seen new users make when rolling out Power BI.

    Mistakes made to under utilize Power BI

    • Using it for Data extraction
      Large tables with a selection of filters that you may or may not look to export. Instead, Power BI is designed for trends, insights and cross slice and dice. Large tables and data dumps do not give insight.
    • Using it for a data visualization to tell a single point
      Design a visual that can convey information quickly, rather than an infographic type solution. If you are looking for that pixel perfect data visualization for a news story that tells a specific point, there may be other options. Paginated reports or predesigned Excel documents are viable options. Design data pipelines that are regularly updated. Create visuals that are designed to be interactive. This will help users drill down and find insights.
    • Ad hoc only reporting
      While this can be a great tool for ad hoc reports, you may be underutilizing and doing extra work. Instead, build reusable data models that are designed for multiple reports. Write DAX business logic and KPI that can serve as a single source of truth. Be sure to document your measures inside the data models. By clearly documenting measures data consumers will understand how to use the data model to build new reports.
    • Current reporting tool / Excel replacement
      A common request is to “lift and shift” all excel reporting into Power BI. These products are different and have different uses. If you are moving to Power BI, don’t try and recreate old solutions. Instead, a better approach is to design new reports that play to Power BI’s strengths. Utilize the rich features and powerful engines that make Power BI beneficial. This is a story of it’s better together. Using just Power BI or just Excel has it’s advantages and dis-advantages. Conversely, using both Power BI and Excel can play to each tool’s strength.
    • Not building a data culture
      Matthew Roche has an amazing blog series on building a data culture with why and how to do this. Building a good data culture is vital for adoption within the organization. The data culture will start with an Executive sponsor who can push for adoption. So, first and foremost, be sure to have a leader who believes in your vision.

    Mistakes made when deploying Power BI solutions

    • Focusing on raw numbers, not business insights
      Instead of simply displaying numbers, great reports often have the following KPI, trends, drill down, interactivity and slicing capabilities. This allows business users to gain meaning information about the direction for the business.
    • Ignoring the deployment approaches
      Many business users are familiar with a typical process for reports; a user submits a ticket to IT. IT writes a bunch of SQL queries to get the data for this request. They then surface the data in tables and simple graphs. In contrast, Power BI does a great job at breaking down this long turnaround and getting the data in users hands quick. An organization should deploy a top-down, blended or bottom-up approach. As a result of utilizing this approach, they can merge the business and IT side of operations and remove silos.
    • Failing to Think like the Business and Act Like I.T.
      The I.T. organization has many strengths related to how to make data available quick and reliably. Power BI is mainly designed for business users. Thus, Power BI has features that borrow from best practices from I.T. One such best practice is the use of Deployment Pipelines.
    • Not utilizing Data Models or ignoring self-service reporting
      Data models, as described in this blog by Matt Allington, contain all the metadata needed for reporting. This includes the business logic and data transformations. However, creating and maintaining these can be time consuming. Instead, it is possible to reuse data models and keep one source of the truth for many reports. The modeling experts can own and maintain the models. Furthermore, business users can connect and build their own Power BI reports utilizing the models. This is done without even needing to write a single line of code.
    • Treating Power BI as a stand alone product, not part of the greater data or AI solution
      You should not treat Power BI should as just a visualization tool (read this blog by Gil Raviv). Instead, Power BI is a business insights tool, a way to serve and communicate the information within the organization. In addition ML and predictive analytics are baked into it, as are ETL processes, data storage and security. As a result a unified approach to a data culture should be built. Users from all business areas need to be aware of the strategy.

    Using Power BI the right way

    Power BI should be unified and part of the entire data stage – not a visualization layer on top of it. A modern data platform typically has 4 steps:

    • Load and Ingest – extract the data out of the source system and transform it.
    • Store – Land this data somewhere so we can run analysis on it.
    • Process (or transform) – Run analytics on your data and draw out KPIs, AI and predictions.
    • Serve – present this data in an easily way for stakeholders to consume it.

    Power BI can be all of these steps. From a single report using power query (Load and Ingest) to import data (Store). Next, you can build a model and DAX measures (Process). Lastly, you can surface the data in visuals on the report pages (Serve).

    This can be a more enterprise level solution and scale well too. Firstly, Dataflows are set to extract and transform data from many sources (Load and Ingest). You can back-up and store in a data lake gen 2 storage (Store). Secondly, the data can take advantage of automated ML (AutoML) and cognitive services. Build DAX expression over them, combining a powerful DAX language with the power of AI (Process). Last, you can package these as reports, dashboards, apps or embedded into other applications (Serve).

    Alternatively, Power BI doesn’t have to be all these steps. A traditional data platform architecture is described by Microsoft in the picture below. You can utilize other tools such as Data Factory to Load and Ingest data. Next, you can use Databricks to Process/Transform the data. Power BI and Analysis services models will serve the data to the end user.
    This is a great example of Power BI fitting into a greater data solution. However, you should implement the deployment with the entire solution in mind. Power BI is not as a tool for simply creating visuals. A good deployment is deeply rooted in the culture. Each step must consider the others in the pipeline, not sit in silos.

    Source: Microsoft

    Bonus: See this great diagram by Melissa Coates, showing Power BI end to end features.

    Azure Synapse

    Microsoft is expanding this ecosystem with Azure Synapse. As they roll it out, they are designing data engineering as a single platform. This combines this entire pipeline and tools into a unified experience. Power BI being a part of this platform.

    Source: Microsoft

    Synapse provides Consistent Security

    When we think about user level security, Azure Active Directory (AAD) is the gold standard for access and security for organizations. Synapse leverages this technology to remove friction between different azure components. You can leverage AAD across the multiple services for data factory, Data Lakes, SQL and Spark compute as well as Power BI.
    The experience of governing data on a user by user basis improves with the Synapse experience.

    A Low Code Data Engineering Solution

    There are many Azure components you can use to produce a well engineered data pipeline. Azure Synapse brings all these tools under the same portal experience. For example, using Azure Data Factory, then writing data into a data lake. Picking up the data and querying flat files with compute engines such as SQL or Spark. Azure Data Factory also has built in features that can simplify data lake creation and management using mapping dataflows.

    More Computing Options

    No longer do We have to choose just SQL or Spark, rather We have options. We can use Provisioned SQL which was previously Azure Data Warehouse. Synapse now offers on-demand SQL, and Spark compute engines. This is where we are really seeing the technology move to where we have separated the storage layer from the compute layer. This means Azure Data Lake Gen2 serves as storage, and SQL and Spark serve as compute.

    One Place for all information

    Whether it is Azure Data Factory, Spark, SQL or Power BI. Synapse has now become the single portal for integrating all these services. This in general simplifies the experience and management of all your data pipelines.

    If you like the content from PowerBI.Tips please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel.  Or follow us on the social channels, Twitter and LinkedIn where we will post all the announcements for new tutorials and content.

    Introducing our PowerBI.tips SWAG store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • MVPs React October 2020

    MVPs React October 2020

    Horary! The Power BI desktop for October finally arrived and it is packed with tons of updates. I’m super excited about this month’s release. We rallied the troops and have a ton of MVPs talking about the latest release of Power BI desktop

    Microsoft Blog Post

    Click the link to the official Microsoft blog post.

    Watch MVPs React

    If you like the content from PowerBI.Tips please follow us on all the social outlets. Stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel.  Or follow us on the social channels, Twitter and LinkedIn where we will post all the announcements for new tutorials and content.

    Introducing our PowerBI.tips SWAG store. Check out all the fun PowerBI.tips clothing and products:

    Check out the new Merch!

    Hasta La Vista Data
    Go Ahead Make My Data
    PBIX Hat


  • MVPs React September 2020

    MVPs React September 2020

    This month we got the Power BI desktop update a little bit later because it was released during the 2020 Ignite conference. As always the MVPs are super excited to talk about the latest Power BI features.

    This month’s release was packed with all sorts of new features. For a full list of features read the official Microsoft Blog post. Some of our favorite features that the MVPs talked about are:

    • Search for a workspace during publishing
    • Premium Per User (PPU)
    • Maintain Visual Layer Order
    • Databricks Connector
    • New Icon

    Watch the Full video

    MVPs attending this event

    Alice Drummond – https://www.discoverei.com/

    Benni De Jagere – https://www.linkedin.com/in/bennidejagere

    Daniel Marsh-Patrick – https://www.linkedin.com/in/daniel-m-p/

    Marc Lelijveld – https://data-marc.com/

    Reid Havens – https://www.havensconsulting.net/

    Seth Bauer – https://powerbi.tips/author/sbauer/

    Mike Carlo – https://powerbi.tips/author/mike-carlo/

    If you like the content from PowerBI.Tips please follow us on all the social outlets. Stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel.  Or follow us on the social channels, Twitter and LinkedIn where we will post all the announcements for new tutorials and content.

    Introducing our PowerBI.tips SWAG store. Check out all the fun PowerBI.tips clothing and products:

    Check out the new Merch!

    Hasta La Vista Data
    Go Ahead Make My Data
    PBIX Hat


  • MVPs React July 2020

    MVPs React July 2020

    This month we are trying something brand new. We are introducing a new series called MVPs react. As you may already know Power BI has monthly desktop releases. If you are as excited about these releases as I am I love talking about all the new features. So, why not get a fun group of MVPs together to discuss everything.

    These events are similar to a fire side chat about Power BI among the best experts within the community. We hope you enjoy the conversation, and learn a couple new things as well.

    Here is our session for the Power BI desktop release for July 2020.

    If you like the content from PowerBI.Tips please follow us on all the social outlets. Stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel.  Or follow us on the social channels, Twitter and LinkedIn where we will post all the announcements for new tutorials and content.

    Introducing our PowerBI.tips SWAG store. Check out all the fun PowerBI.tips clothing and products:

    Check out the new Merch!

    Hasta La Vista Data
    Go Ahead Make My Data
    PBIX Hat


  • Embedding Paginated Reports

    Embedding Paginated Reports

    Over the course of time Power BI has come to encompass a wide variety of technologies and tools. One such product that has been integrated into the suite of Power BI is paginated reports. Before Power BI, paginated reports were served up in SQL Server Reporting Services. That product was the major reporting tool used by businesses using Microsoft products. Now, Power BI brings us the best in visualization, analysis and insight. Paginated reports give you pixel perfect outputs that can be easily printed, emailed, and output to Excel. It strikes me as a bit peculiar that over the years while the focus has been to push into Power BI visualizations heavily, almost every client I’ve ever worked with has always asked for outputs of Power BI reports that are best served up in a paginated way.

    What is really exciting is now we have an ecosystem in which we can bring all these report types together. You can create “perfect world” reports where we can analyze and export in fashions that meet the needs of a wider audience. Today I want to talk specifically about the release of Paginated reports in external embedded scenarios in the November feature release cycle. The integration of paginated reports has been on a steady cadence of releasing new features. I want to serve up an overview of current state for embedding these report types, and where there is room for improvement. I’m also interested in the community feedback to determine if my experiences or blockers are similar or different. Be sure to comment below.

    What you need

    First off, in order for you to use paginated reports in Power BI, you will need to likely upgrade you license. It is a widely known request that something should be done about removing this feature from premium pricing. If you want to change that, throw your weight into the request to change that, you can do that via the community ideas section for Power BI to make this feature available to Pro users -> https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/35959420-paginated-reports-please-make-it-available-in-pr

    Importantly, having Pro license won’t fix the cost for an embedded scenario completely, but one can hope that if it is available to Pro, then the A1 – A3 sku levels would work as well for future embedding.

    Lets get back on track! You will need an A4 sku or P1 in order to use the paginated reports. A full walk-through of how you can develop this solution can be found here. The great thing about an A sku is that you can turn it on and turn it off. This means that if you want to kick the tires and showcase the capabilities before purchasing, you still have the ability to do so.

    What Works

    Aw yea! Look at this beauty, you see that in the embedded sample report? PDF, Excel, Word, PowerPoint! All there, and all exporting the pixel perfect greatness of paginated. I don’t have to assume how much businesses will salivate over this one, because I’ve had all these conversations. With paginated reports in our external facing applications we have the ability to merge all these report types into seamless products for our business users.

    You want to test this out for yourself? You can! Check out the playground in embedded and you can see just how these features serve up in all your export dreams.

    https://microsoft.github.io/PowerBI-JavaScript/demo/v2-demo/index.html

    Here are the current data sources that are supported.

    Not surprisingly, all the Azure sources feed us up all the data & security that we would want to use in terms of access. This will support existing data extraction methods used for paginated in other reporting tools. There is great news from a Power BI model perspective as well! If you want to integrate your Analysis and paginated reports to serve off the same model this is really easy to do now because I can connect to AAS or a Power BI Dataset.

    What is Needed

    There are a few things that you should be aware of that aren’t yet baked into the feature release. This is where I’m wondering how much of a blocker it may be for others as it is for me. There are likely many companies that have hundreds or thousands of paginated reports. The majority of your reports are likely served up via stored procedures with parameters in the SSRS report. These provide data context prior to execution of the procedure thereby limiting the results. This WILL work if you are using Azure SQL DB or Azure Managed instance. It WILL NOT work for on-premises SQL or a Platform as a Service (Paas) solution where you are hosting your own SQL Server in an Azure VM.

    Export to Excel – Dump my data!… Personally, I don’t mind that this feature isn’t in the export options. I actually hope it doesn’t get added in the way it works in legacy tools. One challenge I’ve always had is trying to get business users out of using reports as export dumping tools. With the advent of Power BI, and all the ways we can surface up curated data sources, I want to see this type of practice stop. We need to get better at training people where and how to ingest data and make it easier for them. Rather than let them dump out loads of data and creating silo’d processes of their own.  

    Close

    I’m really excited that paginated reports has been added as a supported report type for embedding. Paginated reports will certainly hold a place in my future deployments. Additionally, I would love full support for on-premises data and Paas solutions in order for major migrations to occur of paginated reports into Power BI. For me, I will need to consider the cost of refactoring all those stored procedures if I want to stay on-premises or make a case for moving all of reporting only to Azure SQL.

    For more details on the release and links to the announcement you can find it here -> The Details: https://powerbi.microsoft.com/en-us/blog/embed-paginated-reports-in-your-own-application-for-your-customers-preview/

    If you like the content from PowerBI.Tips, please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel, and follow us on Twitter where we will post all the announcements for new tutorials and content. Alternatively, you can catch us on LinkedIn (Seth) LinkedIn (Mike) where we will post all the announcements for new tutorials and content.

    As always, you’ll find the coolest PowerBI.tips SWAG in our store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • Power Query – Get Started!

    Power Query – Get Started!

    As a user that builds Power BI reports, did you know the different technologies that come into play when you interact with the tool? This is one of the first questions I ask to new audiences or people I’m training on Power BI. Why is this important? The reason is that it is crucial to understand what part of the tool you are in so that you can separate out the different tasks that you should be doing or trying to accomplish in each area. The other biggest reason is you need to know what you should be searching for in your favorite web browser when you are looking for answers to your current problem.

    With that said, there are two main components to the Power BI Desktop, Power Query (aka. Edit Queries) and the Tabular Model that you have access to in the main part of the tool. I’ll mention the tabular model first, but we won’t be diving into that in this article. It is responsible for compressing our dataset and gives us the speed we see over all the data we want to slice and dice. We can create relationships in our model window, and we can create additional calculations to extend our original dataset by using measures or calculated columns. The underlying language in the Tabular model is “DAX”.

    model
    measures and calculated columns

    Power Query is our ingestion engine, it connects to our data sources and allows us to perform the ETL (Extract, Transform, Load) activities on our raw datasets. This is extremely helpful and an absolute must to create clean data, and shape it into the best form for loading our models that we want to work with in our reports. The underlying language in Power Query is called “M”. When you toggle open edit queries

    edit queries

    You are presented with a new window that makes a clean separation between the two interfaces. Connecting to or creating data creates a new query, clicking the “Advanced Editor” will open another popup window where we can see the “M” code for all of the steps we have taken in that query. Most of everything you do in Power Query will be in the interface as you get started, but getting to the point of understanding how to manipulate the code in the Advanced Editor will change the way you can build reports immensely.

    advanced editor

    The vast majority of Power BI users are extremely new to Power Query, so today I wanted to spend a little time talking about a book that has helped me immensely in understanding how to get the most out of that aspect of the tool.

    One of the best things you can do when trying to understand something is to get an expert to show you how to do it, whether that is in a class setting, a presentation or a book. They have knowledge around the area and can streamline your learning process. There are people who learn in different ways, but I would argue that each type has different levels of retention. For instance, when I go to a session that is heavy in coding and techniques, I take a bunch of notes because I know that while it all makes sense in the session, I’ll forget the specifics and have to refer to my notes when I need to apply what I learned. The same goes with books I read, I grab snippets of techniques and write down a bunch of reference things for later. Whereas, if I go to a class and have to walk through the steps on my own or take them away as homework, it forces me to practically walk through an exercise and the steps in order to complete it. Doing this locks the technique in, and I’m able to recall how to do it when I need it instead of having to look things up again.

    If you are serious about getting better at the ETL portion of your Power BI report building there are numerous resources out there, but today I’d like to spend some time talking about one in particular that I would highly recommend authored by Gil Raviv.

    (Disclaimer: Gil Raviv is a friend, and his book was gifted to me)

    Suffice to say Gil is one of the best to learn from since he was part of the MSFT team that created Power Query. To read more about Gil, check out his bio on his website here -> https://datachant.com/about/

    https://www.amazon.com/Collect-Combine-Transform-Business-Skills/dp/1509307958/ref=cm_cr_arp_d_product_top?ie=UTF8

    The main reason I’m recommending this method to learn is that it isn’t JUST a book. Think of this as an instructional class, where you have a bunch of material for both preparation and homework. What Gil does here is amazing from a sharing perspective. Along with the book, you are given access to almost 200 files that include the data sources and the Power BI Desktop files with the solutions built into them… Take a moment to digest how much content is in here, it is quite astounding.

    The layout of the book is designed with the business user in mind, and focuses on the most often used steps and how you perform them. As you advance through the book, you will have the opportunity to build along with each lesson, if you get stuck or don’t quite understand a lesson, then you have the solution file as a reference. I loved the interaction here, in most of the lessons I just created my own queries right along side the solution queries and if I ever got jammed up I could just click in to the solution query for the correct step. This made things really conducive to staying engaged in the lesson and moving through a lot of material.

    Like our journey with Power BI, it starts simple and moves towards advanced concepts. About mid-way to ¾ of the way through you will be introduced to the advanced editor and M, here you will start to learn how you can manipulate the queries themselves without the UI… Getting here puts you in such a powerful position with your ETL. Working through these lessons has empowered me to easily understand how to manipulate my backend sources, modify my steps I built in the UI with additional filters or actions, and troubleshoot potential issues in Power Query. If all that isn’t enough, Gil gives his top pitfalls and things to avoid section which is an invaluable resource section.

    I really enjoyed this book, and it sets a high bar for me moving forward in that I will be looking for other materials from other authors to provide this level of engagement with something I want to learn. The structure of the book, using the chapters to read through the step by step way to perform the action, having the source material and the solution files all in one spot makes for a fantastic way to learn. Whether this book is your first foray into Power Query, or you choose to go it alone, I highly recommend that you get started in the journey.

    If you like the content from PowerBI.Tips, please follow us on all the social outlets to stay up to date on all the latest features and free tutorials.  Subscribe to our YouTube Channel, and follow us on Twitter where we will post all the announcements for new tutorials and content. Alternatively, you can catch us on LinkedIn (Seth) LinkedIn (Mike) where we will post all the announcements for new tutorials and content.

    As always, you’ll find the coolest PowerBI.tips SWAG in our store. Check out all the fun PowerBI.tips clothing and products:
    Store Merchandise

  • Power BI Dataflows: Change is Coming

    Power BI Dataflows: Change is Coming

    I have been holding on to a copy of Satya Nadella’s book “Hit Refresh” for quite some time. With all the Power BI goodness, the job, etc.… I just hadn’t gotten around to it. However, it made its way into my bag on a recent flight and I found it to be an exceptional story and a very clear view into how Satya plans to take Microsoft into the future. You might say he “open sourced” his plans. After reading this and comparing it to what I’ve been hearing and seeing regarding the fundamental changes in culture and products coming out of Microsoft, I think I’m in a familiar group of those that say he appears to be an exceptional leader who has the talent, vision, and focus to achieve the goals he has set out for himself and Microsoft.

    The main three focus areas for the direction of Microsoft according to Satya revolve around Mixed Reality, Artificial Intelligence (AI), and Quantum Computing. It is important to understand this direction, because it can provide insight into the changes we see in product suites and what future these changes might hold. Setting aside Mixed Reality and Quantum Computing for the moment, we’re already being exposed to how AI is starting to augment Power BI. The latest announcements at PASS Summit revolve around exposing AI delivery mechanisms to business users via Automated Machine Learning features to gain even deeper insights. The work to introduce AI automatically into the tool is already present in features like Explain the Increase/Decrease, Quick Insights and Q&A. Innovations in bringing AI into reporting and analytics is going to continue to change how we look at information in a future that is much closer than I think many are prepared for.

    With the book in mind I was also doing a lot more study in architecture and design in the Azure ecosystem and strengthening my understanding of how the modern data platform is built and can expand to support multiple business needs. Without getting too involved, the overall gist of what I’m seeing is that the process of data ingestion, movement, transformations and storage are being made easier. The 2nd generations of the initial services are being rolled out and the suite of services are starting to do a large part of the heavy lifting in some of the most challenging areas. As a result, these services have a greater potential for wider adoption and becoming a large part of newer modern solutions. In addition, after tying all the services together from source to analytics I started to see a specific service that could be considered the hub for all this analytics activity. Azure Data Lake Storage Gen 2. This service is certainly being positioned as the main storage entity and seems to hold the architectural location as the de facto place where both Enterprise and Business are being funneled for interaction. Data cleansing, machine learning, warehousing, event hubs, etc., etc. can all pull/push from Azure Data Lake Storage Gen 2, and these interactions and manipulations are being made easier with each release.

    Taking what we understand about the overall goals of Microsoft, the centralization around a hub data and activity begins to not just make sense, but be a pivotal part of enabling future objectives to grow and be accessible to every business. Getting “All” of your business data in a single location for analysis will allow you to leverage current and future services to enhance and make use of AI and other technologies quickly, more efficiently and at a much lower cost.

    Power BI Dataflows is the first step in integrating the business into this ecosystem. Power BI Dataflows leverage a familiar product in Power Query, to connect to many sources and perform Extract, Transform and Load operations. They allow flexibility to map data to existing data entities and create new entities that have the potential to streamline and consolidate data silos. These objects that are the result of data flows are stored as CDM folders in Power BI.

    CDM Structure
    CDM Structure

    Two main things to hit here: First, a CDM folder consists of a CSV file for your data, and a model.json structure for metadata definition. Second, “in Power BI” means Azure Data Lake Storage Gen 2 behind the scenes, Microsoft just creates it for you so you don’t need it as a separate service if you aren’t using it for anything else.

    Where this new feature gets exciting is when it is used with your own Azure Data Lake Storage Gen 2. Power BI can connect to your existing Azure Data Lake Gen 2 storage instead and the CDM folders will be put there. This brings the business user into the Enterprise space and allows IT, Data Scientists and business users to collaborate in a single data repository. In addition to the above, we’ve already heard earlier this year that all of Dynamics and now 3rd party line of business and collection tools like SAP and Adobe will also plug into the Azure Data Lake Storage Gen 2 using the CDM folder structure. This means data will be constantly being added to the entities themselves. Power BI Dataflows offer up a unique opportunity to bridge some of the widest gaps that exist between business and IT in the data space.

    CDM Architecture
    CDM Architecture

    For more details on how to use data flows be sure to check out Matthew Roche’s video here -> https://www.youtube.com/watch?v=0bJpCVj3JfQ

    And for the full technical details, take a look at the “Power BI and Dataflows” Whitepaper here by Amir Netz -> https://docs.microsoft.com/en-us/power-bi/whitepapers

    In short order, to be at the top of the competition you’ll have to use Artificial Intelligence to be competitive and stay relevant, and I assume Mixed Reality is going to be a part of that as well. I would argue that what we are seeing here are the building blocks for that future and the efforts to adopt these services will allow us to make exponentially faster gains in analysis and decision making that will give businesses significant competitive advantages. Power BI is front and center in this endeavor as the analytics platform, and that should make any user of the tool excited indeed.

    The preview of Power BI Dataflows is out, based on how these pieces are falling into place across the board, and understanding the direction of Microsoft based on where the ship is being steered, I have a strong inclination that we’re going to be busy re-architecting solutions very soon and that platforms of services will allow businesses to make even more rapid innovations and advancements in their data journey’s. Power BI has already made for a fun ride, but this last month has me feeling like I may have just strapped a rocket to my back that is now being prepped for ignition.

    This is an opinion piece, and as such, I reserve the right to change my opinion as more information is learned. That being said, I’d love to hear feedback from you the reader if you have any on the subject.