{"id":8200,"date":"2021-04-07T21:34:44","date_gmt":"2021-04-07T21:34:44","guid":{"rendered":"https:\/\/wealthrevelation.com\/data-science\/2021\/04\/07\/how-to-make-sure-your-analysis-actually-gets-used\/"},"modified":"2021-04-07T21:34:44","modified_gmt":"2021-04-07T21:34:44","slug":"how-to-make-sure-your-analysis-actually-gets-used","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2021\/04\/07\/how-to-make-sure-your-analysis-actually-gets-used\/","title":{"rendered":"How to Make Sure Your Analysis Actually Gets Used"},"content":{"rendered":"<div id=\"post-\">\n   <!-- post_author Taylor Count -->  <\/p>\n<p><b>By <a href=\"https:\/\/www.linkedin.com\/in\/taylorabrownlow\/\" target=\"_blank\" rel=\"noopener\">Taylor Count<\/a>, Head of Data at Count<\/b>.<\/p>\n<p><img class=\"aligncenter size-full wp-image-125299\" src=\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/make-analysis-used-without-pirates.jpeg\" alt=\"\" width=\"90%\"><\/p>\n<p><em>Photo by\u00a0<a href=\"https:\/\/unsplash.com\/@arstyy?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noopener\">Austin Neill<\/a>\u00a0on\u00a0<a href=\"https:\/\/towardsdatascience.com\/s\/photos\/pirates?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noopener\">Unsplash<\/a>.<\/em><\/p>\n<p>\u00a0<\/p>\n<h3>Beware of Pirates<\/h3>\n<p>\u00a0<\/p>\n<p>One of the most universally demoralizing experiences is to see the results of your hard work go unseen, unappreciated, and unused. In the world of data, that is something we experience all too often. Take the following hypothetical situation:<\/p>\n<ol>\n<li>Jim submits a request to the data team for a deep-dive analysis for a client presentation the following week.<\/li>\n<li>You and Jim spend all week working on the analysis, working closely to make sure he has the right visuals and feels confident presenting the findings.<\/li>\n<li>The day of the presentation arrives, and not a word from Jim. That\u2019s odd.<\/li>\n<li>When you finally track him down, he tells you he \u201cdidn\u2019t end up using the charts after all.\u201d \u201cThey just would\u2019ve confused them,\u201d he adds in a conciliatory tone.<\/li>\n<li>You are fuming. A whole week wasted. Another decision was made without data there to back it up. Why did he even ask in the first place?<\/li>\n<\/ol>\n<p>I like to call these requestors\u00a0<strong>pirates<\/strong>\u00a0because they steal my\u00a0time. Unfortunately, there will always be pirates, but there are ways we can learn to avoid them or at least cope with their existence. Here is a list of tips to make sure your analysis gets the credit it deserves, assembled from my own experience, academic research, and industry best practices.<\/p>\n<p>\u00a0<\/p>\n<h3>1. Ditch the Data Request Forms<\/h3>\n<p>\u00a0<\/p>\n<blockquote>\n<p><em>We must be consultants, not hired hands.<\/em><\/p>\n<\/blockquote>\n<p>Most data teams have a request portal that they use to triage and assign data requests that come from the business. These portals are designed to make it easier for the business and data teams to work together; business users type out exactly what they want, and the data team just makes it happen.<\/p>\n<p>Unfortunately, as we saw from Jim, it\u2019s not that simple. Many business users come to the data team with a chart already in mind, including what the numbers on that chart should show.<\/p>\n<p>At this point, we\u2019re already doomed. If the data doesn\u2019t match the story the requestor wants or is a bit nuanced, then they\u2019ll never use this analysis.\u00a0<strong>We need to know the problem they\u2019re trying to solve.<\/strong><\/p>\n<p>As data professionals, we know the data and the statistical methods better than anyone and can advise on the best approach to using the data to answer the question at hand. The business context in partnership with our data expertise can combine to create analysis far more impactful than what we could produce individually.<\/p>\n<p>In short, we must be consultants, not hired hands.<\/p>\n<p>\u00a0<\/p>\n<h3>2. Numbers Never Walk Alone<\/h3>\n<p>\u00a0<\/p>\n<blockquote>\n<p><em>A chart alone cannot possibly convey everything, and that kind of thinking inhibits our ability to influence the business with our work.<\/em><\/p>\n<\/blockquote>\n<p>Often we\u2019re expected to send over a single chart or dashboard as a completed request. These are nearly impossible for the business user to interpret without a 1:1 explanation.<\/p>\n<p>We\u2019ve been told that data can speak for itself, that a well-crafted chart can communicate all its nuances on its own. This is simply not true. A chart alone cannot possibly convey everything, and that kind of thinking inhibits our ability to influence the business with our work.<\/p>\n<p><img class=\"aligncenter size-large\" src=\"https:\/\/miro.medium.com\/max\/875\/1*LImlz_Lic9buP1In16tx1A.png\" width=\"90%\"><\/p>\n<p><em>You can\u2019t rely on charts to convey insights alone. Make use of text to explain your work. Source:\u00a0<a href=\"https:\/\/count.co\/n\/rrpfH1Hi0L5\" target=\"_blank\" rel=\"noopener\">The Best Player Never to Win a Title<\/a>\u00a0by\u00a0<a href=\"https:\/\/count.co\/?utm_source=medium\">count.co<\/a>.<\/em><\/p>\n<p>When sharing any analysis, I try to always include the following information:<\/p>\n<ul>\n<li>time period of data<\/li>\n<li>date of analysis<\/li>\n<li>author<\/li>\n<li>TL;DR: summary of context and insights<\/li>\n<li>explanation of how to read the chart<\/li>\n<li>how you did the analysis (not the code, but the layman\u2019s explanation)<\/li>\n<li>limitations and next steps<\/li>\n<\/ul>\n<p>This contextual information may seem like a headache, but it makes a huge difference. We haven\u2019t just sent a chart, which, in isolation, can carry the unhelpful subtext \u2018figure it out.\u2019 We\u2019ve sent them an analysis with everything they need to turn that chart into insight, a small gesture that doesn\u2019t go unnoticed.<\/p>\n<p>Breaking the habit of sending charts out on their own gives them a chance to be understood, and ultimately, used.<\/p>\n<p>\u00a0<\/p>\n<h3>3. Make it an Experience<\/h3>\n<p>\u00a0<\/p>\n<blockquote>\n<p><em>To really make sense of your analysis, your users will need to poke and prod it\u2026 Let\u2019s help them get there.<\/em><\/p>\n<\/blockquote>\n<p>Surrounding your chart with context and explanation ensures the reader has everything they need to learn<em>\u00a0<\/em>something from our analysis. But we learn best through experiences[1].<\/p>\n<p>So to really make sense of your analysis, your users will need to poke and prod it. Kolb\u2019s Learning model suggests they\u2019ll need to experiment with our analysis and take the time to reflect on its real-world implications before they can properly understand it. Let\u2019s help them get there.<\/p>\n<p><img class=\"aligncenter size-large\" src=\"https:\/\/miro.medium.com\/max\/875\/1*V8sEG9HOnPgyVV3G7kd-_w.png\" width=\"90%\"><\/p>\n<p><em>David Kolb\u2019s Experiential Learning Model (ELM) [1] Image source: author.<\/em><\/p>\n<p>At a minimum, this involves setting up interactive elements for your analysis. Add filters and parameters that let the user start to interrogate the data. What if you had double the budget? Half it?<\/p>\n<p>This question-answer flow lets the user trust the analysis and understand how it relates to their problem, ultimately giving them the confidence to wield that analysis in the boardroom. This lack of confidence is the number one reason your chart doesn\u2019t make it into that slide deck, so take care here.<\/p>\n<p>\u00a0<\/p>\n<h3>4. Make it Presentation Ready<\/h3>\n<p>\u00a0<\/p>\n<blockquote>\n<p><em>Create engaging and informative visuals that won\u2019t intimidate viewers without sacrificing the complexity of your analysis.<\/em><\/p>\n<\/blockquote>\n<p>Unfortunately, we can\u2019t expect someone to take the time to learn from the analysis in a presentation the way our business partner has (hopefully) done up to this point. This means we now need to create a summary chart that can reflect the key points of our analysis but in far less detail.<\/p>\n<p>Ideally, this is done as the last step of your analysis, once you agreed upon the key insights and how best to compose them into a larger decision or problem to solve. Then you can make use of Data Visualisation best practice [2] to create engaging and informative visuals that won\u2019t intimidate viewers without sacrificing the complexity of your analysis.<\/p>\n<p>\u00a0<\/p>\n<h3>5. Long Live the Analysis<\/h3>\n<p>\u00a0<\/p>\n<blockquote>\n<p><em>Make sure that your analysis lives beyond this single data request and can be used again and again.<\/em><\/p>\n<\/blockquote>\n<p>One part of this process that is severely neglected is the question of turning this analysis into scalable knowledge. How do you make sure the business question you\u2019ve just answered is shared not just with Jim or Jim\u2019s team but with the wider company? And not just this week, but that it can be used in 6 months when the same question comes up again. The answer is unequivocally not a dashboard but something more nuanced.<\/p>\n<p><a href=\"https:\/\/medium.com\/airbnb-engineering\/scaling-knowledge-at-airbnb-875d73eff091\" target=\"_blank\" rel=\"noopener\">AirBnB\u2019s approach<\/a>\u00a0[3] has been to implement a Knowledge Feed that takes the type of detailed analysis we\u2019ve just outlined and publishes it for the whole company to find. The result is a collection of reports that are easily understood by all users but still have access to the raw code and notes for analysts to use as a starting point for future work. The key attributes are documented that give everyone confidence in what they\u2019re seeing (when it was published, the limitations, etc.). And they\u2019ve made this database of knowledge easily parsable so people can quickly find the analysis related to their questions before they\u2019ve submitted their request to the data team.<\/p>\n<p>Now you can make sure that your analysis lives beyond this single data request and can be used again and again.<\/p>\n<p>\u00a0<\/p>\n<h3>D.I.Y. Time<\/h3>\n<p>\u00a0<\/p>\n<p>The benefit of this kind of way of working is it\u2019s easy to test. The next time a request comes through from one of your more friendly business users (avoid pirates), I suggest trying this method out. Instead of materializing the chart they requested, ask to meet with them to better understand what they hope to do with this chart. What decisions is it informing? Who is the audience?<\/p>\n<p>And as you work together in this analysis, I suggest using a data notebook to document the required metadata and explain your work to your business partner. This gives you the flexibility to contextualize your analysis in-line with code and visuals, so you aren\u2019t trying to hack together a Google Doc somewhere.<\/p>\n<p>Once you\u2019re both happy with the analysis and findings, then work on the final chart together, and see how different it looks to what the original request was.\u00a0<strong>I\u2019m willing to bet they\u2019re entirely different.<\/strong><\/p>\n<p><img class=\"aligncenter size-large\" src=\"https:\/\/miro.medium.com\/max\/625\/1*CUdZO_PNZNavj3wovNHvHA.png\" width=\"90%\"><\/p>\n<p><em>Example of Count notebook. Source:\u00a0<a href=\"https:\/\/count.co\/n\/COxbPuAsqRC\" target=\"_blank\" rel=\"noopener\">Who is the Tennis GOAT?<\/a><\/em><\/p>\n<p>Committing this analysis to shared knowledge requires a bit more forethought. There aren\u2019t many natural places for these notebooks to go; Github isn\u2019t user-friendly enough for non-developers, and options like DropBox or Google Docs aren\u2019t technical enough to include the code that\u2019s required.<\/p>\n<p>If you forced me to recommend a tool, I\u2019d have to say\u00a0<a href=\"https:\/\/count.co\/?utm_source=medium\">Count<\/a>, but full disclosure, I did help build it. Count is a data notebook that\u2019s aiming to make this kind of way of working the norm. You can create high-quality analytical reports that are full of context, explanations, customized visuals all in one document giving your work the platform it needs to outlive the transient data request and become knowledge from which the whole company can benefit.<\/p>\n<p><em>If you\u2019ve tried any of these methods, I\u2019d love to hear how it went in the comments!<\/em><\/p>\n<p>\u00a0<\/p>\n<h3>References<\/h3>\n<p>\u00a0<\/p>\n<p>[1] Kolb, D. A.\u00a0<a href=\"https:\/\/www.researchgate.net\/publication\/235701029_Experiential_Learning_Experience_As_The_Source_Of_Learning_And_Development\" target=\"_blank\" rel=\"noopener\"><em>Experiential Learning: Experience as the Source of Learning and Development<\/em><\/a>. New Jersey: Prentice-Hall; 1984.<\/p>\n<p>[2] Mahoney, Michael.\u00a0<a href=\"https:\/\/towardsdatascience.com\/the-art-and-science-of-data-visualization-6f9d706d673e\" target=\"_blank\" rel=\"noopener\"><em>The Art and Science of Data Visualization<\/em><\/a>. Towards Data Science; 2019.<\/p>\n<p>[3] Sharma, C. &amp; Overgooer, Jan.\u00a0<a href=\"https:\/\/medium.com\/airbnb-engineering\/scaling-knowledge-at-airbnb-875d73eff091\" target=\"_blank\" rel=\"noopener\"><em>Scaling Knowledge at Airbnb<\/em><\/a><em>.\u00a0<\/em>AirbnbEng; 2016.<\/p>\n<p>\u00a0<\/p>\n<p><a href=\"https:\/\/towardsdatascience.com\/how-to-make-sure-your-analysis-actually-gets-used-4295bac3073\" target=\"_blank\" rel=\"noopener\">Original<\/a>. Reposted with permission.<\/p>\n<p>\u00a0<\/p>\n<p><b>Related:<\/b><\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/www.kdnuggets.com\/2021\/04\/make-analysis-used.html<\/p>\n","protected":false},"author":0,"featured_media":8201,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8200"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=8200"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8200\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/8201"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=8200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=8200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=8200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}