One way to think about how to deliver data science insights is by using a "Data Science Life Cycle (DSLC)." Unlike product lifecycles, it's a lifecycle that's geared towards exploration.
At first glance, DSLC appears to be a linear process, starting with identification and ending with learning, but the process is actually cyclical. Learning leads to more questions that return the team to the beginning of the process. In addition, mini-cycles often form within the DSLC as research and analysis results prompt questions that require additional research and analysis to answer.
So now you can see how data science teams can function more effectively and efficiently within the DSLC framework by employing the following techniques:
- Working in sprints—relatively brief, intensive, iterative work sessions, often referred to as an analytics sprint.
- Using question boards
- Conducting productive meetings
- Breaking down the work
- Telling interesting stories
Iterating through DSLC Sprints
The DSLC isn’t designed to cycle over a long period of time. Two weeks is sufficient for a cycle (a sprint). That gives the team sufficient time to prepare and analyze the data and compose a story that reveals the knowledge and insight extracted from the data and its significance to the organization during a 3 week sprint. With short cycles, if a specific line of enquiry proves fruitless, the team can change course and head in a different direction or tackle a new challenge.
You may have heard of sprints in the context of agile software development methodologies, such as Scrum, but the term actually originated in product development. A "sprint" is a consistent, fixed period of time during which the team runs through an entire lifecycle. Each sprint should run through all six stages of the DSLC.
Using Question Boards
Data science teams should be small (four to five individuals) and include a research lead, data analyst, and project manager. Although every member of the team should be asking compelling questions, the research lead is primarily responsible for that task.
One of the most effective ways to inspire and share interesting questions is via a question board— usually a large whiteboard positioned near the data science team on which team members and others in the organization post questions or challenges. The board should have plenty of open space with a short stack of sticky notes in one of the corners for recording views on data analysis. You may want to include a large arrow pointing down to the stack of sticky notes with the caption, “Ask a question.”
The question board should be open to everyone in the organization, including the research lead, other data science team members, executives, managers, and employees. Try to make your question board look as enticing as possible. Anyone in the organization should be able to walk by, grab a sticky note, and post a quick question.
Conducting Team Meetings
Given only two weeks to complete each sprint, your data science team should limit the amount of time it spends in meetings and keep those meetings focused on a specific purpose. I recommend that teams conduct five meetings over the course of a two-week sprint, each with a specific purpose and a time limit that the team agrees upon in advance:
- Research planning can benefit significantly from integrating a design sprint, which helps structure and expedite the process.: During this meeting, typically about two hours long, the team chooses the questions/problems it wants to research, and the research lead and data analysts develop a research agenda.
- Question breakdown: During each sprint, the data science team should have at least two one-hour question breakdown meetings, during which they ask questions, evaluate and prioritize questions for the next sprint, and clear uninteresting questions from the board.
- Visualization design: Typically a one-hour meeting, during which the research lead and data analysts formulate rough-draft data visualizations to begin to extract knowledge and insight from the data.
- Storytelling session: During this meeting, typically one hour, the data science team presents a story about what the team learned during the sprint. They present more polished versions of their data visualizations, discuss questions on the board, and tell stories about those questions.
- Team improvement: At the end of each sprint, the team should have a two-hour post-mortem meeting to discuss challenges they encountered during the sprint and talk about improving the process moving forward.
Breaking Down Your Work on Data Science Projects
Breaking down your work involves allocating a sufficient time to all six stages of the DSLC. What often happens is that data science teams get caught up in the research stage — specifically in the process of capturing, cleaning, and consolidating the data in preparation for analysis. Given only two weeks per sprint to deliver a story, the data science team has little time to prep the data. Like agile software development teams, the data science team should look to create a minimally viable product (MVP) during its sprint — in the respect to data science, this would be a minimally viable data set, just enough data to get the job done.
Remember, at the end of a sprint, stakeholders in the organization will want to know "What do we know now that we didn't know before?" If your team gets caught up in data prep, it won't be able to answer that question.
Telling an Interesting Story
Organizations that make significant investments in any initiative want to see a return on investment (ROI), typically in the form of a deliverable. In the world of data science, the deliverable is typically in the form of an interesting story that reveals both the meaning and the significance of the team's discoveries. Unlike a presentation or data visualization, which merely conveys what the team sees, a story conveys what the team believes. A good story provides context for understanding the data, along with guidance on how that understanding can benefit the organization.
An effective story accomplishes the following goals:
- Extracts meaning and insight from the data and simplifies the presentation of it.
- Makes the meaning and insight extracted from the data relevant to the organization and to specific questions or challenges, including those involving machine learning.
- Engages the audience and leaves a lasting impression. While most people quickly forget a presentation, they typically remember a good story.
- Persuades the audience to take action. A good story ends with a call to action, even if that call to action is to "stay tuned" because the data science team is on to something interesting and needs more time to explore. At the end of your story, you don't want your audience asking, "So what?" or, even worse, "Who cares?"
Frequently Asked Questions
What is a data sprint and how does it differ from traditional data analysis?
A data sprint is a short, focused project. Data scientists and analysts work together to solve specific data problems. Unlike regular data analysis that can go on and on, data sprints last 2-3 weeks. They aim to give quick and useful insights. People use methods from Agile to make these sprints fast and effective.
1. Data sprints are short and focused. 2. Teams work together to solve data problems. 3. Data sprints last 2-3 weeks. 4. They give quick and useful insights.
How can businesses benefit from a 10-day data sprint?
You can solve big data problems quickly with a 10-day data sprint. This method helps you check ideas fast, make quicker decisions, and see results right away. Work closely with data experts and use agile techniques. This way, you can boost your data skills in a short time.
Imagine this:
- A team of experts working side-by-side with you. - Clear daily goals to keep everyone on track. - Instant feedback to make sure you're on the right path. - Real results that show the value of your hard work.
You'll make better decisions and get real value quickly.
What does sprint planning involve in a data analysis sprint?
Sprint planning for a data analysis project means you set clear goals. You define what the project will cover. You make a detailed list of tasks for the sprint. The team includes data experts and a project leader.
You all work together to decide which tasks come first, often discussing the priorities in the sprint backlog. You also share out the work to use your resources well. This way, everyone knows the plan and works towards the same goals.
How do you handle data privacy concerns during a data sprint?
Data privacy is a top priority during any data sprint. You have to adhere to strict protocols and guidelines outlined in your privacy policy.
Sensitive data should be handled with care, and ensure compliance with all relevant data protection regulations.
What kind of results can I expect from an analysis during a 3-week sprint?
During a 3-week sprint, you can expect to get clear results for your business. You'll gain useful insights, test ideas, and find clear answers to your data problems. These results can help you make smart choices, improve how you work, and find new ways to grow.
Here's what you'll get from a sprint: - Clear insights - Tested ideas - Well-defined solutions
Sprints are short but powerful. They give you big results in a short time.
How do you measure the success of a data analysis sprint?
To see if your data analysis sprint was a success, you need to do a sprint review. This session checks if you met your goals. It also looks at:
- How accurate the data is - What people think about the data - How well the data solutions work for the business - The value the data solutions bring
The main goal is to make sure the data helps the business and adds value.
Who should be involved in a data sprint team?
A good data sprint team has different types of people. It usually includes data scientists, data analysts, a product owner, and business stakeholders. A diverse team helps get many ideas and better data solutions.
- You'll get input from many viewpoints. - Your solutions will be more useful. - Your team will think of things others might miss.
What types of data projects are best suited for a sprint format?
Data projects work well with clear goals and small tasks. You can break them into short, easy steps. These projects fit well with the sprint method.
Here are some examples:
- Discover new data - Check new data models - Improve business steps
You should use sprints for projects that need quick changes. Agile methods work well here.
Are there any prerequisites for participating in a data sprint?
It's not required, but knowing basic data analytics and agile methods can help. No matter if you're a beginner or a go-getter.
Here are three things you can do: 1. Learn some basic data skills. 2. Understand how agile methods work. 3. Ask our experts for help whenever you need it.
What is the role of a product owner in a data analysis sprint?
The product owner (sometimes called a research lead) has an important job in a data analysis sprint. They're the link between the sprint team and the people who want the work done. They set priorities, explain what the project needs, and make sure the team stays on track to deliver good results, often referring to the product backlog.
Here are three key tasks the product owner does:
1. They help plan the sprint. 2. They manage the list of tasks. 3. They guide the project to finish well.
The product owner keeps everyone focused and moving in the right direction.
This is my weekly newsletter that I call The Deep End because I want to go deeper than results you’ll see from searches or LLMs. Each week I’ll go deep to explain a topic that’s relevant to people who work with technology. I’ll be posting about artificial intelligence, data science, and ethics.
This newsletter is 100% human written 💪 (* aside from a quick run through grammar and spell check).
More Sources
- https://towardsdatascience.com/the-importance-of-storytelling-in-data-science-38eb625356d2
- https://www.geeksforgeeks.org/storytelling-in-data-science/
- https://www.projectpro.io/article/storytelling-with-data-a-key-skill-for-data-scientists/174
- https://www.linkedin.com/pulse/art-storytelling-data-science-communication-aditya-singh-tharran-sb5pe
- https://powerbi.microsoft.com/en-us/data-storytelling/
- https://www.correlation-one.com/blog/data-storytelling
- https://online.jcu.edu.au/blog/the-importance-of-data-storytelling
- https://www.linkedin.com/pulse/from-data-action-importance-storytelling-analysis-parth-sheth
- https://www.yellowfinbi.com/blog/why-data-storytelling-is-important-data-doesnt-speak-for-itself
- https://www.aiimi.com/insights/what-are-data-sprints-how-do-they-work
- https://revistadigitos.com/index.php/digitos/article/download/253/135
- https://www.scrum.org/forum/scrum-forum/26058/data-analysis-sprints
- https://www.linkedin.com/pulse/data-sprint-winning-analytics-framework-light-speed-tanaka-ph-d-
- https://revistadigitos.com/index.php/digitos/article/view/253
- https://www.cprime.com/resources/blog/agile-methodologies-how-they-fit-into-data-science-processes/
- https://towardsdatascience.com/how-to-make-agile-actually-work-for-analytics-e8fb2290276e
- https://www.linkedin.com/advice/0/how-can-agile-methods-improve-data-analytics-performance
- https://www.datascience-pm.com/agile-data-science/
- https://www.modernanalyst.com/Resources/Articles/tabid/115/ID/6081/Agile-Data-Science-How-is-it-Different.aspx