Blog

Analytics Best Practices – Agile Data Science

Anna Godwin

September 15, 2017

BLOG_Analytics-Best-Practices-Part-1.jpg

This is the second in a series of blogs where Data Scientists Anna Godwin and Cory Everington discuss five Analytics Best Practices that are key to building a data-driven culture and delivering value from analytics. In this installment Anna discusses the benefits of using Agile Data Science as a framework for managing data science projects. 


If you’ve worked on any project you’ve likely experienced unexpected changes over the project lifetime. It is those surprises that can impact your ability to complete the project as it was initially designed, with changes in the team, the deliverable, and/or the timeline.  Being flexible as an individual and as a team throughout a project enables you to quickly respond to changes. But projects benefit from being structured. What is the best framework for managing the right balance for data science projects?

For our project work Elder Research employs the Agile Data Science methodology. It combines best practices for Agile software development with best practices for data mining, as embodied by the Cross Industry Standard Process for Data Mining (CRISP-DM).

Agile_Data_Science.png

Agile Data Science provides the following benefits; It:

  • Provides flexible team organization
  • Generates early buy-in from stakeholders
  • Adapts to unforeseen circumstances 

Agile Data Science Brings Organization to the Project Team

While it is possible to use agile methodology when working alone, the approach is designed to help organize the work for a team. Any time team members are working on different aspects of a project there can naturally be confusion, duplication of efforts, or work on tasks that are not focused on project deliverables. Agile Data Science helps mitigate these problems and create a shared vision through sprint planning, daily stand-ups, team transparency, and roadblock awareness. 

Two main components of agile methodology govern project work:

Sprint Planning is a process where project work is broken down into small chunks of work that can be completed in the span of a few hours or days and assigned to team members. It is often done on a two-week cycle to allow team members to iterate quickly and provide flexibility in how the work will be completed. We will go into more detail on how to plan sprints in our upcoming blog post about Task Management.

Daily Stand-ups facilitate communication and project success by creating a forum for each team member to give a short (2-3 minute) update about their progress on assigned tasks and on any roadblocks they encountered that could impact the schedule. Daily stand-ups are designed to be less than 15 minutes. They are not a forum for detailed discussions, instead they provide an opportunity for team members to understand what others are working on and for the project lead to adjust course as necessary.

Agile Data Science Engages Stakeholders

The iterative nature of the agile methodology requires frequent check-ins with project stakeholders.  Sharing results early and often allows the project team to focus on building a quick solution that provides value and gains end-user adoption. A focus on deployment early in the process is core to the Elder Research approach. Whether the deliverable is a complete software solution or an analytic algorithm, it is important to gain early buy-in from stakeholders— and that happens by getting their feedback early in the process. A solution is only successful if implemented by the end-user, so early feedback from key users is crucial for shaping a solution’s design.  And, a strong side benefit is that the key users become invested in the solution.

Agile Data Science Quickly Adapts to Change

Changes come from all directions in project work. For example, a team member moves to another project or a new member joins the team.  Being able to quickly get new team members onboard is a core agile strength.  Team stand-ups and sprint planning provide frequent project updates and introductory assignments that new members can quickly complete until thorough knowledge transfer occurs.

Changes to the group of stakeholders may occur as well. As the stakeholders change, so may the vision for the analytics project.  When it feels like the project target is constantly moving, the adaptive nature of an agile approach works in your favor.  The iterations that take place during an agile project lifecycle provide a key opportunity to either stay the course or to pivot in a different direction. Having multiple “Yes” or “No” decision points with stakeholders drives the difference between project completion and successful project completion.

A final type of unexpected change may come in the form of a technical hurdle. For example, the initial technology stack chosen for the project may not be working well. Again, the quick iterations of Agile Data Science allow for a “fail fast” reality where technical limitations come to light much sooner in the project lifecycle than otherwise. This early insight allows the team to quickly discover an alternative that can provide a better solution for the project during the project timeline.

Find an Approach That Works for You

The key word in Agile Data Science is -- you guessed it -- agile. We highlighted the benefits of adopting a flexible structure to your project work, and at the end of the day, each project team should adopt a structure that works best for them. Be flexible in how you set up your agile approach so you spend less time on project management and focus more on completing the technical project work to create actionable solutions that deliver real value to end-users.

Request a consultation to speak with an experienced data analytics consultant.


Related

Read part one of this blog series Analytics Best Practices – Part 1

Read the blog Using Data Science Methods to Manage Shifting Priorities

Check out the book Mining Your Own Business for more best practices on how to harness the power of data science and predictive analytics, and avoid costly mistakes.


About the Author

Anna Godwin Data Scientist Anna Godwin enjoys working closely with customers to assess needs and deliver custom analytic results to generate business value. She values teaming and brainstorming with clients and colleagues. Prior to joining the Elder Research Commercial Data Science team, Anna worked as an engineer in medical device product development, analyzing controlled experiments on production processes to drive decisions in research, development, and manufacturing capacities. A North Carolina native and Wolfpack fan, Anna holds three degrees from NC State: an MS in Analytics, BS in Biomedical Engineering and a BS in Textile Engineering.