One key difference between the two approaches is what is being delivered. The deliverable for a Crisp-DM (Analytics) project is a piece of insight embedded into a decision space. The deliverable for an Agile (Software) project is a working piece of software or a product.
Another key difference is the concept of the minimum viable product, or MVP. In software development the concept of an MVP works really well. Within Analytics projects the idea of an MVP doesn’t really work because once we create a viable model our job is done.
Software projects developed using Agile methods focus on the needs of the end user. Gathering requirements using user-stories is a very powerful tool and works really well for software projects. Focusing on the end user may not always work well in an analytics context. Usually, focusing on the business objective works much better, because in most cases, the end user will never know of the existence of the analytics output.
Another difference is that for software projects it is usual that a team is assembled at the beginning of the project and the team work together on a full-time basis until the project it delivered. With analytics projects there are some resources, such as the data engineer, that are only required at certain stages in the project.
Agile methods focus on delivering working, tested product though multiple iterations. Iterations should be delivered within a relatively short time frames, normally 2-3 weeks. This concept works well for software projects but doesn’t really work for Analytics projects. Once you have created a tested workable model you are done!
So, in my opinion Agile methods in their current form are not a perfect match for Analytics projects. There are lots of aspects from CRISP-DM that still work really well, but there are some aspects of Agile that I really like and can be applied alongside CRISP-DM to form a better methodology for modern Analytics projects. So keep tuned for the next blog post, where I will discuss my thoughts on how you can adapt CRISP-DM with a healthy sprinkle of Agile.