Winter Is Coming … Maybe

I was recently at data analytics/big data conference at which someone celebrated the fact that data analytics was right at the top of the peak of overinflated expectations on the Gartner hype cycle and looked forward to moving over to the plateau of productivity.  This was worrying as it sort of missed the whole point of the hype cycle – that many technologies, no matter how useful, never manage to make the jump across the trough of disillusionment to reach their real, unhyped potential.

The Gartner Hype Cycle

The AI Winter of the 1980s is a great example of this. Spurred on by some early successes, the hype surrounding artificial intelligence (AI) got way out of control. We were promised computer brains orders of magnitude more powerful than our own, robot butlers and nannies and generally an idyllic AI-enabled life. The reality of AI research at the time, although impressive, couldn’t possibly live up to these expectations and so the resulting disillusionment was massive. Research funding for AI dried up and the field suffered badly overall.

Remembering our over-excited conference speaker, and channeling our inner George Hamilton, danger here! As all of the current media coverage of analytics and big data shows we are right at the top of the peak of inflated expectations and there is a real risk that analytics could flounder in the trough of disillusionment. This could lead to an analytics winter, which is scary.

Scarier still is that this can also happen at a micro level within a company. In order to get analytics projects and teams off the ground we need to extoll the virtues of analytics and evangelize internally for its adoption. Just as it can at the industry level this can lead to overinflated expectations that we cannot meet even when we complete analytics projects successfully. Failure to meet expectations on the early projects undertaken by an analytics team can mean that the team never really gets off the ground and enters its own analytics winter in which the analytics team is starved of resources and may as well just hibernate! Happily, in contrast to the macro-level danger of the whole industry entering an analytics winter, we can take steps to avoid this at the micro-level.

It is probably unavoidable that we will have to evangelize in order to secure the resources for an analytics team in the first place, so it is hard to avoid stoking the expectation fires a little bit. What we can do, though, is make sure that we pick those early analytics projects very carefully so as to to ensure that we will succeed in a way that meets expectations to the greatest extent possible. when picking these first projects make sure of the following:

  • That there is a business process that can directly utilise the output from the proposed analytics project. A great example of this is helping companies develop better call lists for their retention teams. Often, it turns out that companies not using analytics simply select a group of customers at random from their customer database each week and give this list over to their retention team who phone each customer on the list and try to convince them to stay with the company. Obviously, we could use analytics techniques to select this customer list more effectively ensuring that those customers most likely to churn are the ones who receive a retention call (a churn propensity model would do this nicely). The real advantage about a project like this is that the business doesn’t have to change any thing about their process to harness the power of data analytics – the retention team still gets a list of customers each week and make a phone call to each customer, the list is just much better. Picking projects like this is much better than picking projects that require a business to dramatically change their existing processes.
  • That you have easy access to the data that you need – there is nothing worse than announcing the start of your analytics project and then announcing that a 6 month data cleansing project is needed to get the data ready.
  • There must be a way to measure the impact of your project. Whether this is through increased sales, reduced customer retention or anything else doesn’t really matter there simply must be a number from before the analytics project that can be compared to a number after the analytics project. control groups are really useful here too.
  • Early analytics projects should be small and relatively quick to implement – don’t make the business wait two years to discover whether or not analytics works for them. Six months is probably a nice time frame within which to do some good work and to prove success.

Choosing projects according to the criteria above should help to make sure that you avoid a micro analytics winter at your business. As for the whole industry, let’s just hope we don’t see the introduction of shoe-shine boy analytics any time soon!