Creating Artificial Intelligence and Machine Learning solutions is a different ball game to typical software development. Not only do you have all the usual project risks, but you have an additional complexity with AI and ML which stems from the fact that each dataset you address is different to the last.
Algorithm selection is a critical part of the process. Whilst experienced data scientists can make recommendations for the best machine learning method, without actually testing the theory on the dataset at hand one can’t be certain which algorithm will deliver the best results. We see that this results in two, rather concerning outcomes:
- Projects continually drift or sit on the shelf. The organisation may choose to manually try many different approaches. Experienced Data Scientists are in short supply, and their time is limited. Manually tuning algorithms and models is a time-consuming process and projects must wait in line for air time with the team.
- Projects are cancelled when initial results are poor. The Data Science team may be pushed to commit to an algorithm which, after months of development does not deliver the results management expect. Momentum is lost and the project is pulled.
The problem is made even more significant within large organisations because datasets can vary widely, even within the same task. Take a typical task of risk profiling within a mortgage provider. The critical task for this global organisation is to flag, as early as possible, customers likely to default on their mortgage payments. This task has huge monetary values associated with it and is common across the globe. However, the customer population within the United States may have different drivers to those within the European Union (and all the other markets that are served for that matter). Different AI approaches are needed for every market and segment served.
This is where AutoML (short for Automated Machine Learning) comes in. Chatterbox Labs encapsulate AutoML in our Cognitive Engine software product. The Cognitive Engine operates in a different way to typical Data Science tools. Rather that a user approaching the software with a data science question along the lines of “I require a Neural Network, how can I implement this?”, users of the Cognitive Engine approach it with a business question along the lines of “I need to identify customers likely to default on their mortgages, how can I solve this?”.
The Cognitive Engine will then automate the process of evaluating the applicability of different algorithms for this task, without any manual intervention. It will try linear and non-linear Support Vector Machines, a Neural Network, Naïve Bayes and Random Forest algorithms and evaluate them against the provided dataset. As this is an automated process, applying this to different datasets (such as applying it to US customers and EU customers) is trivial.
With this automated approach, building of the solutions is democratized. Projects no-longer wait in line for air time with the central data science team because the Subject Matter Experts (those familiar with their data and requirements) are empowered to build the solutions.
You can see our Cognitive Engine’s AutoML at work in the following video:
If you’d like to find out more about Chatterbox Labs’ AutoML (or other products targeted at Synthetic Data Generation and Explainable AI) please get in touch.