AI Automates Redundant Manual Work

In the recent years, the success of artificial intelligence (AI) in various fields has transformed every walk of life. Not only does AI help improve decision-making in day-to-day operations, AI also automates redundant manual work. Many industries such as finance, banking, insurance, manufacturing, health care, and information technology have gained significant benefits and profits by incorporating AI to their businesses.

Hence, a highly growing demand in AI vendor is inevitable and can be foreseen to peak in the near future. In every AI project, there are always data scientists who play key roles. Essentially, data scientists use AI methodologies to uncover deep insights from data and build models from it. As straightforward as it may sound, in order to deliver an AI solution, a full AI project takes more than a handful of data scientists to complete.

In fact, a real AI project needs to follow a full AI life cycle and takes a full team effort. A complete AI project team should consist of at least data scientists, data engineers, application developers, and business experts. Each and every one of them plays a key role in the different stages of the AI life cycle.

• At the beginning of every AI project, business experts and data scientists will first define the problem, then identify and gather relevant data.
• Later, data engineers will need to extract data from the relevant system for data scientists to work on.
• During the modelling process, data scientists will run many experiments that include data preprocessing, feature engineering, hyper-parameter tuning, and finding optimal algorithms and evaluation metrics.
• Once the desired model is finalised, data engineers will take over from there to build the production data pipeline to automate the entire process from data extraction, to data preprocessing, to training the model, and  storing it.
• Finally, the application developers will need to build APIs for other systems to retrieve predictions from the production pipeline, as well as to work with data scientists to develop end-user applications such as reports, dashboards, and model interface.

Diagram 1 : Typical stages of the AI life cycle

Introducing a Practical and Robust Automated AI System that Covers End-to-End AI Processes

As you can see, a real AI project requires a lot of different professionals and efforts in order to succeed. While there are AI vendors out there, some of them do not provide end-to-end AI solutions. Furthermore, some business users might have ideas to apply AI to their businesses but there aren’t many data scientists and engineers available out there to aid them.

The shortage of capable AI teams has led to a huge demand of a system that can automate the AI life cycle. That is, a system that covers end-to-end AI processes and can also be easily used by non-experts.

To make such system legitimate in the real world, the system must be able to handle data preprocessing, feature engineering, hyper-parameters optimisation, algorithms selection, evaluation, validation, deployment, monitoring, and retraining automatically, with no or minimal human intervention. While such features do exist, it comes in a form of software package and in pieces. As a result, it still requires people with substantial coding and AI knowledge to combine all of them together. Business users such as executives and subject-matter experts that have no programming background will struggle to use them.

To address this challenge, we introduced a practical and robust automated AI system that covers end-to-end AI processes including an intuitive and easy to use graphical user interface, a fully automated modelling engine, a fast model deployment service, an efficient auto-generated API for integration, continuous learning support for the deployed models, and an extensive reporting and monitoring system.

We named this system Mindcraft 2.0, successor to the original Mindcraft system.

Mindcraft 2.0 is created to break down barriers between business experts, data scientists, software engineers, and developers. Therefore, non-experts are now able to apply AI to solve business problems without having in-depth technical skills.

Features of Mindcraft 2.0

Our robust automated modelling engine is built on top of scikit-learn, which is the most widely used machine learning library in Python. Scikit-learn offers a stellar lineup of highly efficient and optimised AI algorithms that can be used in different stages of data science process, namely, classification, regression, clustering, cross-validation, data preprocessing, feature processing, hyper-parameter optimisation, model selection, and model evaluation. While scikit-learn is indeed very powerful, its algorithms are all standalone. Thus, in order to train a model using scikit-learn, experience in programming and AI are still required.

With Juris Mindcraft’s automated modelling engine, we have already combined and automated all the steps together. By just uploading your data and picking the right target class, our engine will automate the entire pipeline to build the end models for you. Internally, after data and feature processing, the engine will tune the hyper-parameter by using the efficient Bayesian optimisation methods. This will significantly speed up the process of finding the best-performing model with the best set of hyper-parameters.

Furthermore, we go one step deeper by implementing weighted model stacking. Instead of just selecting the best individual model as the final model, we also select all other models that perform almost as good as the best model does during the optimisation process. Moreover, every selected model is given different weight, that is, prediction made by models with higher weight will have higher priority and importance.

As a result, a combination of multiple, weighted top models is built, an entity which we like to refer to as a supermodel. The supermodel is more robust and less prone to overfitting, thus often outperforms any single best performing model.

Another crucial feature of Mindcraft 2.0 boasts on top of its automated AI engine, is support for continuous learning. Continuous learning allows a deployed model to automatically update and improve itself whenever new data gets fed into it. When a model goes live, our system will track the performance of the model over time. If there is a sign of declines in performance, our system will again automatically update the model in response to the changing conditions. Hence, a deployed model might be continuously used over years without having to replace it! Speaking of data, our architecture supports many forms of data source, such as CSV, XLSX, SQL, XML, and JSON.

Now, Deployment Only Takes One Click

The other key feature of Juris Mindcraft is fast model deployment. There are times when application and software engineers have a hard time understanding models built by data scientists in order to put them to the production. This is mainly because AI models are usually built in Python or R whereas the production system is written in another programming language such as PHP, Java, Ruby, or Node.js.

As a result, deploying a model to production is not easy and often takes longer times than it should. According to Gartner, it takes an average of three months to deploy a model. Mindcraft 2.0 solves this problem by deploying the models as RESTful service, with auto-generated API in multiple, different languages to cater for different production systems.

Now, deployment only takes one click and other production systems can now get predictions from our API easily, regardless of which programming language is used. Deployment of Mindcraft 2.0 is also scalable. Once models are built from the modelling engine, different types of models can be deployed to different servers depending on the business objectives. Models that reside in many servers provide predictions specifically to each of the different production system.

Diagram 2 shows the architecture of Juris Mindcraft servers. We have also implemented a caching mechanism to cache frequently used models, thus enabling the models to provide predictions in milliseconds!

Diagram 2: Architecture of Juris Mindcraft servers

Other than that, Mindcraft 2.0 provides extensive reports about models such as confusion matrix, ROC curve, PR curve, and a step-by-step underlying process to build the model. Audit for any actions such as each model built, or each prediction made will also be reported. Besides, Juris Mindcraft also provides lift analysis which is able to map each prediction to its monetary value, based on the user’s defined mappings. With this, users can easily tell how likely a model will perform in the real world in terms of profit and monetary gains. Last but not least, users can also compare different models through Juris Mindcraft.

Finally, while the decisions generated from our AI model is often black-boxed, we also offer an explainable version of it. All the decision rules generated by the explainable model will be shown to users so they may understand all the decisions and reasonings behind every prediction.

Currently, we have a dedicated team for Juris Mindcraft, who are solely responsible for making Mindcraft as accurate and intuitive as possible, so citizen data scientists and business experts in your organisation can leverage the power of predictive analytics and artificial intelligence.

Successful application of AI requires an optimal mix of both business domain knowledge and technical expertise. Thankfully, we are in a unique position to help financial institutions with their AI requirements, because we have been in the financial services technology business for over two decades. As for technical expertise, our CTO John Lim and our CEO See Wai Hun have been researching artificial intelligence and big data since 1997, and Mindcraft is the fruition of almost a quarter century’s worth of research and investment. We simply did not just jump on the AI bandwagon when it started gaining traction.

At JurisTech, we are not big believers in jargons; we know you are more interested in results. We’re more than prepared to demo our AI with your data and do backtesting to test Mindcraft’s accuracy in the real world.