Image credit: Unsplash

What is one of the most common mistakes data scientists make? Overfitting. 

What is overfitting?

Overfitting is one of the biggest concerns in machine learning (ML). It is a concept in data science whereby a machine learning model performs very well for training data but gives a poor performance with new data. For instance, when the AI model trains far too long or too complex on the training dataset, the model can start to learn the “noise,” or irrelevant information. What happens is that when the model memorises all the “noise” and fits too closely to the training set, the model becomes “overfitted”. Thus, unable to generalise well to a completely new or unseen data to make accurate predictions. This becomes a serious issue for machine learning models that are used to make predictions. 

This is where ensemble learning comes into play.

What is ensemble learning?

Ensemble learning is a popular machine learning technique that combines several models to improve the prediction accuracy, generalisability and its robustness. Noise, variance, and bias are the main sources of error in machine learning models. Ensemble methods play a huge part in reducing these error-causing elements, ensuring the accuracy of the machine learning algorithms. There are three types of ensemble learning — bagging, boosting, and stacking.

Bagging: A method where multiple versions of a predicted model, which is usually a weak model, are aggregated and each model is trained independently.

Boosting: A method that combines a set of weak learners into a strong learner to minimise training errors. Unlike bagging where each model runs independently, boosting requires the algorithms to “work together”. Every model that runs will determine what features the next model will focus on.

Stacking: A technique that combines different families of algorithms together to strengthen the model’s robustness. Instead of just selecting the best individual algorithm as the final model, the model will select other algorithms that perform almost as good as the best algorithms do during the optimisation process. 

And all these algorithms are supported in Juris Mindcraft!

Juris Mindcraft, effortless AI for intelligent business decisions

Juris Mindcraft is our very own proprietary artificial intelligence (AI). It is an automated AI system that uses advanced machine learning techniques to build powerful AI models. We developed this technology with one goal in mind — to assist enterprises, especially banks and financial institutions to make intelligent business decisions and gain insights to solve real-world problems. 

Juris Mindcraft automates the end-to-end machine learning model development process from importing data, data pre-processing, ML modelling, all the way to deployment. It empowers business users and data scientists to take action on explainable AI recommendations, achieving critical business objectives.

How does Juris Mindcraft solve overfitting?

“Juris Mindcraft uses multiple well-known ensemble learning techniques to achieve a supreme model.”

Juris Mindcraft is able to use advanced machine learning techniques to learn from historic data and recognise patterns to build powerful predictive AI models. It also goes one step further by implementing weighted model stacking, where every selected algorithm is given a different weight based on its performance. The higher the weightage of the prediction models, the higher the priority and the importance of the algorithm. As a result, a combination of multiple weighted top models is built — an entity which we like to refer to as a supermodel. 

The supermodel is more robust and less prone to overfitting, thus outperforming any single best performing AI model! This has also led to Juris Mindcraft being capable of continuous and autonomous self-learning to improve its prediction accuracy. Besides that, Juris Mindcraft is also able to automatically update and improve itself whenever new data gets fed into it. Once the AI model goes live, the system will track the model’s performance over time and will automatically replace the existing predictive model with one that yields better accuracy (with human consent, of course!).

AI Supermodel in action!

The pandemic has triggered specific implications such as managing and mitigating credit risk. For instance, the risk of loss arising from a borrower being unlikely to pay its loan obligations in full may impact all credit-sensitive transactions. This includes loans, securities, and derivatives. Changes in creditworthiness differ by sectors and subsectors to a greater degree than they did in previous recessions. Thus, it has become a huge challenge for banks and financial institutions to accurately identify credit risk during the scoring and loan approval stage.

Juris Mindcraft, however, will be able to make a difference. As mentioned above, Juris Mindcraft uses machine learning modelling as a basis to better predict the potential risk and likelihood of a customer defaulting. Furthermore, if Juris Mindcraft is 90% confident that a customer with a good credit rating will be able to pay back the loan without any complications, the application can be approved instantly without any human intervention!

If you are not confident in its decisioning, rest assured — the decision(s) made by Juris Mindcraft is not black-boxed. Juris Mindcraft is an explainable AI, where explanations are given behind every decision reached. The supermodel in Juris Mindcraft enables the system to adapt as more data gets fed into it. Hence, the more data you input into the model, the more likely it is for its prediction accuracy to improve. With Juris Mindcraft and its AI-based algorithm, banks and financial institutions can now crunch huge quantities of customer data in a few seconds to verify customers’ creditworthiness and determine whether to grant them a loan. Thus, enriching the credit risk management process by reducing time-to-market and ensuring accurate credit scores. 

Sounds amazing? That is only the tip of the iceberg! There is so much more to what Juris Mindcraft can do.

When we use the term ‘Artificial Intelligence’, we really mean it

We have a dedicated team for Juris Mindcraft that is in charge of making our AI as accurate and intuitive as possible, so that your data scientists and business experts can put more focus on things that matter — getting insights to solve real-world problems. With Juris Mindcraft, you can now leverage the power of predictive analytics and artificial intelligence to make intelligent business decisions. Do hit us up if you are keen to know more! 

About JurisTech

JurisTech (Juris Technologies) is a leading Malaysian-based fintech company, specialising in enterprise-class software solutions for banks, financial institutions, and telecommunications companies in Malaysia, Southeast Asia, and beyond.

As one of the Fintech pioneers in Malaysia, our vision is to enable financial inclusion for the financial industry with our diverse range of solutions. Check out our latest AI-powered technology Juris Mindcraft, which helps banks and financial institutions to transform their digital landscape.