Ensemble Learning Gradient Boosting in Improving Classification and Prediction in Machine Learning

Abstract

Ensemble methods have been studied extensively in machine learning.   It is a meta-algorithms that combine several techniques in machine learning to create one predictive model. Ensemble Learning improves machine learning results by combining several models. This approach allows the production of better predictive performance compared to just a single model. Gradient Boosting is part of an ensemble technique that attempts to create a strong classifier from several weak classifiers. This paper focuses on three critical issues that need to be addressed in the boosting process. First is the classification techniques that are used; second is a combination method for conjoining several selected classifiers and last is the combined three classifiers. Afterward, a comparison study is conducted between the proposed Ensemble Learning Gradient Boosting (EL-GB) using three widely used classification techniques that consist of AdaBoost, Gradient Boosting, and XGB Classifier. Two financial ratio datasets of the banking industry have been employed in the experiments. The results show that the proposed EL-GB classifier has achieved a great performance with an accurate value of 98%. This performance is comparable with XGB Classifier that achieved 98% while AdaBoost is only 96%. In terms of data processing, the proposed EL-GB is easier to implement via matching process upon all available data, so the predict() function can be called to make predictions on the new data. It iteratively corrects weak classifiers. These results illustrate the capability of the proposed EL-GB to work on the banking industry data which can be used to detect a level of control in a bank while undertaking financial distress.