Banking news-events representation and classification with a novel hybrid model using DistilBERT and rule-based features
Main Article Content
Abstract
This paper discusses a novel hybrid approach to text classification that integrates a machine learning algorithm along with DistilBERT, a pre-trained deep learning framework for natural language processing, offers a base model fine-tuned on Indian Banking News-Events with a rule-based method that is used by filtering false positives and dealing with false negatives to enhance the results given by the previous classifier. The major benefit is that by incorporating unique rules for certain chaotic or overlapping categories that have not been effectively trained, the system can be quickly fine-tuned. This research also compares the effectiveness of state-of-art deep contextual language representation DistilBERT used in our proposed hybrid model with the most preferred context independent language representation TFIDF, on supervised learning of the classification of multiclass Banking News-Events. Both representations are fed into the machine learning classifiers, Logistic Regression, Linear SVC, Decision Tree, and Random Forest. The findings indicate that DistilBERT can better transfer generic domain knowledge to other domains as compared to the baseline TFIDF and results into higher accuracy with Random Forest. The result is further fed into the Rules based method as per our proposed model. And, our proposed hybrid model resulted into improved accuracy.
Downloads
Metrics
Article Details
Licensing
TURCOMAT publishes articles under the Creative Commons Attribution 4.0 International License (CC BY 4.0). This licensing allows for any use of the work, provided the original author(s) and source are credited, thereby facilitating the free exchange and use of research for the advancement of knowledge.
Detailed Licensing Terms
Attribution (BY): Users must give appropriate credit, provide a link to the license, and indicate if changes were made. Users may do so in any reasonable manner, but not in any way that suggests the licensor endorses them or their use.
No Additional Restrictions: Users may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.