Multiclass Classifier for Stock Price Prediction

The stock market has been a crucial factor of investments in the financial domain. Risk modeling and profit generation heavily rely on the sophisticated and intricate stock movement prediction task. Stock Price forecasting is complex that could have a significant influence on the financial market. The Machine Learning (ML) type of artificial intelligence (AI) provides a more accurate forecast for binary and multiclass classification. Different effective methods have been recommended to resolve the problem in the binary classification case but the multiclass classification case is a more delicate one. This paper discusses the application of multiclass classifier mappings such as One v/s All (OvA) and One v/s One (OvO) for stock movement prediction. The proposed approach comprises four main steps: data collection, assign a multi-label (up, down, or same), discover the best classifier methods, and comparison of classifiers on evaluation metrics of 10k cross-validation for stock price movement. In this study, a stock NASDAQ dataset for about ten years of ten companies from yahoo finance on daily basis is used. The resultant Stock Price prediction uncovers Neural Network classifier has good performance in some case whereas Multiclass (One V/s One) and (One V/s All) have overall better performance among all other classifiers as AdaBoost, Support Vector Machine, OneR, Bagging, Simple Logistic, Hoeffding trees, PART, Decision Tree and Random Forest. The Precision, Recall, F-Measure, and ROC area comparison results show that Multiclass (One V/s All) is better than Multiclass (One V/s one). The proposed method Multiclass classification (One v/s All) yields an accuracy of 97.63% for average prediction performance on all ten stock companies, also the highest accuracy achieved as 98.7% for QCOM. The individual stock-wise evaluation of the Multiclass (One V/s All) classifier is found to achieve the highest accuracy among all other classifiers which is outperforming all the recent proposals.


Introduction
Despite the rapid development in the world's economy and technology, the stock market's passion and enthusiasm have never diminished. Although the stock market [1] is one of the most effective financial systems available, it is also unpredictable. The Stock market price forecast has continuously been an area for research and development. It is mainly since the market is non-linear, volatile, and dynamic, and unpredictable. Moreover, the stock market's groups and movements are affected by several economic factors such as political events, general economic conditions, commodity price index, investors' expectations, movements of other stock markets, the psychology of investors, etc. To minimize the high risk, the investor needs information as a reference for decision making of which stock they should buy, sell, and maintain for the future. Consequently, creating an accurate as well as consistent technique is not easy. In recent years to analyze the market trends, the trending technology proposed is machine learning with artificial intelligence [2].
Machine learning is the most powerful tool, including different algorithms to develop their performance in a specific case study effectively. Classification algorithms fall into two kinds: multiclass and binary. Binary classification is classifying instances right into one of two classes, while multiclass classification is identifying instances right into one of three or more unique classes. Multiclass classification is fundamental to a great deal of real-world machine learning applications that need the ability to immediately distinguish between hundreds of different classes. The issue of multiclass classification is come across in different applications such as Medicine, Market, Computer vision, Cybersecurity, etc. Many practical multiclass problems are labeling images, for example, face recognition, or tagging locations in vacation photos.
Most data relating to the stock market and financial fields are binary and multiclass classification. Most research studies have been conducted for stock market movement based on a non-linear SVM model [3] designed and implemented on the real-time stock market data. It used a novel stock technical data transformation technique, text feature extraction method, and non-linear SVM classification algorithm to predict the stock trend on a daily, weekly, monthly basis. According to previous research studies, most of the algorithm for classification is best suited with binary classification functionality but for multiclass, it leads to a challenging task. That's why multiclass classification attracts researchers for stock movement prediction.
In this stock movement prediction paper, the multiclass problem exists for multi-label classes as UP, DOWN, or SAME differentiating the market movement on daily basis. Although the multiclass problem [4] is more difficult by its very nature, it is possible to achieve significantly better than the random performance by using both the one v/s all and the one v/s one approach.The concept of using the multiclass classification mapping approach is to change the original multiclass problem into smaller-sized binary parts using a binarization approach which is less complicated for classification. It can be recognized that there is a significant absence of studies in the multiclass category for utilizing ONE v/s ONE as well as ONE v/s ALL techniques [5] in the stock market. More importantly, none of the studies have achieved great precision to predict the stock movement to reach maximum accuracy using multiclass classification mapping.
The main research questions to be answered are: 1. Comparison of multiclass classifier (One v/s One and One v/s All) against the most known classifiers like AdaBoost, Support Vector Machine, OneR, Bagging, Simple Logistic, Hoeffding trees, PART, Decision Tree, Random Forest, and Multilayer Perceptron for stock prediction.
2. Which is the best multiclass classifier mapping among One v/s One and One v/s All that achieved the highest accuracy?
The remainder of the research is structured in this manner. In section two, the related work is outlined. In section three, the research methodology is explained in detail. The results and discussion are found in section four. The conclusion and future works of the paper are explained in section five.

2.Related Work
Several recent studies have been conducted for the multiclass problem. The research study [6] modified the multiclass classification technique such as One v/s One, One v/s All, and also Directed Acyclic Graph (DAG) for binary classifiers. They used Iris, wine, glass, and vowel dataset for the research. The highest accuracy on wine data as 100% is using SVM. The One v/s All on the wine dataset is 96.66% and on the Glass dataset is 71.96%. Similarly, The One v/s One on wine dataset is 99.43% and on Glass dataset is 71.49%. The past research [7] provided a study on three sorts of multiclass classification mapping method for categorizing dataset that contains numerous classes on portable based mobile phone gait recognition. The methods were ONE vs. ALL, ONE vs. ONE as well as random correction code (RCC) with 5 sorts of random width variables included. The results showed that using the multiclass classification mapping method partially improved the overall accuracy especially on One v/s One they got 87.78% and One v/s All 70%.
Numerous studies on stock price movements have been completed using feature engineering, feature selection, and various classification techniques. Researchers [8] intended a unique feature engineering approach for multiclass classification for the stock forecast as well as this was taken into consideration the very first study to make use for multiclass classification making use of set techniques. That research study recommended a novel multiclass classification technique called Gradient Boosting Machine with Feature Engineering (GBM-wFE) and Principal Component Analysis (PCA) as the feature selection. It located that GBM-wFE outmatched as well as the overall forecast results of Mean Absolute Percentage Error (MAPE) 0.0406% was achieved. The authors [9] suggested a novel feature engineering technique to predict the stock prices based upon historic data utilizing both binary and also multiclass classification. The monthly stock movement was predicted for each month. The overall prediction result was improved by 25.64% compared to applying the same procedure without feature engineering. The research [10] was presented on the design and implementation of a novel binary classification framework that predicted stock market trends. Also explored and discovered the very best feature selection algorithm. PCA was found to be the very best contrasted to others.
There is rare research found on multiclass classification problems (One v/s One and One v/s All) on financial data such as stock price movement. Effective machine learning approaches offered a rapid evaluation of credit reports while updating older ones on a day-to-day time scale. Relevant studies [11] had based on credit history analyses utilizing the expert system. The study has been conducted on stock market data that shown neural networks and SVM exceed various other strategies by providing much better prediction accuracy. The ONE v/s ONE SVM (77.77 %) and ONE v/s ALL SVM (33.33 %) are made use of yet its performance was not to the anticipated accuracy to forecast corporate credit scores. The research [12] had been done on the Indian stock market on daily basis to predict the future moment of the stock trends using the Artificial Neural Networks (ANN) and SVM formula with Multiclass classification: One v/s All SVM (OVA-SVM) algorithms. It achieved the highest accuracy of 71.14% in the OVA-SVM model. Based on the above recent research works, it can be observed that there is a lack of researches for stock movement prediction to classify stock instances into a Multiclassification problem. The paper aims to explore a novel multiclass classifier to solve multiclass (up, down, same) issues for stock movement forecast. In this research, it discovers One v/s ALL multiclass classification mapping achieves the highest accuracy against all latest classifier technique on evaluation metrics of 10k cross-validation basis too. The proposed Multiclass classifiers modes and evaluation metrics are studied and applied in detail in this paper.

Research Methodology
In the proposed paper, research is based on the historical stock data that applies supervised machine learning methods. Our study framework is made up of four major phases such as dataset collection, pre-processing, applied Multiclass classification (One v/s One and One v/s All) classifiers, and evaluation analysis. Figure-1 highlights the general study structure in this study.

a. Tools and Software
This research was conducted using the Java application based on WEKA (Waikato Environment for Information Analysis)which was developed by Holmes, Donkin, & Witten in 1994 [13]. Twelve latest and known machine learning classifiers out of the available algorithms in WEKA are used in this research.

c. Pre-Processing
The study proposed a method to pre-process the data to produce a Multiclass class for day-to-day prediction and assign a label as Up, Down, or Same that shows in Table 2. The stock movement to contrast the forecasted as well as actual percentage change every day designates the class to day-to-day information.
To discover the day-to-day activity right here, stock activity is the difference between the daily close and open price: The stock cost motion in regards to the percentage (%) is computed as follows:

e. Proposed Multiclass Classifier
In multiclass classification, there are mostly 2 sorts of Multiclass classification strategies: One v/s All (OVA) and One v/s One (OVO) that commonly used. OVA and OVA facilitate the application of the data preprocessing techniques to balance the data before the training set goes to the classifier. The OVA approach takes one class as a minority and the remaining classes are combined and transformed into the majority class. This procedure is made for the n classes of the dataset. Multiple class labels are present in the dataset. Here; the variety of classifier models relies on the classification strategy we are putting on. The Confusion matrix is easy to derive but complex to recognize in Multiclass classification [27].
One v/s One: -N-class instances then N * (N-1)/ 2 binary classifier models. In the One-vs-One classification, for the N-class instances dataset, we have to generate the N * (N-1)/ 2 binary classifier models. Using this classification approach, we divided the primary dataset right into one dataset for each class opposite to every other class. The One v/s One approach divides a Multiclass classification right into one binary classification issue per each pair of classes.
One v/s All: -N-class instances after that N binary classifier models. In one v/s All classification, for the N-class instances dataset, we have to generate the N-binary classifier models. The variety of class tags present in the dataset and also the variety of generated binary classifiers should coincide. The One v/s All technique splits a Multiclass classification into one binary classification issue per class. The OVR minimizes the problem of multiclass classification to several binary category problems. It can be categorized right into One v/s All as well as One v/s One. The strategies established based on minimizing the Multiclass issue into numerous binary troubles can likewise be called problem transformation strategies. In a one v/s one strategy it divides the dataset right into the set (like displayed in number) and then does classification. Note that one hypothesis separates just two classes irrespective of the other classes. In a One v/s All strategy divide the datasets such that a hypothesis separates one class label from all of the rest that show in figure 3.The algorithm details and parameters in WEKA as default are shown in the following Table 3.

f. 10-k fold cross-validation and Evaluation Metrics
Examining a Machine Learning model can be rather tricky. Generally, we split the data established right into training and testing collections and also use the training readied to educate the model and also testing readied to evaluate the model. K-fold Cross-Validation (CV) gives a solution to this issue by splitting the data right into folds and also making sure that each fold is utilized as testing evaluated some factor. Over-fitting is the most common problem prevalent in most machine learning models. K-fold cross-validation can be conducted to verify if the model is over-fitted at all. The same process takes place for all k folds as shown in figure 4. Cross-validation [28] is a resampling procedure used to evaluate machine learning models on a limited data sample. Cross-validation is primarily used in applied machine learning to estimate the skill of a machine learning model on unseen data.

Figure 4: 10-k Cross-Validation
Accuracy is the most significant criterion in the evaluation of the performance success of the prediction methods. Therefore, the commonly used error metrics are used both in evaluating the results of the prediction models and in comparison with each other. Some metrics [29] such as TP rate, FP rate, precision, recall, F-Measures, ROC Curve are used to compare the performance success of the prediction models used in the present paper.

g. ROC Curve
Making predictions has become an essential part of every business enterprise and all academic fields. One critical aspect of evaluating and comparing these prediction models, algorithms, or technologies is the assessment of their accuracy. Receiver operating characteristic curves address that problem. They show how accurate a model, algorithm, or technology is, depending on its past results. Formally, a receiver operating characteristics (ROC) [30] graph is a technique for visualizing, organizing, and selecting classifiers, set of categories from the dataset, based on their performance.

4.Result and Discussion
The experimental conducted on entire 10 company's dataset on daily basis and tested using 10k fold crossvalidation for all the 12 classifiers as AdaBoost, Support Vector Machine (SVM), OneR, Bagging, Simple Logistic, Hoeffding trees, PART, Decision Tree, Random Forest, Multiclass (One V/s One) and (One V/s All). The accuracy of the correction is shown in Table 4.  Figure 7 shows the correct classification average accuracy of all 10 companies for each classifier. Horizontally, the classifier is listed and vertically the average correction accuracy is listed in the graph. The performance graph contains:  Table 4 shows the overall accuracy of each company and Figure 7 represents average accuracy for all companies for the various classifier based on that the preliminary prediction is Neural Network has good performance in some case whereas Multiclass (One V/s One) and (One V/s All) have overall better performance among all classifier. The comparison between Multiclass One V/s One (OVO) and One V/s All (OVA) on different heuristics as Precision, Recall, and F-Measure is given below. Table-5 shows Precision, Table-6 shows  Recall and Table-7 shows the F-Measures value of both classifiers for UP, DOWN, the SAME label as well as for Average.  The evaluation measures of Multiclass One v/s One and One v/s All as Precision, Recall, F-measure for all classes and its average is shown above in tables. Also, the average weighted ROC area is shown for both classifiers. The results of Multiclass One v/s All are better than One v/s One.

5.Achieved the Benchmark
This study results have achieved the highest accuracy on stock market data using Multiclass (One v/s All) among all other classifiers and also complete benchmark with the previous studies that are shown in table 8. The best accuracy achieved by the previous study [12] using Multiclass One v/s ALL (SVM) is 71.14% on the SBIN dataset whereas the proposed the Multiclass One v/s ALL (Logistic) achieved 98.71% for the QCOM company.

6.Conclusion and Future Works
Predicting the direction of movements of the stock market is important for the development of effective market trading strategies. It usually affects a financial trader's decision to buy or sell a stock. Successful prediction of stock prices may promise attractive benefits for investors. This study attempted to predict the direction of stock price movement in the NASDAQ Stock. This work collects NASDAQ listed stocks from yahoo finance for the last twenty years as a dataset. The dataset contains various companies such as AMD, CMCSA, CSCO, FAST, LRCX, MCHP, MSFT, NTAP, QCOM, and SWKS.
Daily stock movements are predicted for each day evaluations are made with the actual movement to validate the model. The technology uses the interface of Java and WEKA to judge varied styles of machine learning classifiers over the given dataset. Latest and known classifier techniques are applied on dataset such as Ada Boost, Support Vector Machine (SVM), Neural Network (NN) Multilayer-Perceptron, Random Forest, Decision Tree (J48), PART, Hoeffding Tree, Simple Logistic, Bagging, OneR, Multiclass Classifier (One V/s ALL), Multiclass Classifier (One V/s One). The study has tested all the techniques on 10k fold cross-validation on stock market movement as up, down, and same. After the analysis in the WEKA domain, it was found that the performance metric of Neural Network (NN) Multilayer-Perceptron, Multiclass classifier (One V/s One), and (One V/s All) are better.
The individual stock-wise evaluation of the Multiclass (One V/s All) classifier is the highest in accuracy among all classifier techniques. The highest accuracy achieved was 98.7% of QCOM. Also, the average prediction performance of the Multiclass (One V/s All) on all stock companies was found significantly 97.63% best than any other techniques. The Precision, Recall, and F-Measure for all 3 labels and average comparison show that Multiclass (One V/s All) is better than Multiclass (One V/s one). The ROC average weighted area shows Multiclass (One V/s All) as the best classifier among all companies.
Future expansion studies can consider selecting the best features of the dataset through feature selection or optimization for better performance. Other classification techniques under the reinforcement of the project can be to make the use of Machine Learning algorithms to more efficiently sort out the area where the data related to Sentiment Analysis is being obtained i.e. to more precisely understand the Tweet or Comment made by the user & use that data into the analysis and understanding of the Stock which will eventually produce better prediction results.