Comparative Analysis of Machine Learning Algorithms for Stock Price Forecasting on Big Data Platforms
Author(s): Dr. Thallapally Sameera Pallavi
Publication #: 2610021
Date of Publication: 09.04.2022
Country: India
Pages: 1-21
Published In: Volume 8 Issue 2 April-2022
DOI: https://doi.org/10.62970/IJIRCT.v8.i2.2610021
Abstract
There is mixed evidence that modern machine learning models are able to outperform naive benchmarks in predicting the price of the share. This study benchmarks ridge regression, random forest, gradient-boosted trees, multilayer perceptron, and support vector regression against a random-walk baseline for the next-day price prediction problem and quantifies the cost scale of training in the library of a single node and in Apache Spark MLlib. The data are 619,040 daily records for 505 S&P 500 stocks from 2013 to 2018. A total of nineteen technical features were calculated, the model was trained from 2013 to 2015 and the forecasts were evaluated sequentially on 138,536 stock-days from 2017 to early 2018. No algorithm outperformed the random walk. All of the models had negative out of sample r-squareds and no significant improvements were seen in Diebold-Mariano tests, with directional accuracy ranging from 49.7 percent to 50.2 percent. The random forest yielded a gross Sharpe ratio of 0.94 which was statistically significant from zero. For random forest training, Spark was up to 1.7 times faster than scikit-learn for the 1.38M rows, but for ridge regression and boosting training, Spark was significantly slower for one machine; and for kernel regression, it was impractical beyond tens of thousands of rows. The results confirm the need for evaluation on the basis of benchmarks and for matching the platform with the algorithm and the volume of data.
Keywords: stock price forecasting; machine learning; big data; Apache Spark; random forest; gradient boosting; efficient market hypothesis
Download/View Count: 7
Share this Article