Optimized Auto Encoder on High Dimensional Big Data Reduction: an Analytical Approach

Abstract

Big data comprises of huge volume of data, which is exponentially increasing with time. Since the data is
too large in size; the traditional data management tools are ineffective in processing these data effectively. The big
data encompasses huge count of variables, hence analyzing each of the variables at a microscopic level is not
feasible, as it might consume days or even months to have a meaningful analysis. This is time-consuming and
costlier. Therefore, the Dimensionality Reduction (DR) techniques can be utilized. In general, the DR is a technique
for reducing the count of input variables with fewer losses. These input features can cause deprived performance for
ML algorithms. This paper introduces an optimized auto-encoder based dimensionality reduction model to deal
with large datasets. The weight of the auto-encoder is fine-tuned by a selfadaptive Bumble Bees Mating
Optimization (SA-BBMO) algorithm, which is the conceptual upgrading of standard BBMO. Further, to validate
the appropriateness of the projected dimensionality reduction model, the experiments are conducted using big
datasets. The corresponding results acquired are compared over the nonlinear dimensionality reduction techniques
like PCA, K-PCA, LDA etc, in terms of Reconstruction error, Convergence, V-Measures, Silhouet Coefficient and
Computation Time.