Parallel Discretization of Data Preparation Optimization in Data Mining

Abstract:

In data mining, the discretization of data can improve the efficiency of data mining effectively. In this paper, we propose a data preprocessing algorithm (AOA) to obtain the optimal discretization using parallel comparison. For different data sets, we first perform the feature detection of the data set to obtain the distribution characteristics of the data set. Then the outliers of the data set are detected according to the distribution characteristics. IN addition, the discretization results are obtained by comparing the minimum Euclidean distance of the entropy, the variance index and the stability parameter of the different discretization methods. In simulation experiment, we compare the AOA with four typical data discretization methods in different databases by running the association rule mining algorithm on the discretization data obtained using AOA and other four methods, respectively. The results show that, under different minimum support thresholds, the number of association rules extracted from the discretization data obtained using AOA is the least, indicating higher efficiency of AOA.