Combining multiple classifiers for wrapper feature selection

Wrapper feature selection methods are widely used to select relevant features. However, wrappers only use a single classifier. The downside to this approach is that each classifier will have its own biases and will therefore select very different features. In order to overcome the biases of individual classifiers, this study introduces a new data mining method called wrapper-based decision trees (WDT), which combines different classifiers and uses decision trees to classify selected features. The WDT method combines multiple classifiers so selecting classifiers for use in the combinations is an important issue. Thus, we investigate how the number and nature of classifiers influence the results of feature selection. Regarding the number of classifiers, results showed that few classifiers selected more relevant features whereas many selected few features. Regarding the nature of classifier, decision tree classifiers selected more features and the features that generated accuracies much higher than other classifiers.