When should we ignore examples with missing values?

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

In practice, the dataset collected from data mining usually contains some missing values. It is common practice to perform case deletion by ignoring those data with missing values if the missing rate is certainly small. The aim of this paper is to answer the following question: When should one directly ignore sampled data with missing values? By using different types of datasets having various numbers of attributes, data samples, and classes, it is found that there are some specific patterns that can be considered for case deletion over different datasets without significant performance degradation. In particular, these patterns are extracted to act as the decision rules by a decision tree model. In addition, a comparison is made between cases with deletion and imputation over different datasets with the allowed missing rates and the decision rules. The results show that the classification performance results obtained by case deletion and imputation are similar, which demonstrates the reliability of the extracted decision rules.

Original languageEnglish
Pages (from-to)53-63
Number of pages11
JournalInternational Journal of Data Warehousing and Mining
Volume13
Issue number4
DOIs
StatePublished - 1 Oct 2017

Keywords

  • Case Deletion
  • Categorical Data
  • Classification
  • Data Mining
  • Imputation
  • Machine Learning
  • Missing Values
  • Numerical Data

Fingerprint

Dive into the research topics of 'When should we ignore examples with missing values?'. Together they form a unique fingerprint.

Cite this