Tests and variables selection on regression analysis for massive datasets

Tsai Hung Fan, Kuang Fu Cheng

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

According to Lindley's paradox, most point null hypotheses will be rejected when the sample size is too large. In this paper, a two-stage block testing procedure is proposed for massive data regression analysis. New variables selection criteria incorporating with classical stepwise procedure are also developed to select significant explanatory variables. Our approach is not only simple in computation for massive data but also confirmed by the simulation study that our approach is more accurate in the sense of achieving the nominal significance level for huge data sets. A real example with moderate sample size verifies that the proposed procedure is accurate compared with the classical method, and a huge real data set is also demonstrated to select appropriate regressors.

Original languageEnglish
Pages (from-to)811-819
Number of pages9
JournalData and Knowledge Engineering
Volume63
Issue number3
DOIs
StatePublished - Dec 2007

Keywords

  • Hypothesis testing
  • Lindley's paradox
  • Massive data
  • Regression analysis
  • Stepwise selection

Fingerprint

Dive into the research topics of 'Tests and variables selection on regression analysis for massive datasets'. Together they form a unique fingerprint.

Cite this