Evaluating the performance of machine learning models for automatic diagnosis of patients with schizophrenia based on a single site dataset of 440 participants

Lung Hao Lee, Chang Hao Chen, Wan Chen Chang, Po Lei Lee, Kuo Kai Shyu, Mu Hong Chen, Ju Wei Hsu, Ya Mei Bai, Tung Ping Su, Pei Chi Tu

Research output: Contribution to journalArticlepeer-review

Abstract

Background Support vector machines (SVMs) based on brain-wise functional connectivity (FC) have been widely adopted for single-subject prediction of patients with schizophrenia, but most of them had small sample size. This study aimed to evaluate the performance of SVMs based on a large single-site dataset and investigate the effects of demographic homogeneity and training sample size on classification accuracy. Methods The resting functional Magnetic Resonance Imaging (fMRI) dataset comprised 220 patients with schizophrenia and 220 healthy controls. Brain-wise FCs was calculated for each participant and linear SVMs were developed for automatic classification of patients and controls. First, we evaluated the SVMs based on all participants and homogeneous subsamples of men, women, younger (18-30 years), and older (31-50 years) participants by 10-fold nested cross-validation. Then, we hold out a fixed test set of 40 participants (20 patients and 20 controls) and evaluated the SVMs based on incremental training sample sizes (N = 40, 80, ..., 400). Results We found that the SVMs based on all participants had accuracy of 85.05%. The SVMs based on male, female, young, and older participants yielded accuracy of 84.66, 81.56, 80.50, and 86.13%, respectively. Although the SVMs based on older subsamples had better performance than those based on all participants, they generalized poorly to younger participants (77.24%). For incremental training sizes, the classification accuracy increased stepwise from 72.6 to 83.3%, with >80% accuracy achieved with sample size >240. Conclusions The findings indicate that SVMs based on a large dataset yield high classification accuracy and establish models using a large sample size with heterogeneous properties are recommended for single subject prediction of schizophrenia.

Original languageEnglish
Article numbere1
JournalEuropean Psychiatry
Volume65
Issue number1
DOIs
StateAccepted/In press - 2021

Keywords

  • Automatic classification
  • functional connectivity
  • homogeneous
  • schizophrenic disorder
  • support vector machine
  • training sample size

Fingerprint

Dive into the research topics of 'Evaluating the performance of machine learning models for automatic diagnosis of patients with schizophrenia based on a single site dataset of 440 participants'. Together they form a unique fingerprint.

Cite this