Title: Evaluating the performance of machine learning models for automatic diagnosis of patients with schizophrenia based on a single site dataset of 440 participants

Lung Hao Lee, Chang Hao Chen, Wan Chen Chang, Po Lei Lee, Kuo Kai Shyu, Mu Hong Chen, Ju Wei Hsu, Ya Mei Bai, Tung Ping Su, Pei Chi Tu

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Support vector machines (SVMs) based on brain-wise functional connectivity (FC) have been widely adopted for single subject prediction of patients with schizophrenia, but most of them had small sample size. This study aimed to evaluate the performance of SVMs based on a large single-site dataset and investigate the effects of demographic homogeneity and training sample size on classification accuracy. Methods: The resting fMRI dataset comprised 220 patients with schizophrenia and 220 healthy controls. Brain-wise FCs was calculated for each participant and linear SVMs were developed for automatic classification of patients and controls. First, we evaluated the SVMs based on all participants and homogeneous sub-samples of men, women, younger (18-30 years) and older (31-50 years) participants by 10-fold nested cross-validation. Then, we hold out a fixed test set of 40 participants (20 patients, 20 controls) and evaluated the SVMs based on incremental training sample sizes (N=40,80…400). Results: We found that the SVMs based on all participants had accuracy of 85.05 %. The SVMs based on male, female, young and older participants yielded accuracy of 84.66%, 81.56%, 80.50% and 86.13% respectively. Although the SVMs based on older subsamples had better performance than those based on all participants, they generalized poorly to younger participants (77.24%). For incremental training sizes, the classification accuracy increased stepwise from 72.6% to 83.3%, with >80% accuracy achieved with sample size > 240. Conclusions: The findings indicate that SVMs based on a large dataset yield high classification accuracy and establishing models using a large sample size with heterogeneous properties are recommended for single subject prediction of schizophrenia.

Original languageEnglish
JournalEuropean Psychiatry
DOIs
StateAccepted/In press - 2021

Keywords

  • Automatic classification
  • Functional connectivity
  • Homogeneous
  • Schizophrenic disorder
  • Support vector machine
  • Training sample size

Fingerprint

Dive into the research topics of 'Title: Evaluating the performance of machine learning models for automatic diagnosis of patients with schizophrenia based on a single site dataset of 440 participants'. Together they form a unique fingerprint.

Cite this