Simultaneous Heterogeneous Sensor Localization, Joint Tracking, and Upper Extremity Modeling for Stroke Rehabilitation

Jiun Fu Chen, Chieh Chih Wang, Eric Hsiao Kuang Wu, Cheng Fu Chou

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


In stroke rehabilitation systems and applications, reliability, accuracy, and occlusion should be taken into consideration. Unfortunately, most existing approaches focus primarily on the first two issues. However, during the stroke rehabilitation process, occlusion leads to incorrect judgements even for medical staff. In order to tackle these three important issues simultaneously, we propose a heterogeneous sensor fusion framework composed of an RGB-D camera and a wearable device to consider occlusion and provide robust joint locations for rehabilitation. To fuse multiple sensor measurements when compensating for occlusion, we apply heterogeneous sensor simultaneous localization, tracking, and modeling to estimate the locations of joints and sensors and construct an upper extremity model for occlusion situation. Virtual measurements based on this model are used to estimate the joint's location during occlusion, and a virtual relative orientation technique is applied to relax system limitations regarding orientation. Experimental results using the proposed approach with synthetic data and data collected from ten subjects show a 4.6 cm error on average and about 15 cm error on average during occlusion. This constitutes a more robust approach for stroke patients which takes into account these three important issues.

Original languageEnglish
Article number8967220
Pages (from-to)3570-3581
Number of pages12
JournalIEEE Systems Journal
Issue number3
StatePublished - Sep 2020


  • Motion tracking
  • RGB-D camera
  • sensor fusion
  • stroke rehabilitation
  • upper extremity model
  • wearable device


Dive into the research topics of 'Simultaneous Heterogeneous Sensor Localization, Joint Tracking, and Upper Extremity Modeling for Stroke Rehabilitation'. Together they form a unique fingerprint.

Cite this