Deep Neural Network-Based Video Processing to Obtain Dual-Task Upper-Extremity Motor Performance Toward Assessment of Cognitive and Motor Function

Dementia is an increasing global health challenge. Motoric Cognitive Risk Syndrome (MCR) is a predementia stage that can be used to predict future occurrence of dementia. Traditionally, gait speed and subjective memory complaints are used to identify older adults with MCR. Our previous studies indic...

Full description

Saved in:
Bibliographic Details
Main Authors: Zilong Liu (Author), Changhong Wang (Author), Guanzheng Liu (Author), Bijan Najafi (Author)
Format: Book
Published: IEEE, 2023-01-01T00:00:00Z.
Subjects:
Online Access:Connect to this object online.
Tags: Add Tag
No Tags, Be the first to tag this record!

MARC

LEADER 00000 am a22000003u 4500
001 doaj_e30c607ee3d44b4a8d45e40dd54fa89a
042 |a dc 
100 1 0 |a Zilong Liu  |e author 
700 1 0 |a Changhong Wang  |e author 
700 1 0 |a Guanzheng Liu  |e author 
700 1 0 |a Bijan Najafi  |e author 
245 0 0 |a Deep Neural Network-Based Video Processing to Obtain Dual-Task Upper-Extremity Motor Performance Toward Assessment of Cognitive and Motor Function 
260 |b IEEE,   |c 2023-01-01T00:00:00Z. 
500 |a 1558-0210 
500 |a 10.1109/TNSRE.2022.3228073 
520 |a Dementia is an increasing global health challenge. Motoric Cognitive Risk Syndrome (MCR) is a predementia stage that can be used to predict future occurrence of dementia. Traditionally, gait speed and subjective memory complaints are used to identify older adults with MCR. Our previous studies indicated that dual-task upper-extremity motor performance (DTUEMP) quantified by a single wrist-worn sensor was correlated with both motor and cognitive function. Therefore, the DTUEMP had a potential to be used in the diagnosis of MCR. Instead of using inertial sensors to capture kinematic data of upper-extremity movements, here we proposed a deep neural network-based video processing model to obtain DTUEMP metrics from a 20-second repetitive elbow flexion-extension test under dual-task condition. In details, we used a deep residual neural network to obtain joint coordinate set of the elbow and wrist in each frame, and then used optical flow method to correct the joint coordinates generated by the neural network. The coordinate sets of all frames in a video recording were used to generate an angle sequence which represents rotation angle of the line between the wrist and elbow. Then, the DTUEMP metrics (the mean and SD of flexion and extension phase) were derived from angle sequences. Multi-task learning (MTL) was used to assess cognitive and motor function represented by MMSE and TUG scores based on DTUEMP metrics, with single-task learning (STL) linear model as a benchmark. The results showed a good agreement (r <inline-formula> <tex-math notation="LaTeX">$\ge0.80$ </tex-math></inline-formula> and ICC <inline-formula> <tex-math notation="LaTeX">$\ge0.58$ </tex-math></inline-formula>) between the derived DTUEMP metrics from our proposed model and the ones from clinically validated sensor processing model. We also found that there were correlations with statistical significance (p &#x003C; 0.05) between some of video-derived DTUEMP metrics (i.e. the mean of flexion time and extension time) and clinical cognitive scale (Mini-Mental State Examination, MMSE). Additionally, some of video-derived DTUEMP metrics (i.e. the mean and standard deviation of flexion time and extension time) were also associated with the scores of timed-up and go (TUG) which is a gold standard to measure functional mobility. Mean absolute percentage error (MAPE) of MTL surpassed that of STL (For MMSE, MTL: 18.63&#x0025;, STL: 23.18&#x0025;. For TUG, MTL: 17.88&#x0025;, STL: 22.53&#x0025;). The experiments with different light conditions and shot angles verified the robustness of our proposed video processing model to extract DTUEMP metrics in potentially various home environments (r <inline-formula> <tex-math notation="LaTeX">$\ge0.58$ </tex-math></inline-formula> and ICC <inline-formula> <tex-math notation="LaTeX">$\ge0.71$ </tex-math></inline-formula>). This study shows possibility of replacing sensor processing model with video processing model for analyzing the DTUEMP and a promising future to adjuvant diagnosis of MCR via a mobile platform. 
546 |a EN 
690 |a Dementia 
690 |a motoric cognitive risk syndrome 
690 |a telehealth 
690 |a tele-medicine 
690 |a deep residual neural network 
690 |a mobile health 
690 |a Medical technology 
690 |a R855-855.5 
690 |a Therapeutics. Pharmacology 
690 |a RM1-950 
655 7 |a article  |2 local 
786 0 |n IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol 31, Pp 574-580 (2023) 
787 0 |n https://ieeexplore.ieee.org/document/9978749/ 
787 0 |n https://doaj.org/toc/1558-0210 
856 4 1 |u https://doaj.org/article/e30c607ee3d44b4a8d45e40dd54fa89a  |z Connect to this object online.