Decoding Silent Speech Based on High-Density Surface Electromyogram Using Spatiotemporal Neural Network

• Main Authors: Xi Chen (Author), Xu Zhang (Author), Xiang Chen (Author), Xun Chen (Author)
• Format: Book
• Published: IEEE, 2023-01-01T00:00:00Z.
• Subjects: article
• Online Access: Connect to this object online.
• Notes: 1558-0210; 10.1109/TNSRE.2023.3266299
• Keywords: Silent speech recognition; high-density surface electromyography; spatiotemporal feature; language model; time sequence decoding; Medical technology; R855-855.5; Therapeutics. Pharmacology; RM1-950
• Source: IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol 31, Pp 2069-2078 (2023)

Finer-grained decoding at a phoneme or syllable level is a key technology for continuous recognition of silent speech based on surface electromyogram (sEMG). This paper aims at developing a novel syllable-level decoding method for continuous silent speech recognition (SSR) using spatio-temporal end-to-end neural network. In the proposed method, the high-density sEMG (HD-sEMG) was first converted into a series of feature images, and then a spatio-temporal end-to-end neural network was applied to extract discriminative feature representations and to achieve syllable-level decoding. The effectiveness of the proposed method was verified with HD-sEMG data recorded by four pieces of 64-channel electrode arrays placed over facial and laryngeal muscles of fifteen subjects subvocalizing 33 Chinese phrases consisting of 82 syllables. The proposed method outperformed the benchmark methods by achieving the highest phrase classification accuracy (97.17 &#x00B1; 1.53&#x0025;, <inline-formula> <tex-math notation="LaTeX">${p} &lt; 0.05$ </tex-math></inline-formula>), and lower character error rate (3.11 &#x00B1; 1.46&#x0025;, <inline-formula> <tex-math notation="LaTeX">${p} &lt; 0.05$ </tex-math></inline-formula>). This study provides a promising way of decoding sEMG towards SSR, which has great potential applications in instant communication and remote control.