Effective Phoneme Decoding With Hyperbolic Neural Networks for High-Performance Speech BCIs

• Main Authors: Xianhan Tan (Author), Qi Lian (Author), Junming Zhu (Author), Jianmin Zhang (Author), Yueming Wang (Author), Yu Qi (Author)
• Format: Book
• Published: IEEE, 2024-01-01T00:00:00Z.
• Subjects: article
• Online Access: Connect to this object online.

Summary

Objective: Speech brain-computer interfaces (speech BCIs), which convert brain signals into spoken words or sentences, have demonstrated great potential for high-performance BCI communication. Phonemes are the basic pronunciation units. For monosyllabic languages such as Chinese Mandarin, where a word usually contains less than three phonemes, accurate decoding of phonemes plays a vital role. We found that in the neural representation space, phonemes with similar pronunciations are often inseparable, leading to confusion in phoneme classification. Methods: We mapped the neural signals of phoneme pronunciation into a hyperbolic space for a more distinct phoneme representation. Critically, we proposed a hyperbolic hierarchical clustering approach to specifically learn a phoneme-level structure to guide the representation. Results: We found such representation facilitated greater distance between similar phonemes, effectively reducing confusion. In the phoneme decoding task, our approach demonstrated an average accuracy of 75.21% for 21 phonemes and outperformed existing methods across different experimental days. Conclusion: Our approach showed high accuracy in phoneme classification. By learning the phoneme-level neural structure, the representations of neural signals were more discriminative and interpretable. Significance: Our approach can potentially facilitate high-performance speech BCIs for Chinese and other monosyllabic languages.

Item Description

1534-4320
1558-0210
10.1109/TNSRE.2024.3457313