Neurotechnology
Speech Synthesis from Neural Decoding of Spoken Sentences
Anumanchipalli, Chartier, Chang · 2019 · Nature
Synthesized intelligible speech directly from cortical activity in the vocal motor area.
Research objective
Reconstruct audible speech from neural signals as a step toward voice restoration for ALS and locked-in patients.
Methodology
Recorded high-density ECoG arrays over speech-motor cortex while participants spoke aloud. A two-stage decoder mapped neural activity to articulatory movements, then to acoustic waveforms.
Key findings
- Listeners could identify reconstructed words with high accuracy.
- Articulatory representations generalized across sentences.
- Performance degraded gracefully with reduced electrode counts.
Strengths
- First convincing demonstration of cortical speech synthesis.
- Articulatory intermediate representation improves generalization.
Limitations
- ECoG is semi-invasive and requires craniotomy.
- Tested on participants able to vocalize; generalization to paralyzed users required follow-up work.
Practical implications
- Foundation for the 2023 Chang lab work that restored speech to a paralyzed patient at ~78 words/minute.
- Validates motor-cortex decoding for naturalistic communication.
Related entities
Related research
High-Performance Brain-to-Text Communication via Handwriting
An intracortical BCI decoded imagined handwriting into text at 90 characters per minute.
Read summary
Millisecond-Timescale, Genetically Targeted Optical Control of Neural Activity
Founding paper of optogenetics - using light to control genetically targeted neurons with millisecond precision.
Read summary