A man paralyzed from the neck down used an implanted sensor that processed his brain signals to create text, achieving a typing speed rivaling that of his able-bodied peers.
The intracortical brain-computer interface (BCI) decoded his attempted handwriting movements from neural activity and translated them to text in real time at a rate of 90 characters per minute, more than double the previous record for typing with a BCI, reported Francis Willett, PhD, of Howard Hughes Medical Institute at Stanford University, and co-authors in Nature.
The 65-year-old man, referred to as T5 in the paper, had a spinal cord injury in 2007 that immobilized his limbs. Nine years later, as part of the BrainGate2 clinical trial, researchers placed two BCI chips, each the size of baby aspirin, in the part of his motor cortex that governs hand movement.
As the man imagined writing, a machine-learning algorithm used signals from individual neurons in his brain to recognize patterns he produced with each letter, allowing him to communicate as quickly as someone his age typing on a smartphone.
"We've learned that the brain retains its ability to prescribe fine movements a full decade after the body has lost its ability to execute those movements," Willett said in a statement.
Complicated intended movements that involve changing speeds and curved trajectories -- like handwriting -- can be interpreted more easily by artificial intelligence algorithms than simpler motions like straight lines, Willett noted. "Alphabetical letters are different from one another, so they're easier to tell apart," he said.
Commercially available assistive typing devices rely predominantly on being able to make eye movements or deliver voice commands, observed Pavithra Rajeswaran, MEng, and Amy Orsborn, PhD, both of the University of Washington in Seattle, in an accompanying commentary.
Eye-tracking keyboards can let people with paralysis type at around 47.5 characters per minute but have limitations: it's hard to reread an email to compose your reply while you're typing with your eyes, they noted.
So far, BCIs for typing have not been able to compete with simpler assistive technologies. But the approach in this study "leapfrogs far beyond past devices, in terms of both performance and functionality," Rajeswaran and Orsborn wrote.
At first, T5 concentrated on writing imaginary letters on lined paper with an imaginary pen. Then he was given groups of sentences and instructed to make a mental effort to write each one; later, he was asked to give answers to open-ended questions. A deep-learning model called a recurrent neural network learned the neural activity patterns he produced with each character.
Over time, the algorithm showed better ability to differentiate his neural firing patterns. Eventually, it was able to decipher the correct letter 94.1% of the time. An offline autocorrect function boosted that accuracy to over 99%.
This work could one day benefit people who've lost the use of their upper limbs or the ability to speak due to spinal cord injuries, strokes, or amyotrophic lateral sclerosis (ALS), the researchers said. At this point, however, it's a proof of concept that a high-performance handwriting BCI is possible: it is not a clinically viable system.
The approach still needs to demonstrate similar results in other people, allow text to be edited, and show it will perform well as neural activity patterns change, Willett and co-authors noted. The intracortical technology used in the study also needs to show longevity, safety, and efficacy before it can be used clinically.
Disclosures
The study was supported by the National Institute of Neurological Disorders and Stroke, NIH BRAIN Initiative, National Institute on Deafness and Other Communication Disorders, Howard Hughes Medical Institute, the U.S. Department of Veterans Affairs, L. and P. Garlick, S. and B. Reeves, the Simons Foundation, and the Wu Tsai Neurosciences Institute.
Researchers reported relationships with Neuralink, Paradromics, Synchron, Enspire DBS, Facebook Reality Labs, MIND-X, Inscopix, and Heal. Several authors are inventors on a patent application that covers the neural decoding approach in this work.
