”This technology has important scientific significance. For the first time, an invasive brain-computer interface system has enabled patients who have completely lost voluntary muscle control to communicate with foreign languages, even with their eyes closed.” Huazhong University of Science and Technology School of Artificial Intelligence and Automation Wu Dongrui, director of the Brain-Computer Interface and Machine Learning Laboratory, said.
The technology mentioned by Wu Dongrui is the latest research result of brain-computer interface recently. An ALS patient realized the reading of his brain signals through the brain-computer interface system, and completed the complete expression of the sentence through the selection and combination of letters on the machine.
A sentence that a healthy person can blurt out in just a few seconds is almost an “impossible task” for patients with severe ALS.
And through the brain-computer interface system, the 36-year-old patient from Germany realized an instant expression of his thoughts to his 4-year-old son – “Son, I love you. You want to watch Disney’s Robin Hood with me.” ?”
Invasive brain-computer interface system “implanted electrodes + neurofeedback” with a spelling speed of 1 character per minute
The ideal brain-computer interface is to detect the electrical activity of neurons through implanted electrodes, develop decoding algorithms, and directly translate and express the patient’s thoughts. But the structure of the human brain is too complex to recognize and decode directly.
The technology was jointly developed by the Wyss Center for Biological and Neural Engineering in Switzerland and the scientific research team of the University of Tübingen in Germany. They did not develop a decoding algorithm to decipher the patient’s thoughts, but through the invasive brain-computer interface system “implantable electrodes + neurofeedback” ” way to capture the patient’s intent.
Through brain surgery, the researchers implanted two 3.2-millimeter-sized microelectrode arrays on the surface of the patient’s cerebral cortex. Each electrode array has 64 needle electrodes to record neural signals.
This patient is a fully locked-in syndrome patient with conscious and cognitive abilities. Since he has completely lost voluntary muscle control, he can only adjust the firing rate of neurons in the brain based on auditory feedback.
Through this new technology, he selects one letter at a time, and then forms words, phrases and sentences to express his needs, and then conveys his ideas through the machine, realizing communication with the outside world at a speed of about one character per minute.
When the machine gives a letter and at the same time translates the patient’s EEG recorded by the electrodes into “yes” or “no”, and compiles it into the frequency of the sound, the patient gives feedback by judging whether the frequency of the sound is consistent with his own thoughts. , regulate their own brain activity, and finally make the machine display accurate sentences.
That is to say, when the neural firing rate is higher than a certain threshold, the machine learning is used to decode in real time to determine that it wants to express “yes”; when the neural firing rate is lower than a certain threshold, it is determined that the thought is “no”.
According to the paper’s results, “On 107 of the 135 days of the study, he was able to match a range of target tones with about 80 percent accuracy. And, on only 44 of those 107 days, the patient Can express intelligible sentences.”
Xu Fang, an associate researcher at Shenzhen University of Technology, Chinese Academy of Sciences, and Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, believes that this method is ingenious. It adopts a relatively easy-to-implement technology to realize the communication between gradually frozen people in a completely locked state. In essence, the function implemented by this brain-computer interface is the choice of “yes” or “no” in the patient’s mind, and the patient directly feedbacks whether the judgment is accurate.
Wang Haochong, founder and CEO of Zhentai Intelligence, said that from the perspective of practical application, the technology provides a more direct way of communication and can better express the patient’s intention. But he also pointed out that the speed of spelling based on auditory feedback needs to be improved. “Currently, the technology’s spelling speed is 1 character per minute, and it can be further improved in terms of interaction efficiency.”
Brain-computer interface becomes the only way for patients with complete locked-in syndrome to communicate with the outside world
Well-known ALS patient Stephen William Hawking, a well-known physicist, used Intel’s ACAT to communicate with others, which captures eye and facial muscle movements and translates his thoughts. “Translate” into words.
So, why can patients with severe ALS can only communicate with the outside world through a brain-computer interface? What difficulties do ALS patients have in the process of communicating with the outside world?
The full name of amyotrophic lateral sclerosis is amyotrophic lateral sclerosis, which is a motor nerve cell disease caused by the degeneration of motor neurons in the central nervous system that control skeletal muscles.
Achieving communication with patients in a completely locked state is an important challenge for brain-computer interfaces. Due to differences in neurons that control extraocular muscles and skeletal muscles, some patients still have the ability to control some muscles and eye movements. Some ALS patients can communicate with the outside world through muscle movement, eye tracking, etc.
However, as the disease worsens, these patients will gradually lose the ability to move muscles and eyeballs as if they were “frozen”. Later in the course of the disease, it will enter a “total atresia” state, and even swallowing becomes difficult.
The patient with severe ALS in the study was diagnosed 6 years ago when his eyes were still moving. But now, due to the inability of skeletal muscles, limbs cannot move, pronunciation is difficult, and it is no longer possible to communicate through muscle movement, eye tracking, etc. In addition, non-invasive brain-computer interface systems will gradually become unusable due to the gradual weakening of neural signals, such as motor imagery.
The patient’s cognitive function has not been greatly affected, and he can still have clear consciousness and thinking, which is also an important prerequisite for him to communicate with the outside world through auditory feedback and the brain-computer interface system.
Wu Dongrui believes that this technology can allow patients to have the ability to communicate with the outside world even when they cannot move anywhere in the body, improving the quality of life of patients. Therefore, invasive brain-computer interface systems may be their “ultimate means” to communicate with the outside world.
At present, there is no clear cure for ALS, and the World Health Organization has identified ALS as “one of the five major terminal diseases.” The torture of illness and the “zero communication” with the outside world make the patients with ALS suffer physically and mentally.
Xu Fang pointed out that it is worth noting that in the process of this research, the urgent desire of ALS patients to establish communication with the outside world can be clearly seen. “The patient’s willpower is very strong. Even if his body is completely immobile, he still insists on a strong desire to express himself, trying various methods to make his brain adapt to the operation of the machine.”
In addition, the technology is also used in a looser environment, the team said, the brain-computer interface system can theoretically be used at home. Therefore, it is more convenient for patients and their families to use the relevant systems to establish communication, and the familiar environment is also conducive to the daily care of patients’ families.
Although patients have established an effective way of communicating with the outside world through the brain-computer interface system, whether it can be used sustainably is still a problem that cannot be ignored.
According to the results of the paper, after using the technology for a period of time, the patient’s free spelling ability has declined, and currently the main answer is “yes” or “no”. “It could be that the scab around the implanted electrodes is blocking the nerve signals, or it could be that the patient’s brain is losing control of the device,” Wu Dongrui said.
The invasive brain-computer interface system used in this study faces technical challenges such as limited electrode lifetime, the need to find suitable feedback modes, and low information transmission rates. Wu Dongrui pointed out that the brain-computer interface system still has the following three room for optimization at the technical level.
First, the signal quality of the implanted electrodes will slowly degrade with the implantation time, which will slowly degrade the system performance. Therefore, electrodes with better biocompatibility are needed. “The lifespan of implanted electrodes is affected by scabbing of peripheral neurons, so in general their lifespan is less than five years,” he said.
Second, the location and quantity of electrode implantation can also be further optimized to extract more stable and differentiated features.
Third, even if the electrodes are fine, the quality of the nerve signal is inherently unstable and changes over time. Therefore, the parameters of the decoding algorithm may need to be calibrated on a daily or irregular basis.
Humans’ pursuit of information exchange efficiency has never stopped. From the ancient “flying pigeons biography” to today’s “network video”, it is all to improve the efficiency of information exchange.
Wang Haochong believes that the current bottlenecks of brain-computer interface technology mainly lie in three aspects: electrodes, chips and algorithms. Take the brain-computer interface device in this study as an example, the device is expensive, and the cost of using it in the first two years is close to 500,000 US dollars (about 3.18 million yuan).
”The high cost of early surgery is the main reason for the heavy economic burden on patients. With the continuous maturity of technology, the cost of system hardware can be reduced on a large scale, but more suitable implanted electrode materials and more efficient brain-computer interaction algorithms are still important. The large-scale application of this technology needs to break through the difficulties.” He said.
So, how will the field of brain-computer interface technology develop in the future? According to a study in 2022, this field will usher in at least the following four development trends:
First, the brain-computer information interaction means will change from the original electricity-based to the integration of various means such as electricity, light, magnetism, and sound; Second, the enhancement capability of brain-computer fusion has gradually changed from behavior enhancement to various perception enhancements, and even advanced cognition enhancement; third, the hardware of brain-computer technology research is flexible, wireless, and more miniaturized, more high-throughput and low-power consumption. Fourth, brain-computer interface technology is expected to lead the development of new technologies in clinical application fields such as diagnosis, treatment, and rehabilitation of neurological/mental diseases.
In general, brain-computer interface technology expresses ideas in the most direct way by recognizing the intention of the brain, which greatly improves the efficiency of people’s access to information. From improving the lifestyle of paralyzed patients to clinical treatment of neuropsychiatric diseases, brain-computer interfaces still have a long way to go.
Although there are still many technical bottlenecks in this field, with people’s continuous cognition of the brain mechanism and the continuous development of engineering technology, the brain-computer interface technology will become more and more mature. As the ultimate means of human-computer interaction, the development of brain-computer interface technology is worth looking forward to.
