How to generate higher-quality photographs stays the core problem dealing with scientists.
Recently, Mingshi Brain-Machine Interface, a rising star in the home brain-computer interface area, introduced that it has achieved the world’s first practical interactive verification of visible reconstruction for “complex graphics + multiple colors”.
According to a reporter from the Science and Technology Innovation Board Daily, much like the reconstruction of motor and language capabilities, the restore and reconstruction of imaginative and prescient is one other essential route for brain-computer interface expertise in the medical area. Its historical past will be traced again to the Nineties, but it surely nonetheless faces an insurmountable technological hole, specifically find out how to obtain the leap from easy mild spots to complicated graphics and even coloration notion in visible notion by electrical stimulation of the visible cortex of the mind.
“The significance of Mingshi Brain-Machine Interface’s current research lies in the dynamic analysis and reconstruction of graphic outlines and basic colors. This marks that China has entered the world’s first echelon in this field and proposes a highly promising technological solution. However, whether it will ultimately succeed still needs to be verified by future public, rigorous scientific data and large-scale clinical trials,” Dr. Liu Bing, the founder and CEO of Mingshi Brain-Machine Interface, advised the Science and Technology Innovation Board Daily reporter.
From Retinal Prostheses to Visual Cortex Prostheses
When mild enters the eye, it first passes by the cornea and lens, the outer and center layers of the eye, after which reaches the retina at the again of the eye. The photoreceptor cells in the retina convert the mild indicators into organic present pulses, that are transmitted to the visible heart of the mind by way of the optic nerve. After complicated neural processing, the clear visible photographs we understand are lastly shaped.
This is the strategy of human imaginative and prescient. Any downside in this course of could result in visible impairment and even blindness. For instance, cataracts could make the lens cloudy, blocking the regular passage of sunshine; some retinal illnesses comparable to age-related macular degeneration and diabetic retinopathy can harm or kill photoreceptor cells, stopping them from successfully receiving mild indicators; and situations comparable to glaucoma, trauma, or tumor compression could harm the optic nerve or the visible heart of the mind, ensuing in blocked sign transmission.
To assist blind individuals “see the light again”, scientists have developed visible prostheses that may present “artificial vision” for the blind. According to the implantation website, visible prostheses are primarily divided into two classes: one is the retinal prosthesis that focuses on intraocular restore, and the different is the visible cortex prosthesis that tries to bypass the eye and immediately act on the visible cortex of the mind.
Among them, retinal prostheses are at present extra broadly used. Its core precept is to exchange or restore the photoreceptor cells in the retina which have misplaced their perform by a man-made gadget, thereby serving to sufferers restore some visible notion. Globally, a number of landmark merchandise and firms have emerged in this area. For instance, the Argus II epiretinal prosthesis developed by Second Sight in the United States was authorised by the US FDA in 2013, changing into the first retinal prosthesis to obtain FDA approval. In the similar 12 months, the Alpha-IMS subretinal prosthesis developed by Retina Implant AG in Germany additionally acquired authorization from the European EMA and was allowed to enter the market.
However, as a consequence of numerous constraints, the commercialization of those merchandise after their launch was not clean, and so they progressively pale out of the market. On the one hand, the excessive prices of R & D, manufacturing, surgical implantation, and postoperative upkeep enormously restricted the popularization of the merchandise; on the different hand, the early retinal prosthesis merchandise have been nonetheless in their infancy in phrases of expertise, and the visible decision they might present was restricted, removed from pure imaginative and prescient, making it tough to really meet the precise wants of sufferers.
Recently, at the “2025 Brain-Computer Interface Conference” held in Shanghai, Yu Xinguang, the director of the Brain Science and Neurology Center of Guangdong Heyou International Hospital, additionally systematically summarized the deficiencies and limitations of synthetic retinal prosthesis expertise.
He identified that amongst a number of visible restore options comparable to retinal prostheses, optic nerve prostheses, and visible cortex prostheses, retinal prostheses are relevant to the most restricted affected person group, primarily for sufferers who’re blind as a consequence of retinal degeneration, accounting for under about 10% of all blind individuals. In distinction, visible cortex prostheses are virtually relevant to all completely blind sufferers for implantation, with the widest scope of software.
“Meanwhile, the field of view of these retinal prostheses is usually relatively narrow. For example, the field of view of the Argus II is less than 30 degrees. Users have to frequently turn their heads to scan the surrounding scene to compensate for the limited field of view. But this not only affects visual perception and the efficiency of daily activities but also easily causes discomfort over time,” Yu Xinguang stated.
Against this background, the business has progressively shifted its analysis focus from retinal prostheses to visible cortex prostheses. As early as May 2019, Second Sight introduced that it might cease producing the Argus II and absolutely shift its R & D assets to the improvement of the Orion visible cortex prosthesis system.
Breakthroughs and Limitations of Artificial Vision
Visual cortex prostheses are the cutting-edge route in the present area of visible restore. Its core precept is that the visible impairments of many blind individuals are primarily as a consequence of harm to the eyes or optic nerves, however the areas in their brains that course of imaginative and prescient are sometimes intact. Visual cortex prostheses can bypass the broken eyes and optic nerves and immediately transmit visible info to the mind, thereby serving to blind individuals restore some visible capabilities.
Neuralink, based by Elon Musk, has centered on this route. Their visible restoration challenge known as “Blindsight” has not solely acquired the Breakthrough Device Designation from the US FDA but in addition plans to enter medical trials round 2026.
It must be famous that though the precept is obvious, the R & D of visible cortex prostheses nonetheless wants to beat a collection of giant technological obstacles. Among many challenges, find out how to generate higher-quality photographs is arguably the most core downside.
As Philip Troyk, a professor of biomedical engineering at the Illinois Institute of Technology, emphasised, “The current work in this area (R & D of visual cortex prostheses) is not aimed at restoring biological vision but exploring the possibility of artificial vision.” In different phrases, the synthetic imaginative and prescient that the present expertise can present to customers continues to be very restricted.
Liu Bing advised the Science and Technology Innovation Board Daily reporter that the results of conventional visible restore applied sciences, whether or not retinal prostheses or early cortical stimulation visible prostheses, principally keep at permitting customers to understand remoted ‘mild spots’, that are scientifically referred to as phosphenes. This is like with the ability to mild up only some pixels on a display screen however unable to kind a significant picture.
Currently, scientists are nonetheless looking for options. Research instructions embody growing the density of electrodes to enhance spatial decision, optimizing the implantation place of electrodes, and innovating stimulation methods. For instance, some imagine that extra electrodes usually are not essentially higher. The key lies in the implantation place. If the electrodes are distributed in a number of areas of the visible cortex, it could stimulate extra mild spots in a bigger area of view.
In addition, innovation in stimulation patterns can be necessary. Yu Xinguang gave an instance, saying, “Existing research has shown that if the electrodes are stimulated in a specific order according to the writing stroke sequence, the human brain can better recognize the shape of letters.”
This time, Mingshi Brain-Machine Interface efficiently accomplished the practical interactive verification of visible reconstruction for “complex graphics + multiple colors”. According to the Science and Technology Innovation Board Daily reporter, its core lies in adopting a closed-loop adaptive path of ‘brain-machine twin studying’ as an alternative of the conventional open-loop, mounted stimulation path.
The conventional path is like “one-way indoctrination”. A set of stimulation patterns is preset first, and the mind is made to adapt to the machine. However, the mind itself is dynamically altering, and glued algorithms usually result in a fast decline in effectiveness.
“In contrast, our path is a ‘two-way dialogue’,” Liu Bing defined to the Science and Technology Innovation Board Daily reporter. “The system can not only precisely stimulate and encode the visual cortex through a high-density electrode array but also read the neural feedback from the brain in real-time. The stimulation strategy and decoding model can be dynamically optimized, thereby achieving common adaptation and learning with the user’s brain. This fundamentally solves the industry problem of long-term stability.”
Regarding the precept of attaining coloration notion, he additional stated, “We simulate the neural activity patterns triggered by light of different wavelengths in specific functional areas of the visual cortex through specific electrical stimulation sequences. When the brain receives such ‘simulated’ signals, it will interpret the corresponding colors.”
“Currently, we have stably achieved the distinction and perception of basic colors such as red, green, and blue, proving the feasibility of encoding and transmitting color information through cortical electrical stimulation. This is the first step from the black-and-white world to the colorful world. Of course, there is still a long way to go to restore all the colors and delicate tones in nature, but the door has been opened,” Liu Bing stated.
He in contrast the present progress to “creating the first ‘display prototype’ that can show graphics and colors” and stated that the Mingshi Brain-Machine Interface group will considerably enhance the ‘show decision’ by growing the electrode density, optimizing the encoding algorithm, and integrating key applied sciences comparable to pc imaginative and prescient.
This article is from the WeChat official account “Science and Technology Innovation Board Daily”. Author: Xu Hong. Republished with permission from 36Kr.