- Title: Scientists 'see' through the eyes of a mouse by decoding brain signals
- Date: 3rd May 2023
- Summary: GENEVA, SWITZERLAND (MAY 2, 2023) (Reuters) (SOUNDBITE) (English) MACKENZIE MATHIS, NEUROSCIENTIST AND ASSISTANT PROFESSOR AT EPFL, SAYING: "We took data that was recorded from the brains of mice that was recorded at the Allen Institute in Seattle, Washington. And what they had done, the experimentalists, is show mice a very classic film. And in this film, you can see a person sort of running down the hallway, getting to a car, and the mouse is just passively watching this, much like you or I might be in a movie theatre. And then what we wanted to do is at the same time they're recording from this mouse brain. And so we asked the question: could we actually reconstruct what the animal was watching just purely from the neural data? So we used our new algorithm CEBRA to build this latent representation of the embedding space. And then you can take this embedding space and essentially use that as the basis for a neural decoding algorithm, much like you do in brain machine interfaces, and then predict exactly the sequence of frames the mouse was watching, even though we didn't know, the algorithm didn't know, of course, what the mouse was watching in that particular moment."
- Embargoed: 17th May 2023 15:42
- Keywords: EPFL MACKENZIE MATHIS Nature Swiss Federal Institute of Technology brain machine interface decoding mouse brains neuroscience
- Location: Geneva, Switzerland / Unidentified Location
- City: Geneva, Switzerland / Unidentified Location
- Country: Switzerland
- Topics: Europe,Science
- Reuters ID: LVA004931202052023RP1
- Aspect Ratio: 16:9
- Story Text: Scientists from the Swiss Federal Institute of Technology (EPFL) have developed a new machine learning algorithm which has the potential to reveal the hidden structure in data recorded from the brain, and predict complex information, such as what mice see.
Called CEBRA, the algorithm builds artificial neural network models that capture brain dynamics with a high degree of accuracy, according to a study published on Wednesday (May 3) in the journal Nature.
The researchers have demonstrated the algorithm's capabilities by reconstructing a black and white 1960s movie clip of a man running to a car and opening the trunk, as seen by a mouse. The algorithm can decode what a mouse sees while watching the movie after being trained to map brain signals and movie features.
"We asked the question: could we actually reconstruct what the animal was watching just purely from the neural data?" explained EPFL neuroscientist Mackenzie Mathis.
"So we used our new algorithm CEBRA to build this latent representation of the embedding space. And then you can take this embedding space and essentially use that as the basis for a neural decoding algorithm, much like you do in brain machine interfaces, and then predict exactly the sequence of frames the mouse was watching."
The researchers used open-access data from the Allen Institute in Seattle, WA, for the video decoding. Brain signals were obtained either directly by measuring brain activity via electrode probes inserted into the visual cortex area of the mouse’s brain, or using optical probes which consist of using genetically modified mice, engineered so that activated neurons glow green.
The CEBRA-constructed movie almost matches the original completely, with some slight distortions.
"With this algorithm, we could do this with over 95% accuracy on these movies. So we think this is sort of a first demonstration that it's actually possible to do this brain machine interface style decoding," said Mathis.
Although it is not yet possible to fully reconstruct what a human sees based on brain signals alone, CEBRA is a step in that direction.
The researchers say the goal of CEBRA is to uncover structure in complex systems, and it could have clinical applications beyond neuroscience.
"If we can use these more powerful tools in the clinic, it could be used for things like visual neuroprosthetics, potentially restoring vision or doing arm movements. So those patients that are paralysed or want to restore or even enhancement in this way, added Mathis.
(Production: Matt Stock) - Copyright Holder: REUTERS
- Copyright Notice: (c) Copyright Thomson Reuters 2023. Open For Restrictions - http://about.reuters.com/fulllegal.asp
- Usage Terms/Restrictions: None