Science and technology
21 Jun 2021A robot on EBRAINS has learned to combine vision and touch
Brain models are helping to discover better robot navigation methods.
On the new EBRAINS research infrastructure, scientists of the Human Brain Project have connected brain-inspired deep learning to biomimetic robots.
How the brain lets us perceive and navigate the world is one of the most fascinating aspects of cognition. When orienting ourselves, we constantly combine information from all six senses in a seemingly effortless way – a feature that even the most advanced AI systems struggle to replicate.
On the new EBRAINS research infrastructure, cognitive neuroscientists, computational modellers and roboticists are now working together to shed new light on the neural mechanisms behind this, by creating robots whose internal workings mimic the brain.
The Whiskeye-Robot (right) and his virtual counterpart on the EBRAINS Neurorobotics Platform
“We believe that robots can be improved through the use of knowledge about the brain. But at the same time, this can also help us better understand the brain”
Professor of Cognition and Systems Neurosciences at the University of Amsterdam
In the Human Brain Project, Pennartz has collaborated with computational modellers Shirin Dora, Sander Bohte and Jorge F. Mejias to create complex neural network architectures for perception based on real-life data from rats. Their model, dubbed “MultiPrednet” consists of modules for visual and tactile input, and a third that merges them.
“What we were able to replicate for the first time, is that the brain makes predictions across different senses. So you can predict how something will feel from looking at it, and vice versa”.
Professor of Cognition and Systems Neurosciences at the University of Amsterdam
The way these networks “train” resembles how scientists think our brain learns: By constantly generating predictions about the world, comparing them to actual sensory inputs, and then adapting the network to avoid future error signals.
To test how the MultiPrednet performs in a body, the researchers teamed up with Martin Pearson at Bristol Robotics Lab. Together they integrated it into Whiskeye, a rodent-like robot that autonomously explores its environment, using head-mounted cameras for eyes, and 24 artificial whiskers to gather tactile information.
The researchers observed first indications that the brain-based model has an edge over traditional deep learning systems: Especially when it comes to navigation and recognition of familiar scenes, the MultiPredNet seems to perform better – a discovery the team now investigates further.
To accelerate this research, the robot has been recreated as a simulation on the Neurorobotics Platform of the EBRAINS research infrastructure.
“This allows us to do long duration or even parallel experiments under controlled conditions. We also plan to use the High Performance and Neuromorphic Computing Platforms for much more detailed models of control and perception in the future.”
Senior Research Fellow at Bristol Robotics Lab
To accelerate this research, the robot has been recreated as a simulation on the Neurorobotics Platform of the EBRAINS research infrastructure.
"It’s a unique situation! We were able to say, here’s an interesting model of perception based on neurobiology, and it would be great to test it on a larger scale with supercomputers and embodied in a robot. Doing this is normally very complicated, but EBRAINS makes it possible.“
Professor of Cognition and Systems Neurosciences at the University of Amsterdam
“To understand cognition, we will need to explore how the brain acts as part of the body in an environment. Cognitive neuroscience and robotics have much to gain from each other in this respect. The Human Brain Project brought these communities together, and now with our standing infrastructure, it’s easier than ever to collaborate.”
Scientific Research Director of the HBP
“The robots of the future will benefit from innovations that connect insights from brain science to AI and robotics. With EBRAINS, Europe can be at the center of this shift to more bio-inspired AI and technology.”
CEO of EBRAINS and Director General of the HBP
Perception and Recognition of Objects and Scenes
Media Contact:
Peter Zekert
Tel.: +49 2461 61 96860
press@humanbrainproject.eu
Further information:
https://ebrains.eu
https://www.humanbrainproject.eu
ABOUT THE RESEARCHERS
Cyriel Pennartz is full professor at the Department of Cognition and Systems Neurosciences of the University of Amsterdam. He is one of the leaders of the Human Brain Project and member of its Scientific and Infrastructure Board. In this function he has been working to integrate empirical neuroscience, computational modelling of brain networks and theories on perception, memory and consciousness. Chief characteristics of his work are its multidisciplinarity and integrative approach to neuroscience, combined with a theory- and model-driven basis for in-depth experimentation and with technologically innovative capacities.
Martin Pearson is a Senior Research Fellow and theme lead for biomimetic and neurorobotics research at the Bristol Robotics Laboratory. He has a B.Eng. degree in Electrical and Electronic Engineering from Manchester University with an MSc. and PhD. in Robotics from the University of the West of England, Bristol. He has authored/co-authored 19 journal articles and over 50 peer reviewed conference papers in the fields of robotics, animal behaviour and embedded neuromorphic hardware.
ABOUT THE HUMAN BRAIN PROJECT
The Human Brain Project (HBP) is the largest brain science project in Europe and stands among the biggest research projects ever funded by the European Union. At the interface of neuroscience and information technology, the HBP investigates the brain and its diseases with the help of highly advanced methods from computing, neuroinformatics and artificial intelligence, and drives innovation in fields like brain-inspired computing and neurorobotics.
ABOUT EBRAINS
EBRAINS is a new digital research infrastructure, created by the EU-funded Human Brain Project, to foster brain-related research and to help translate the latest scientific discoveries into innovation in medicine and industry, for the benefit of patients and society.
It draws on cutting-edge neuroscience and offers an extensive range of brain data sets, a multilevel brain atlas, modelling and simulation tools, easy access to high-performance computing resources and to robotics and neuromorphic platforms.
All academic researchers have open access to EBRAINS’ state-of-the art services. Industry researchers are also very welcome to use the platform under specific agreements. For more information about EBRAINS, please contact us at info@ebrains.eu or visit www.ebrains.eu.
