EBRAINS atlas used in study to understand how the brain detects deepfakes

Do our brains process natural and deepfaked voices differently? In a new study published in Communications Biology, scientists from Switzerland and Australia have shown that this is the case. Neuroimaging experiments identified two brain regions that respond differently to natural and deepfake voices. The results add to our understanding of neural mechanisms behind our ability to recognise deceptive information in digital environments. To precisely characterise the brain network, the researchers used probabilistic maps from the openly accessible Julich Brain Atlas on EBRAINS.

The proliferation of deepfake technology in digital environments poses new challenges to society. But little is known so far about the human ability to reliably recognize audio and visual deepfakes. A new study led by the ETH Zürich has now added new information on the brai-level. By using advanced deepfake technologies, the scientists produced high-quality voice identity clones from natural speakers. In an identity matching task, 25 participants were able to identify deepfakes in two thirds of the cases. During the task, their brain activity was measured.

The analyses revealed a central cortico-striatal network that decoded the vocal acoustic pattern and deepfake-level (auditory cortex), as well as natural speaker identities (nucleus accumbens). This network is embedded in a broader neural identity and object recognition network. The neurocognitive mechanisms identified during deepfake processing open windows for strengthening human resilience to fake information.

As part of their analyses the scientists also used the probabilistic maps of Julich-Brain. This cytoarchitectonic brain atlas currently contains maps of more than 200 regions and is an openly available ressource for the neuroimaging community. The maps are based on the analysis of cell body stained tissue sections, which were mapped with image analysis and statistical tools in 10 postmortem brains for each area. Julich-Brain is part of EBRAINS, Europe’s digital infrastructure for brain research.

Original publication:

Roswandowitz, C., Kathiresan, T., Pellegrino, E. et al. Cortical-striatal brain network distinguishes deepfake from real speaker identity. Commun Biol 7, 711 (2024). https://doi.org/10.1038/s42003-024-06372-6