Spiking neural networks reach a new level

Three studies recently published within just one month by the journal Cerebral Cortex present new models of spiking networks in the brain. Electrical spikes of neurons are at the core of neural information processing. With millions of neurons and billions of synapses, the new generation of open models are powerful tools to study the complex dynamics in large networks, with implications for basic neuroscience, neuromorphic computing and AI. On EBRAINS, researchers will be able to use the large-scale neuronal network models as adaptable building blocks in their investigations and combine them with other brain simulation tools across scales.

Spikes are electric impulses that a neuron can fire out to all its connected cells. These signals are key to the relay of information in the brain. The way they originate at the neurons and can be recorded with different methods, but practical limitations prevent us from being able to experimentally trace the spiking in more than a few hundred neurons simultaneously on the scale of larger networks.

New computational models with enhanced network size and complexity now give a window into phenomena that occur in interactions across large spiking networks. The mathematical models are based on abstract networks with simplified point neurons and investigated with the NEST simulator on EBRAINS. The new models expand from a previous highly detailed microcircuit model representing neuronal network of below a square millimeter of cortical surface (Potjans & Diesmann 2014).

The three new models reflect further cell types, larger cortical volumes and the incorporation of long-range connectivity. The full-scale models of the neuronal network in cortical tissue replicate the enormous density of connections of neurons and synapses found in nature. Network dynamics therefore can be studied at a large scale, while the behavior of the individual elements is traceable at all points.

To go even deeper, the network simulations can be connected to LFPy, a simulation engine on EBRAINS for the study of electrical fields based on highly detailed neuron models. Similarly, NEST also has an interface for co-simulation with The Virtual Brain on EBRAINS, a whole-brain network simulation engine based on mean-field models. For researchers this enables so-called multiscale investigations - spanning the macroscopic level, the large-scale network level, and the level of morphologically detailed neurons.

The new papers in detail:

Model 1 – the role of different inhibitory neuron types in the network

The first study by Jiang et al. extends the original microcircuit by replacing the generic model of an inhibitory neuron with specific models for the three most prominent inhibitory cell types found in nature.  The resulting network model reproduces characteristic features of these cell types in the circuit.  This paves the way for investigations on the computational role of the different cell types, and their relevance in changes of global network states like the sleep-wake cycle.

-Jiang et al.: A Layered Microcircuit Model of Somatosensory Cortex with Three Interneuron Types and Cell-Type-Specific Short-Term Plasticity

Model 2 – from the microscopic- to the mesoscale

The second study by Senk et al. expands the size of the network to four-by-four millimeters of cortical surface. The volume corresponds to common electrophysiological recording technologies like the so-called Utah array. At this scale, the cortical networks exhibit an interesting spatial organization: the further two neurons are apart, the less likely it is that they are connected. The circuit explains a dynamical phenomenon observed in nature but absent in the original microcircuit model, the propagation of activity waves. The study also shows that the weak pairwise correlation between the activity of individual neurons is compatible with highly coherent local field potentials, a common measure of mesoscopic activity.

-Senk et al.: Reconciliation of weak pairwise spike-train correlations and highly coherent local field potentials across space

Model 3 – networks of networks – local and brain-wide network connectivity in the human brain

The third study by Pronold et al. makes the step to the scale of the human brain. In this model, each cortical area in a hemisphere is represented by one of the original microcircuits adapted to the local anatomy. The model predicts experimental data on all scales from the single-neuron spiking activity to the area-level functional connectivity as obtained by resting-state fMRI. The study furthermore shows that a single-spike perturbation propagates through the network within a time close to the limit imposed by the physical delays. The spiking hemisphere model connects 3,5 million neurons with over 42 billion synapses.

-Pronold et al.: Multi-Scale Spiking Network Model of Human Cerebral Cortex

The developments for the three new models were led by researchers Sacha van Albada and Johanna Senk at Forschungszentrum Jülich, with collaborators in Germany, Norway, Portugal, the Netherlands and the UK. Prof. Dr. Sacha van Albada, a department leader at the Institute for Advanced Simulation, Computational and Systems Neuroscience (IAS-6) and affiliated to University of Cologne, remarks:

“In future work we want to bring the three aspects represented in these models together in a unified model: diversity of cell types, spatial organization, and brain-scale circuits.” The network models have already elucidated different theoretical neuroscience questions, the researcher says. “But of course, there are many more research questions than can be answered by a single laboratory. This is why we will provide the full open-source models on EBRAINS so that every researcher can use them for further studies.”

Dr. Johanna Senk, team leader at IAS-6 and lecturer at University of Sussex, emphasizes that “Large neuronal network models are required to integrate the wealth of experimental brain data across modalities and scales like electron microscopy, electrophysiology, morphological reconstructions, cyto-architecture, receptor mapping, DTI, and fMRI into a coherent framework.”

With respect to the required technology and infrastructure, she adds: “The digital workflows and simulation software that are provided by EBRAINS are crucial, as well as keeping up with supercomputer development. We are now especially looking forward to using the capabilities of the first exascale supercomputer in Europe, JUPITER.”

Spiking networks in computing technology and AI

Besides neuroscientific investigations, the use of spiking networks has emerged as a new path for artificial intelligence. “For us as theorists, it has been especially interesting why spiking networks in the brain are so energy-efficient when they process information”, says Markus Diesmann, director of IAS-6. Learning from the sparse way that the spikes encode information could help researchers to overcome the massive energy-demand of most commonly used AI training methods.

This has also piqued the interest of computer engineers. In “neuromorphic” computing, the principle of sparsely connected spiking elements is transferred into hardware. For the past few years, the original circuit model has been a de-facto standard benchmark for optimizing neuromorphic machines. “The original circuit model led to a race for ever higher simulation speed and lower energy consumption. Over the past five years, the community brought down the energy consumption by a factor of 100”, Diesmann says.

Outlook: Digital team science

For the new models, the authors are currently working to make not only the papers but also the executable code descriptions available.  “We have seen with our previous open-source models that they become used as building blocks for studies in different contexts and sometimes modified for new questions or applications”, Diesmann remarks. On EBRAINS, this is how the expanded models will be set up as well, with the option to be directly executed in fast downscaled versions, run in full scale with supercomputing time, or be downloaded for use on the researcher’s laptop.

Paper links:

Han-Jia Jiang, Guanxiao Qi, Renato Duarte, Dirk Feldmeyer, Sacha J van Albada, A layered microcircuit model of somatosensory cortex with three interneuron types and cell-type-specific short-term plasticity, Cerebral Cortex, Volume 34, Issue 9, September 2024, bhae378, https://doi.org/10.1093/cercor/bhae378

Johanna Senk, Espen Hagen, Sacha J van Albada, Markus Diesmann, Reconciliation of weak pairwise spike–train correlations and highly coherent local field potentials across space, Cerebral Cortex, Volume 34, Issue 10, October 2024, bhae405, https://academic.oup.com/cercor/article/34/10/bhae405/7841852

Jari Pronold, Alexander van Meegen, Renan O Shimoura, Hannah Vollenbröker, Mario Senden, Claus C Hilgetag, Rembrandt Bakker, Sacha J van Albada, Multi-scale spiking network model of human cerebral cortex, Cerebral Cortex, Volume 34, Issue 10, October 2024, bhae409, https://academic.oup.com/cercor/article/34/10/bhae409/7826014

Tobias C. Potjans Markus Diesmann (2014) The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model. Cerebral Cortex 24(3), 785–806. https://doi.org/10.1093/cercor/bhs358

Contacts

Prof. Dr. Sacha van Albada
Forschungszentrum Jülich, IAS-6 Computational and Systems Neuroscience
Group Leader Theoretical Neuroanatomy

and

Institut für Zoologie, Universität zu Köln
s.van.albada@fz-juelich.de

Dr. Johanna Senk
Forschungszentrum Jülich, IAS-6, Computational and Systems Neuroscience
Team Leader Future Simulation Technologies
Sussex AI, School of Engineering and Informatics
University of Sussex

J.Senk@sussex.ac.uk