Newsroom
Noise Is More Than a Nuisance: Spike-Timing Differences Solve the Weight Transport Problem in Neural Networks

An international team of researchers involving STRUCTURES external member Mihai Petrovici and group members at the Kirchhoff Institute for Physics has proposed a new solution to the so-called weight transport problem, a longstanding challenge for models of deep learning in physical neuronal networks. Their solution relies only on biologically plausible, local mechanisms.
Modern AI often relies on the error backpropagation algorithm, which learns by adjusting weights, i.e. strengths of connections between artificial neurons. However, this algorithm makes an assumption that is difficult to reconcile with how real brains work: information travelling forward (sensory representations) and backward (response errors) in the network must take the same path, i.e. pass through the same connections with the same weights. In contrast, the connections between biological neurons are one-way, meaning the forward and backward signals need to travel through separate pathways that would need to have the same connection strengths. However, synapses obey locality: they can only sense what happens at their own location. There is no known mechanism for one synapse to “read off” the exact strength of another, distant one. This is the weight transport problem.
Neuromorphic computer chips, which mimic neurons and synapses directly in hardware for more energy-efficient computations, face the same obstacle.
In a new study published in Nature Communications, an international team of researchers involving STRUCTURES external member Mihai Petrovici and YRC member Andreas Baumbach introduces Spike-based Alignment Learning (SAL) – a framework that solves the weight transport problem without any need to copy connection information across the network. The authors instead propose spike-timing differences as an explanation: as two connected neurons fire, tiny, naturally occurring delays between their firings carry a subtle signal about how mismatched their forward and backward connections are. By tracking these delays, each connection can gradually correct itself using only local information, keeping the forward and backward pathways in sync. What makes this work is the irregular timing of neuron firing. Without this background noise contribution, single synapses might be too weak to make the receiving neuron fire, whereas strong ones would saturate the response. The team tested their model across several brain-inspired network types and found SAL to greatly enhance the performance of spiking networks for different tasks while avoiding non-local information exchange; compared to previously proposed local weight transport mechanisms, SAL used a much simpler learning rule to greater effect, restoring symmetry across a wide range of synaptic weights.
This result was obtained as part of STRUCTURES' Comprehensive Project CP 5: Quantum Systems and Neural Networks.
Further information: