Smooth Exact Gradient Descent Learning in Spiking Neural Networks
Neurons and Cognition
2025-01-29 v2 Neural and Evolutionary Computing
Abstract
Gradient descent prevails in artificial neural network training, but seems inept for spiking neural networks as small parameter changes can cause sudden, disruptive (dis-)appearances of spikes. Here, we demonstrate exact gradient descent based on continuously changing spiking dynamics. These are generated by neuron models whose spikes vanish and appear at the end of a trial, where it cannot influence subsequent dynamics. This also enables gradient-based spike addition and removal. We illustrate our scheme with various tasks and setups, including recurrent and deep, initially silent networks.
Cite
@article{arxiv.2309.14523,
title = {Smooth Exact Gradient Descent Learning in Spiking Neural Networks},
author = {Christian Klos and Raoul-Martin Memmesheimer},
journal= {arXiv preprint arXiv:2309.14523},
year = {2025}
}