Fast Learning

Approximate, Computationally Efficient Online Learning in Bayesian Spiking

Bayesian Spiking Neurons (BSNs) provide a probabilistic interpretation of how neurons perform inference and learning. Online learning in BSNs typically involves maximum-likelihood expectation-maximisation (ML-EM) based parameter estimation, which is computationally slow and limits the potential of studying networks of BSNs. An online learning algorithm, Fast Learning (FL), is presented that is more computationally efficient than the benchmark ML-EM for a fixed number of time steps as the number of inputs to a BSN increases (e.g. 16.5 times faster run times for 20 inputs). Although ML-EM appears to converge 2-3.6 times faster than FL, the computational cost of ML-EM means that MLEM takes longer to simulate to convergence than FL. FL also provides reasonable convergence performance that is robust to initialization of parameter estimates that are far from the true parameter values. However, parameter estimation depends on the range of true parameter values. Nevertheless, for a physiologically meaningful range of parameter values, FL gives very good average estimation accuracy, despite its approximate nature. The FL algorithm, therefore, provides an efficient tool, complementary to ML-EM, for exploring BSN networks in more detail in order to better understand their biological relevance. Moreover, the simplicity of the FL algorithm means it can be easily implemented in neuromorphic VLSI such that one can take advantage of the energy efficient spike coding of BSNs.