OPIUM

Title:

Learning the pseudoinverse solution to network weights

Authors:

Jonathan Tapson, André van Schaik

Journal:

Neural Networks

Year:

2013

Abstract:

The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the ''learning method''—computation of the pseudoinverse by singular value decomposition—is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse precisely, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory- efficient than the conventional computation of pseudoinverses by singular value decomposition.

Note:

We have discovered since publication that unfortunately there is an error in notation in the original manuscript. The error has been corrected in the pdf provided below.

Files:

• opium.pdf - The manuscript
• mnist_opium.py - Python code for MNIST digit classification
• mnist.py - Python library to read the MNIST data
• MG_opium.py - Python code for Mackey-Glass time series prediction
• OPIUM.py - Python library for the Greville and OPIUM learning rule

For the MNIST classification experiments, you will also need to download the MNIST databases(opens in a new window) and place them in a sub-directory called 'data' in the directory where you put the Python code.