The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field’s Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.
Archive of code and parameter files
A zip file containing the code used to run the simulations, as well as the specific parameter files used to generate the data shown in the paper.
Archive.zip
MNIST main params network over time
The data file where network state is recorded throughout the simulation training period, for the MNIST parameters used for the main results in the paper.
MNIST_pp_0p001.hdf5
MNIST main params calculated values
A data file containing calculated values measuring quantities such as reconstruction performance, average neuronal activity, etc for different points in the training sequence. These values were used directly to generate the figures in the manuscript.
MNIST_pp_0p03.pkl.pkl
Natural images main params network over time
The data file where network state is recorded throughout the simulation training period, for the natural image parameters used for the main results in the paper.
Natim.hdf5
Natural images very sparse params network over time
The data file where network state is recorded throughout the simulation training period, for the very sparse natural image parameters used for the Figure 10b in the paper.
pp_0p001.hdf5
Natural image very sparse main params calculated values
A data file containing calculated values measuring quantities such as reconstruction performance, average neuronal activity, etc for different points in the training sequence. These values were used directly to generate Figure 10b in the manuscript.
pp_0p001.pkl
MNIST very sparse params network over time
The data file where network state is recorded throughout the simulation training period, for the very sparse MNIST parameters used for Figure 10a in the paper.
MNIST_pp_0p0010.hdf5
MNIST very sparse params calculated values
A data file containing calculated values measuring quantities such as reconstruction performance, average neuronal activity, etc for different points in the training sequence. These values were used directly to generate Figure 10a in the manuscript.
MNIST_pp_0p001.pkl
MNIST non-sparse params network over time
The data file where network state is recorded throughout the simulation training period, for the MNIST parameters used for the Figure 10c in the paper.
MNIST_pp_0p30.hdf5
MNIST non-sparse params calculated values
A data file containing calculated values measuring quantities such as reconstruction performance, average neuronal activity, etc for different points in the training sequence. These values were used directly to generate Figure 10c in the manuscript.
MNIST_pp_0p3.pkl
Natural images main params calculated values
A data file containing calculated values measuring quantities such as reconstruction performance, average neuronal activity, etc for different points in the training sequence. These values were used directly to generate the figures in the manuscript.
pp_0p02_main_results.pkl