Stanford Unsupervised Feature Learning and Deep Learning Tutorial

Tutorial Website: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

Sparse Autoencoder

Sparse Autoencoder vectorized implementation, learning/visualizing features on MNIST data

• load_MNIST.py: Load MNIST images
• sample_images.py: Load sample images for testing sparse auto-encoder
• gradient.py: Functions to compute & check the cost and gradient
• display_network.py: Display visualized features
• sparse_autoencoder.py: Sparse autoencoder cost & gradient functions
• train.py: Train sparse autoencoder with MNIST data and visualize learned featured

Preprocessing: PCA & Whitening

Implement PCA, PCA whitening & ZCA whitening

• pca_gen.py

Softmax Regression

Classify MNIST digits via softmax regression (multivariate logistic regression)

• softmax.py: Softmax regression cost & gradient functions
• softmax_exercise: Classify MNIST digits

Self-Taught Learning and Unsupervised Feature Learning

Classify MNIST digits via self-taught learning paradigm, i.e. learn features via sparse autoencoder using digits 5-9 as unlabelled examples and train softmax regression on digits 0-4 as labeled examples

• stl_exercise.py: Classify MNIST digits via self-taught learning

Building Deep Networks for Classification (Stacked Sparse Autoencoder)

Stacked sparse autoencoder for MNIST digit classification

• stacked_autoencoder.py: Stacked autoencoder cost & gradient functions
• stacked_ae_exercise.py: Classify MNIST digits

Linear Decoders with Autoencoders

Learn features on 8×8 patches of 96×96 STL-10 color images via linear decoder (sparse autoencoder with linear activation function in the output layer)

• linear_decoder_exercise.py

Working with Large Images (Convolutional Neural Networks)

Classify 64×64 STL-10 images using features learned via linear decoder (previous section) and convolutional neural networks

• cnn.py: Convolution neural networks. Convolve & Pooling functions
• cnn_exercise.py: Classify STL-10 images