一个Self Taught Learning的简单例子

idea:

Concretely, for each example in the the labeled training dataset xl, we forward propagate the example through a convolutional and a pooling layer to obtain the activation of the hidden units a(2). We now represent this example using a(2) (the “replacement” representation), and use this to as the new feature representation with which to train the softmax classifier.

design:

In this exercise, our goal is to distinguish between the digits from 0 to 4. We will use an “unlabeled” dataset with all 10 digits to learn the filters; we will then use a labeled dataset with the digits 0 to 4 with which to train the softmax classifier.

procedure:

Step 1: Generate the input and test data sets

Step 2: Train RICA

Step 3: Extracting features

Step 4: Training and testing the softmax regression model

Step 5: Classifying on the test set

reference: