tensorflow中使用mnist数据集训练全连接神经网络-学习笔记

tensorflow中使用mnist数据集训练全连接神经网络

——学习曹健老师“人工智能实践：tensorflow笔记”的学习笔记， 感谢曹老师

前期准备：mnist数据集下载，并存入data目录：

文件列表：四个文件，分别为训练和测试集数据

Four files are available on 官网  http://yann.lecun.com/exdb/mnist/ :

train-images-idx3-ubyte.gz:  training set images (9912422 bytes)
train-labels-idx1-ubyte.gz: 
training set labels (28881 bytes)

t10k-images-idx3-ubyte.gz:  
test set images (1648877 bytes)

t10k-labels-idx1-ubyte.gz:  
test set labels (4542 bytes)

一、主要思路：

1、训练集输入数据X为28×28图像，和 Y_  onehot label

2、构建一个三层NN，input layer，one hidden layer，outputlayer

3、使用指数衰减学习率，交叉熵loss，移动平均loss构建NN

二、主要代码：

forward构建：

//mnist_forward.py

import tensorflow as tf
INPUT_NODE = 784
OUTPUT_NODE = 10
LAYER1_NODE = 500

def get_weight(shape, regularizer):
    w = tf.Variable(tf.truncated_normal(shape, stddev=0.1))
    if regularizer != None:
        tf.add_to_collection('losses', tf.contrib.layers.l2_regularizer(regularizer)(w))
    return w

def get_bias(shape):
    b = tf.Variable(tf.zeros(shape))
    return b

def forward(x,regularizer):
    w1 = get_weight([INPUT_NODE, LAYER1_NODE], regularizer)
    b1 = get_bias([LAYER1_NODE])
    y1 = tf.nn.relu(tf.matmul(x, w1) + b1)

w2 = get_weight([LAYER1_NODE, OUTPUT_NODE], regularizer)
    b2 = get_bias([OUTPUT_NODE])
    y = tf.matmul(y1, w2) + b2
    return y

backward构建：

//mnist_backward.py

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import mnist_forward
import os

BATCH_SIZE = 200
LEARNING_RATE_BASE = 0.1
LEARNING_RATE_DECAY = 0.99
REGULARIZER = 0.0001
STEPS = 50000
MOVING_AVERAGE_DECAY = 0.99
MODEL_SAVE_PATH = "./model/"
MODEL_NAME = "mnist_model"

def backward(mnist):
    x = tf.placeholder(tf.float32, [None, mnist_forward.INPUT_NODE])
    y_ = tf.placeholder(tf.float32, [None, mnist_forward.OUTPUT_NODE])
    y = mnist_forward.forward(x, REGULARIZER)
    global_step = tf.Variable(0, trainable=False)

ce = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=y, labels=tf.argmax(y_,1))
    cem = tf.reduce_mean(ce)
    loss = cem + tf.add_n(tf.get_collection('losses'))

learning_rate = tf.train.exponential_decay(
            LEARNING_RATE_BASE,
            global_step,
            mnist.train.num_examples / BATCH_SIZE,
            LEARNING_RATE_DECAY,
            staircase = True
            )

train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss, global_step = global_step)
    ema = tf.train.ExponentialMovingAverage(MOVING_AVERAGE_DECAY, global_step)
    ema_op = ema.apply(tf.trainable_variables())
    with tf.control_dependencies([train_step, ema_op]):
        train_op = tf.no_op(name='train')

saver = tf.train.Saver()
    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
　　 sess.run(init_op)

for i in range(STEPS):
            xs, ys = mnist.train.next_batch(BATCH_SIZE)
            _, loss_value, step = sess.run([train_op, loss, global_step], feed_dict={x: xs, y_: ys})
            if i % 1000 ==0:
                print("After %d training step(s), loss on training batch is %g." % (step, loss_value))
                saver.save(sess, os.path.join(MODEL_SAVE_PATH, MODEL_NAME), global_step = global_step)

def main():
    mnist = input_data.read_data_sets("./data/", one_hot = True)
    backward(mnist)

if __name__ == '__main__':
    main()