超简单tensorflow入门优化程序&&tensorboard可视化

程序1

任务描述： x = 3.0, y = 100.0，运算公式 x×W+b = y，求 W和b的最优解。

使用tensorflow编程实现：

#-*- coding: utf-8 -*-)

import tensorflow as tf

# 声明占位变量x、y

x = tf.placeholder("float",shape=[None,1])

y = tf.placeholder("float",[None,1])

# 声明变量

W = tf.Variable(tf.zeros([1,1]))

b = tf.Variable(tf.zeros([1]))

# 操作

result = tf.matmul(x,W) +b

# 损失函数

lost = tf.reduce_sum(tf.pow((result-y),2))

# 优化

train_step = tf.train.GradientDescentOptimizer(0.001).minimize(lost)

with tf.Session() as sess:

    # 初始化变量

    sess.run(tf.global_variables_initializer())

    # 这里x、y给固定的值

    x_s = [[3.0]]

    y_s = [[100.0]]

    step =0

    while(True):

        step += 1

        feed = {x: x_s, y: y_s}

        # 通过sess.run执行优化

        sess.run(train_step, feed_dict=feed)

        if step % 50 ==0:

            print 'step: {0},  loss: {1}'.format(step,sess.run(lost,feed_dict=feed))

            if sess.run(lost,feed_dict=feed) < 0.00001 or step >3000:

                print ''

                print 'final loss is: {}'.format(sess.run(lost,feed_dict=feed))

                print 'final result of {0} =  {1}'.format('x×W+b',3.0*sess.run(W)+sess.run(b))

                print("W : %f" % sess.run(W))

                print("b : %f" % sess.run(b))

                break

输出：

step: 50,  loss: 1326.19543457

step: 100,  loss: 175.879058838

step: 150,  loss: 23.325012207

step: 200,  loss: 3.09336590767

step: 250,  loss: 0.410243988037

step: 300,  loss: 0.0544071868062

step: 350,  loss: 0.00721317622811

step: 400,  loss: 0.000956638017669

step: 450,  loss: 0.000126981700305

step: 500,  loss: 1.68478582054e-05

step: 550,  loss: 2.23610550165e-06

final loss is: 2.23610550165e-06

final result of x×W+b =  [[ 99.99850464]]

W : 29.999552

b : 9.999846

任务很简单，初始学习率设置为0.0001,550论迭代后优化完成，如果初始学习率设置的高一点，如0.005,会加快收敛。求得 W = 29.999552, b = 9.999846

x×W+b = 3.0×29.999552+9.999846 = 99.999846 ，约等于目标 100.0 了。

程序2

任务描述： x、y是二维矩阵， x = [[1.0, 3.0], [3.2, 4.]], y = [[6.0, 3.0], [5.2, 43.]]，运算公式 x×W+b = y，求 W和b的最优值。

# -*- coding: utf-8 -*-)

import tensorflow as tf

# 声明占位变量x、y, 形状为[2,2]

x = tf.placeholder("float",shape=[2,2])

y = tf.placeholder("float",[2,2])

# 声明变量

W = tf.Variable(tf.zeros([2,2]))

b = tf.Variable(tf.zeros([1]))

# 操作

result = tf.matmul(x,W) +b

# 损失函数

lost = tf.reduce_sum(tf.pow((y-result),2))

# 优化

train_step = tf.train.GradientDescentOptimizer(0.001).minimize(lost)

with tf.Session() as sess:

    # 初始化变量

    sess.run(tf.global_variables_initializer())

    # 这里x、y给固定的值

    x_s = [[1.0, 3.0], [3.2, 4.]]

    y_s = [[6.0, 3.0], [5.2, 43.]]

    step = 0

    while(True):

        step += 1

        feed = {x: x_s, y: y_s}

        # 通过sess.run执行优化

        sess.run(train_step, feed_dict=feed)

        if step % 500 == 0:

            print 'step: {0},  loss: {1}'.format(step, sess.run(lost, feed_dict=feed))

            if sess.run(lost, feed_dict=feed) < 0.00001 or step > 10000:

                print ''

                print 'final loss is: {}'.format(sess.run(lost, feed_dict=feed))

                print("W : {}".format(sess.run(W)))

                print("b : {}".format( sess.run(b)))

                result1 = tf.matmul(x_s, W) + b

                print 'final result is: {}'.format(sess.run(result1))

                print 'final error is: {}'.format(sess.run(result1)-y_s)

                break

输出：

step: 500,  loss: 59.3428421021

step: 1000,  loss: 8.97444725037

step: 1500,  loss: 1.40089821815

step: 2000,  loss: 0.22409722209

step: 2500,  loss: 0.036496296525

step: 3000,  loss: 0.00602086028084

step: 3500,  loss: 0.00100283313077

step: 4000,  loss: 0.000168772909092

step: 4500,  loss: 2.86664580926e-05

step: 5000,  loss: 4.90123693453e-06

final loss is: 4.90123693453e-06

W : [[ -2.12640238  20.26368904]

 [  3.87999701  -4.58247852]]

b : [-3.51479006]

final result is: [[  5.99879789   3.00146341]

 [  5.20070982  42.99909973]]

final error is: [[-0.00120211  0.00146341]

 [ 0.00070982 -0.00090027]]

程序3 （增加可视化）

任务描述： X是128个二维数组[X1, X2]， Y是X中X1和X2 相关的函数，Y = x1 +10*x2，
运算公式 Y = (X*w1+b1)*(w2)+b2，求 w1、b1、w2、b2的最优值。

# -*- coding: utf-8 -*-)

import tensorflow as tf

from numpy.random import RandomState

# 定义训练数据batch的大小

batch_size = 8

# 在shape上使用None表示该维度的具体数值不定

x = tf.placeholder(tf.float32, shape=(None, 2), name='x-input')

y_ = tf.placeholder(tf.float32, shape=(None, 1), name='y-input')

# 定义神经网络的参数

w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1))

w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1))

bias1 = tf.Variable(tf.random_normal([3], stddev=1, seed=1))

bias2 = tf.Variable(tf.random_normal([1], stddev=1, seed=1))

# 定义神经网络前向传播的过程，即操作

a = tf.nn.relu(tf.matmul(x, w1) + bias1)

y = tf.nn.relu(tf.matmul(a, w2) + bias2)

# 定义损失函数和反向传播算法

loss = tf.reduce_sum(tf.pow((y-y_),2))

train_step = tf.train.AdamOptimizer(0.001).minimize(loss)  # 梯度下降优化算法

# produce the data，通过随机数生成一个模拟数据集

rdm = RandomState(seed = 1)   # 设置seed = 1 ，使每次生成的随机数一样

dataset_size = 128

X = rdm.rand(dataset_size, 2)

Y = [[x1 +10*x2] for (x1, x2) in X]

# creare a session，创建一个会话来运行TensorFlow程序

with tf.Session() as sess:

    # 定义命名空间，使用tensorboard进行可视化

    with tf.name_scope("inputs"):

        tf.summary.histogram('X', X)

    with tf.name_scope("target"):

        tf.summary.histogram('Target', Y)

    with tf.name_scope("outputs"):

        tf.summary.histogram('Y', y)

    with tf.name_scope('loss'):

        tf.summary.histogram('Loss', loss)

    summary_op = tf.summary.merge_all()

    summary_writer = tf.summary.FileWriter('./log/', tf.get_default_graph())

    # 初始化变量

    sess.run(tf.global_variables_initializer())

    # 设定训练的轮数

    STEPS = 10000

    for i in range(STEPS+1):

        # get batch_size samples data to train，每次选取batch_size个样本进行训练

        start = (i * batch_size) % dataset_size

        end = min(start + batch_size, dataset_size)

        # 通过选取的样本训练神经网络并更新参数

        sess.run(train_step, feed_dict={x: X[start: end], y_: Y[start: end]})

        if i % 500 == 0:

            # 每隔一段时间计算在所有数据上的loss并输出

            total_cross_entropy,summary = sess.run([loss,summary_op], feed_dict={x: X, y_: Y})

            print ("After %d training steps, loss on all data is %g" % (i, total_cross_entropy))

            # 在训练结束之后，输出神经网络的参数

            log_writer = tf.summary.FileWriter('./log/')

            log_writer.add_summary(summary, i)

    print sess.run(w1)

    print sess.run(w2)

输出：

After 0 training steps, loss on all data is 2599.94

After 500 training steps, loss on all data is 873.661

After 1000 training steps, loss on all data is 667.791

After 1500 training steps, loss on all data is 483.075

After 2000 training steps, loss on all data is 300.244

After 2500 training steps, loss on all data is 159.576

After 3000 training steps, loss on all data is 74.0152

After 3500 training steps, loss on all data is 30.0223

After 4000 training steps, loss on all data is 10.8486

After 4500 training steps, loss on all data is 3.86847

After 5000 training steps, loss on all data is 1.67753

After 5500 training steps, loss on all data is 0.870904

After 6000 training steps, loss on all data is 0.473931

After 6500 training steps, loss on all data is 0.262818

After 7000 training steps, loss on all data is 0.132299

After 7500 training steps, loss on all data is 0.0585541

After 8000 training steps, loss on all data is 0.022748

After 8500 training steps, loss on all data is 0.00789603

After 9000 training steps, loss on all data is 0.00259982

After 9500 training steps, loss on all data is 0.000722203

After 10000 training steps, loss on all data is 0.000218332

[[-0.81131822  0.74178803 -0.06654923]

 [-2.4427042   1.72580242  3.50584793]]

[[-0.81131822]

 [ 1.53606057]

 [ 2.09628034]]

tensorboard可视化

程序3运行之后在程序所在目录下生成 log 文件夹，保存了运行过程中的中间参数；

在log文件夹同级目录下，执行tensorboard指令：

tensorboard --logdir=log

在当前系统的浏览器里输入TensorBoard返回的ip地址，这里是
"http://dcrmg:6006"，就可以看到程序中记录下的参数可视化：

X是二维数组，Target的是128个分布在1.0~10.0之间的浮点数。预测值Y在4000轮迭代之后越来越接近真实Target值。

直方图分布：