假设最小化函数 y = x2 , 选择初始点 x0= 5


1. 学习率为1的时候,x在5和-5之间震荡。

 #学习率为1

 import tensorflow as tf
training_steps = 10
learning_rate = 1
x = tf.Variable(tf.constant(5, dtype=tf.float32),name="x")
y = tf.square(x) train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(y) with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(training_steps):
sess.run(train_op)
x_value = sess.run(x)
print("After %s iteration(s): x%s is %f."%(i+1,i+1,x_value)) #输出结果:
After 1 iteration(s): x1 is -5.000000.
After 2 iteration(s): x2 is 5.000000.
After 3 iteration(s): x3 is -5.000000.
After 4 iteration(s): x4 is 5.000000.
After 5 iteration(s): x5 is -5.000000.
After 6 iteration(s): x6 is 5.000000.
After 7 iteration(s): x7 is -5.000000.
After 8 iteration(s): x8 is 5.000000.
After 9 iteration(s): x9 is -5.000000.
After 10 iteration(s): x10 is 5.000000.

2.学习率为0.001的时候,下降速度过慢,在901轮时才收敛到0.823355.

 #学习率为0.001
training_steps = 1000
learning_rate = 0.001
x = tf.Variable(tf.constant(5,dtype=tf.float32),name="x")
y = tf.square(x) train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(y) with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(training_steps):
sess.run(train_op)
if i % 100 ==0:
x_value = sess.run(x)
print("After %s iteration(s): x%s is %f."%(i+1,i+1,x_value)) #结果为: After 1 iteration(s): x1 is 4.990000.
After 101 iteration(s): x101 is 4.084646.
After 201 iteration(s): x201 is 3.343555.
After 301 iteration(s): x301 is 2.736923.
After 401 iteration(s): x401 is 2.240355.
After 501 iteration(s): x501 is 1.833880.
After 601 iteration(s): x601 is 1.501153.
After 701 iteration(s): x701 is 1.228794.
After 801 iteration(s): x801 is 1.005850.
After 901 iteration(s): x901 is 0.823355.

3.使用指数衰减的学习率,在迭代初期得到较高的下降速度,可以在较小的训练轮数下取得不错的收敛程度。

 TRAINING_STEPS = 100
global_step = tf.Variable(0)
LEARNING_RATE = tf.train.exponential_decay(0.1, global_step, 1, 0.96, staircase=True) x = tf.Variable(tf.constant(5, dtype=tf.float32), name="x")
y = tf.square(x)
train_op = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(y, global_step=global_step) with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(TRAINING_STEPS):
sess.run(train_op)
if i % 10 == 0:
LEARNING_RATE_value = sess.run(LEARNING_RATE)
x_value = sess.run(x)
print ("After %s iteration(s): x%s is %f, learning rate is %f."% (i+1, i+1, x_value, LEARNING_RATE_value)) #输出结果: After 1 iteration(s): x1 is 4.000000, learning rate is 0.096000.
After 11 iteration(s): x11 is 0.690561, learning rate is 0.063824.
After 21 iteration(s): x21 is 0.222583, learning rate is 0.042432.
After 31 iteration(s): x31 is 0.106405, learning rate is 0.028210.
After 41 iteration(s): x41 is 0.065548, learning rate is 0.018755.
After 51 iteration(s): x51 is 0.047625, learning rate is 0.012469.
After 61 iteration(s): x61 is 0.038558, learning rate is 0.008290.
After 71 iteration(s): x71 is 0.033523, learning rate is 0.005511.
After 81 iteration(s): x81 is 0.030553, learning rate is 0.003664.
After 91 iteration(s): x91 is 0.028727, learning rate is 0.002436.

Python ---------- Tensorflow (二)学习率的更多相关文章

  1. 基于Ubuntu+Python+Tensorflow+Jupyter notebook搭建深度学习环境

    基于Ubuntu+Python+Tensorflow+Jupyter notebook搭建深度学习环境 前言一.环境准备环境介绍软件下载VMware下安装UbuntuUbuntu下Anaconda的安 ...

  2. Python 基础 二

    Python 基础 二 今天对昨天学习的Python基础知识进行总结,学而不思则惘,思而不学则殆! 一.先对昨天学习的三大循环的使用情况进行总结: 1.while循环的本质就是让计算机在满足某一条件的 ...

  3. 初学Python(二)——数组

    初学Python(二)——数组 初学Python,主要整理一些学习到的知识点,这次是数组. # -*- coding:utf-8 -*- list = [2.0,3.0,4.0] #计算list长度 ...

  4. Python学习二:词典基础详解

    作者:NiceCui 本文谢绝转载,如需转载需征得作者本人同意,谢谢. 本文链接:http://www.cnblogs.com/NiceCui/p/7862377.html 邮箱:moyi@moyib ...

  5. 有关python下二维码识别用法及识别率对比分析

    最近项目中用到二维码图片识别,在python下二维码识别,目前主要有三个模块:zbar .zbarlight.zxing. 1.三个模块的用法: #-*-coding=utf-8-*- import ...

  6. PYTHON练习题 二. 使用random中的randint函数随机生成一个1~100之间的预设整数让用户键盘输入所猜的数。

    Python 练习 标签: Python Python练习题 Python知识点 二. 使用random中的randint函数随机生成一个1~100之间的预设整数让用户键盘输入所猜的数,如果大于预设的 ...

  7. 从Scratch到Python——Python生成二维码

    # Python利用pyqrcode模块生成二维码 import pyqrcode import sys number = pyqrcode.create('从Scratch到Python--Pyth ...

  8. Python Tensorflow CNN 识别验证码

    Python+Tensorflow的CNN技术快速识别验证码 文章来源于: https://www.jianshu.com/p/26ff7b9075a1 验证码处理的流程是:验证码分析和处理—— te ...

  9. Tensorflow实现学习率衰减

    Tensorflow实现学习率衰减 觉得有用的话,欢迎一起讨论相互学习~Follow Me 参考文献 Deeplearning AI Andrew Ng Tensorflow1.2 API 学习率衰减 ...

随机推荐

  1. 干货,比较全面的c#.net公共帮助类

    比较全面的c#帮助类 比较全面的c#帮助类,日常工作收集,包括前面几家公司用到的,各式各样的几乎都能找到,所有功能性代码都是独立的类,类与类之间没有联系,可以单独引用至项目,分享出来,方便大家,几乎都 ...

  2. localStorage存值取值以及存取JSON,以及基于html5 localStorage的购物车

    localStorage.setItem("key","value");//存储变量名为key,值为value的变量 localStorage.key = &q ...

  3. asp.net中kindeditor配置

    <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>KindEditor< ...

  4. php中各种定义变量的方法

      1.定义常量define("CONSTANT", "Hello world."); 常量只能包含标量数据(boolean,integer,float 和 s ...

  5. win10 UWP 获取系统信息

    获取系统信息 Windows.System.Profile.AnalyticsVersionInfo analyticsVersion = Windows.System.Profile.Analyti ...

  6. 转:C++输入一行字符串的一点小结

    原文链接: http://www.wutianqi.com/?p=1181 大家在学习C++编程时,一般在输入方面都是使用的cin.而cin是使用空白(空格,制表符和换行符)来定字符串的界的.这就导致 ...

  7. UVa12325, Zombie's Treasure Chest

    反正书上讲的把我搞得晕头转向的,本来就困,越敲越晕...... 转网上一个大神写的吧,他分析的很好(个人感觉比书上的清楚多了) 转:http://blog.csdn.net/u010536683/ar ...

  8. Python Nose框架编写测试用例方法

    1. 关于Nose nose项目是于2005年发布的,也就是 py.test改名后的一年.它是由 Jason Pellerin 编写的,支持与 py.test 相同的测试习惯做法,但是这个包更容易安装 ...

  9. C#基本功之泛型

    一.没有泛型之前 在没有泛型之前,我们是怎么处理不同类型的相同操作的: 示例1 //下面是一个处理string类型的集合类型 public class MyStringList { string[] ...

  10. bootstrap 鼠标悬停显示

    1. <button type="button" rel="drevil" data-content="报名截止时间:'+time+'" ...