Coursera, Deep Learning 1, Neural Networks and Deep Learning - week3, Neural Networks Basics
NN representation
这一课主要是讲3层神经网络
下面是常见的 activation 函数.sigmoid, tanh, ReLU, leaky ReLU.
Sigmoid 只用在输出0/1 时候的output layer， 其他情况基本不用，因为tanh 总是比sigmoid 好.
两种 ReLU 使用起来总是要比sigmoid 和 tanh 快。ReLU 是最常用的 activation.
为什么Activation function 要是non-linear的？因为如下图所示如果activation 是linear的，那么最终output 只是 input 的线性函数.
Gradient of activation function
Gredient of 2 layer NN.
Random initialization
Coursera, Deep Learning 1, Neural Networks and Deep Learning - week3, Neural Networks Basics的更多相关文章
- 【DeepLearning学习笔记】Coursera课程《Neural Networks and Deep Learning》——Week2 Neural Networks Basics课堂笔记
Coursera课程<Neural Networks and Deep Learning> deeplearning.ai Week2 Neural Networks Basics 2.1 ...
- 【DeepLearning学习笔记】Coursera课程《Neural Networks and Deep Learning》——Week1 Introduction to deep learning课堂笔记
Coursera课程<Neural Networks and Deep Learning> deeplearning.ai Week1 Introduction to deep learn ...
- 课程一(Neural Networks and Deep Learning)，第四周（Deep Neural Networks） —— 3.Programming Assignments: Deep Neural Network - Application
Deep Neural Network - Application Congratulations! Welcome to the fourth programming exercise of the ...
- 课程一(Neural Networks and Deep Learning)，第二周（Basics of Neural Network programming）—— 4、Logistic Regression with a Neural Network mindset
Logistic Regression with a Neural Network mindset Welcome to the first (required) programming exerci ...
- Neural Networks and Deep Learning
Neural Networks and Deep Learning This is the first course of the deep learning specialization at Co ...
- [C3] Andrew Ng - Neural Networks and Deep Learning
About this Course If you want to break into cutting-edge AI, this course will help you do so. Deep l ...
- 《Neural Networks and Deep Learning》课程笔记
Lesson 1 Neural Network and Deep Learning 这篇文章其实是 Coursera 上吴恩达老师的深度学习专业课程的第一门课程的课程笔记. 参考了其他人的笔记继续归纳 ...
- 第四节，Neural Networks and Deep Learning 一书小节(上)
最近花了半个多月把Mchiael Nielsen所写的Neural Networks and Deep Learning这本书看了一遍,受益匪浅. 该书英文原版地址地址:http://neuralne ...
- 课程四(Convolutional Neural Networks)，第二 周（Deep convolutional models: case studies） —— 0.Learning Goals
Learning Goals Understand multiple foundational papers of convolutional neural networks Analyze the ...
- 课程一(Neural Networks and Deep Learning)，第三周（Shallow neural networks）—— 3.Programming Assignment : Planar data classification with a hidden layer
Planar data classification with a hidden layer Welcome to the second programming exercise of the dee ...
随机推荐
- 洛谷P3620 数据备份
好吧,我一开始说这是个神级数据结构毒瘤题,后来改成神题了. 主要是贪心做法的巧妙转化: 首先发现选择的一对必须相邻,于是我们搞出差分. 然后考虑选取最小值时,最小值两侧的数要么同时选,要么都不选. 然 ...
- ImageMagick: 6.8.3 升级到 6.8.9 遇到的问题
最终还是决定升级到目前最新版:6.8.9,不知何时才真正明白为什么现在都是java8,但还是有很多软件系统使用在java5上. 虽然新版本能带来各种好处,但现实中不能忽略一个问题:原来的代码很可能无法 ...
- 第四篇：记录相关操作 SQL逻辑查询语句执行顺序
http://www.cnblogs.com/linhaifeng/articles/7372774.html 一 SELECT语句关键字的定义顺序 SELECT DISTINCT <selec ...
- windows环境下 安装python2和python3
一. python 安装 1. 下载安装包 https://www.python.org/ftp/python/2.7.14/python-2.7.14.amd64.msi # 2.7安装包 htt ...
- Codeforce 867 C. Ordering Pizza (思维题）
C. Ordering Pizza It's another Start[c]up finals, and that means there is pizza to order for the ons ...
- String 中常用的几种方法
/* String(char[] value)传递字符数组 将字符数组转换为字符串 字符数组不查询编码表 */ public static void fun1(){ char[] ch = {'a', ...
- 线程（Thread）
package cn.gouzao.demo3; public class ThreadDemo extends Thread{ public void run(){ for(int i=0;i< ...
- qml： 打包 和 发布
Qt 提供了打包工具windeployqt, 利用该工具可以很方便的解决qt的依赖问题(注:通过实际验证,发现该工具只能解决大部分的依赖问题,不知道是不是本人 没有正确的使用的问题)． qt源码编译r ...
- Mysql连接查询、子查询、联合查询 整理
连接查询 连接语法的基本形式 from 表1 [连接方式] join 表2 [on 连接条件]: 交叉连接 语法: from 表1 [cross] join 表2 ; //可 ...
- saltstack syndic
#syndic 相当于master的代理,master通过syndic代理控制node主机 master <------ syndic+master <---------- node ma ...