The Backpropagation Algorithm
https://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf
7.1 Learning as gradient descent We saw in the last chapter that multilayered networks are capable of computing a wider range of Boolean functions than networks with a single layer of computing units. However the computational effort needed for finding the correct combination of weights increases substantially when more parameters and more complicated topologies are considered. In this chapter we discuss a popular learning method capable of handling such large learning problems — the backpropagation algorithm. This numerical method was used by different research communities in different contexts, was discovered and rediscovered, until in 1985 it found its way into connectionist AI mainly through the work of the PDP group [382]. It has been one of the most studied and used algorithms for neural networks learning ever since. In this chapter we present a proof of the backpropagation algorithm based on a graphical approach in which the algorithm reduces to a graph labeling problem. This method is not only more general than the usual analytical derivations, which handle only the case of special network topologies, but also much easier to follow. It also shows how the algorithm can be efficiently implemented in computing systems.
The optimization algorithm repeats a two phase cycle, propagation and weight update. When an input vector is presented to the network, it is propagated forward through the network, layer by layer, until it reaches the output layer. The output of the network is then compared to the desired output, using a loss function. The resulting error value is calculated for each of the neurons in the output layer. The error values are then propagated from the output back through the network, until each neuron has an associated error value that reflects its contribution to the original output. Backpropagation uses these error values to calculate the gradient of the loss function. In the second phas
e, this gradient is fed to the optimization method, which in turn uses it to update the weights, in an attempt to minimize the loss function.
The Backpropagation Algorithm的更多相关文章
- CheeseZH: Stanford University: Machine Learning Ex4:Training Neural Network(Backpropagation Algorithm)
1. Feedforward and cost function; 2.Regularized cost function: 3.Sigmoid gradient The gradient for t ...
- BP反向传播算法的工作原理How the backpropagation algorithm works
In the last chapter we saw how neural networks can learn their weights and biases using the gradient ...
- 反向传播算法 Backpropagation Algorithm
假设我们有一个固定样本集,它包含 个样例.我们可以用批量梯度下降法来求解神经网络.具体来讲,对于单个样例(x,y),其代价函数为:这是一个(二分之一的)方差代价函数.给定一个包含 个样例的数据集,我们 ...
- 神经网络(9)--如何求参数: backpropagation algorithm(反向传播算法)
Backpropagation algorithm(反向传播算法) Θij(l) is a real number. Forward propagation 上图是给出一个training examp ...
- Feedforward and BackPropagation Algorithm
在下图所示的Neural Network中,我们将拥有三个节点的layer1及layer4分别称为输入和输出层,而中间的两层layer2,layer3称为隐藏层(hidden layer).输入数据X ...
- 一文弄懂神经网络中的反向传播法(Backpropagation algorithm)
最近在看深度学习的东西,一开始看的吴恩达的UFLDL教程,有中文版就直接看了,后来发现有些地方总是不是很明确,又去看英文版,然后又找了些资料看,才发现,中文版的译者在翻译的时候会对省略的公式推导过程进 ...
- [Converge] Backpropagation Algorithm
Ref: CS231n Winter 2016: Lecture 4: Backpropagation Ref: How to implement a NN:中文翻译版本 Ref: Jacobian矩 ...
- (六) 6.2 Neurons Networks Backpropagation Algorithm
今天得主题是BP算法.大规模的神经网络可以使用batch gradient descent算法求解,也可以使用 stochastic gradient descent 算法,求解的关键问题在于求得每层 ...
- 吴恩达机器学习笔记30-神经网络的反向传播算法(Backpropagation Algorithm)
之前我们在计算神经网络预测结果的时候我们采用了一种正向传播方法,我们从第一层开始正向一层一层进行计算,直到最后一层的ℎ
随机推荐
- php一些常规动态设置与获取
error_reporting(E_ALL); ini_set('display_errors', TRUE); ini_set('display_startup_errors', TRUE);ini ...
- Json学习一(基础概念知识学习)
1.Json简单介绍 JSON(JavaScript Object Notation) 是一种轻量级的数据交换格式.它使得人们非常easy的进行阅读和编写. 同一时候也方便了机器进行解析和生成.它是基 ...
- kafka学习之-配置详解
# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreement ...
- VC++实现编辑框输入提示效果
编辑框在第一次输入时最好给出一个虚拟的输入提示信息文本,这样的效果更佳友好.,我在编辑框添加灰色提示字(html+VC)一文中简单介绍了一些方法,但是效果欠佳. 原始的编辑框CEdit类没有这样的功能 ...
- cesium可视化空间数据2
圆柱圆锥体 <!DOCTYPE html> <html> <head> <!-- Use correct character set. --> < ...
- 【SJSB】 android-javascript开发框架
这是一个本人开发的android壳,作用就是为html5开发提供各种接口以调用native. 只是第一个版本,随笔以纪念.详细的说明和api都在 项目的readme中. 项目地址: https://g ...
- Java课后简答题
1.简述Java的特点. 面向对象.跨平台性.健壮性.安全性.可移植性.多线程性.动态性等. 2.简述JRE与JDK的区别. JRE(Java Runtime Environment,Java运行时环 ...
- row_number()over函数的使用(转)
(转)http://hi.baidu.com/122439049/blog/item/0c9c48131b2734d5f7039e13.html row_number() OVER (PARTITIO ...
- python2.0 s12 day3
s12 day3 视频每节的内容 03 python s12 day3 本节内容概要 第三天的主要内容 上节没讲完的: 6.集合 7.collections 1)计数器 2)有序字典 3)默认字典 4 ...
- 什么是LTE?
LTE是英文Long Term Evolution的缩写.LTE也被通俗的称为3.9G,具有100Mbps的数据下载能力,被视作从3G向4G演进的主流技术.它改进并增强了3G的空中接入技术,采用OFD ...