output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.
http://neuralnetworksanddeeplearning.com/chap1.html
. Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.
http://neuralnetworksanddeeplearning.com/chap3.html
// This is a paper.js widget to show a single neuron learning. In
// particular, the widget is used to show the learning slowdown that
// occurs when the output is saturated.
//
// The same basic widget is used several times, in slightly different
// configurations. paper.js makes it somewhat complex to reuse the
// code, so I have simply duplicated the code. This can give rise to
// bugs if one is not careful to keep the code in sync, so I have
// separated the code into two pieces.
//
// The first piece is the header code. This changes between widgets.
// It sets up things like the starting weight, bias, cost function,
// and so on -- things which may vary betweens widgets.
//
// The second piece is the body code. This is almost exactly the same
// for the different widgets. Note, however, that the costGraphX and
// epochX variables change name, due to a bug in the way paperjs
// handles scope.
//
// We can make these changes by searching on costGraph1 and replacing
// with costGraph2, costGraph3 etc, by replacing epoch1 with epoch2,
// epoch3 etc, and by replcacing cost1 with cost2, cost3 etc.
//
// This separation makes it easy to maintain the duplicated code. // HEADER CODE var startingWeight = 0.6;
var startingBias = 0.9;
var eta = 0.15;
var numFrames = 300; quadratic_cost = {
fn: function(a) {return a*a/2;},
derivative: function(a) {return a*a*(1-a);},
scaling: 240 // used to scale on the graph
} cross_entropy_cost = {
fn: function(a) {return -Math.log(1-a);},
derivative: function(a) {return 1/(1-a);},
scaling: 30
} cost1 = quadratic_cost; // A path for the graph.
costGraph1 = new Path();
costGraph1.strokeColor = "#2A6EA6"; // BODY CODE // STATIC ELEMENTS
//
// Note that this includes some paper.js items which will later be
// modified, e.g., the variables output and weightText. This section
// merely sets the static parts of the elements. var input = new PointText(new Point(8, 40));
input.fontSize = 18;
input.content = "Input: 1.0"; arrow(new Point(100, 35), new Point(230, 35), 0.8); // input arrow var neuron = new Path.Circle(new Point(260, 35), 30);
neuron.strokeColor = "black"; arrow(new Point(290, 35), new Point(380, 35), 0.8); // output arrow // The output text's content will be set dynamically, later
var output = new PointText(new Point(390, 40));
output.fontSize = 18; // The weight text and bar
var weightText = new PointText(new Point(120, 52));
weightText.fontSize=14;
var weightBar = new Path.Rectangle(new Rectangle(120, 57, 90, 9));
weightBar.strokeColor = "grey";
weightBar.strokeWidth = 1;
var weightTick = new Path(new Point(165, 57), new Point(165, 71));
weightTick.strokeColor = "black";
var weightSlider = new Path.Line(
new Point(165, 61.5), new Point(165+weight*20, 61.5));
weightSlider.strokeColor = "#2A6EA6";
weightSlider.strokeWidth = 9; // The bias text and bar
var biasText = new PointText(new Point(230, 82));
biasText.fontSize = 14;
var biasBar = new Path.Rectangle(new Rectangle(230, 88, 90, 9));
biasBar.strokeColor = "grey";
biasBar.strokeWidth = 1;
var biasTick = new Path(new Point(275, 88), new Point(275, 102));
biasTick.strokeColor = "black";
var biasSlider = new Path.Line(
new Point(275, 92.5), new Point(275+bias*20, 92.5));
biasSlider.strokeColor = "#2A6EA6";
biasSlider.strokeWidth = 9; // Axes for the graph
arrow(new Point(100, 250), new Point(100, 120));
arrow(new Point(100, 250), new Point(130+numFrames/2, 250)); // Labels on the axes
var costText = new PointText(new Point(60, 145));
costText.fontSize = 18;
costText.content = "Cost"; var epoch1LabelText = new PointText(new Point(140+numFrames/2, 255));
epoch1LabelText.fontSize = 18;
epoch1LabelText.content = "Epoch"; // Marker for the current epoch
var epoch1Tick = new Path(new Point(100, 250), new Point(100, 255));
epoch1Tick.strokeColor = "black"; var epoch1Number = new PointText(new Point(100, 267));
epoch1Number.fontSize = 14;
epoch1Number.justification = "center"; // We group the epochTick and epochNumber, to make it easy to move
epoch1 = new Group([epoch1Tick, epoch1Number]); // Initialize the dynamic elements. It's convenient to do this in a
// function, so that function can also be called upon a (re)start of
// the widget. var weight, bias;
initDynamicElements(); function initDynamicElements() {
weight = startingWeight;
bias = startingBias;
weightText.content = paramContent("w = ", weight);
weightSlider.segments[1].point.x = 165+weight*20;
biasText.content = paramContent("b = ", bias);
biasSlider.segments[1].point.x = 275+bias*20;
output.content = outputContent(weight, bias);
epoch1.position.x = 100;
epoch1Number.content = "0";
costGraph1.removeSegments();
} function paramContent(s, x) {
sign = (x >= 0)? "+": "";
return s+sign+x.toFixed(2);
} // The run button var runBox = new Path.Rectangle(new Rectangle(430, 230, 60, 30), 5);
runBox.fillColor = "#dddddd"; var runText = new PointText(new Point(460, 250));
runText.justification = "center";
runText.fontSize = 18;
runText.content = "Run"; var runIcon = new Group([runBox, runText]); runIcon.onMouseEnter = function(event) {
runBox.fillColor = "#aaaaaa";
} runIcon.onMouseLeave = function(event) {
runBox.fillColor = "#dddddd";
} var playing = false;
var count = 0; runIcon.onClick = function(event) {
initDynamicElements();
this.visible = false;
weight = startingWeight;
bias = startingBias;
playing = true;
} // The actual procedure function onFrame(event) {
if (playing) {
a = outputValue(weight, bias);
delta = cost1.derivative(a);
weight += -eta*delta;
bias += -eta*delta;
weightText.content = paramContent("w = ", weight);
weightSlider.segments[1].point.x = 165+weight*20;
biasText.content = paramContent("b = ", bias);
biasSlider.segments[1].point.x = 275+bias*20;
output.content = outputContent(weight, bias);
if (count % 2 === 0) {epoch1.position.x += 1;}
costGraph1.add(new Point(epoch1.position.x, 250-cost1.scaling*cost1.fn(a)));
epoch1Number.content = count;
count += 1;
if (count > numFrames) {
count = 0;
runIcon.visible = true;
playing = false;
}
}
} function outputValue(weight, bias) {
return sigmoid(weight+bias);
} function outputContent(weight, bias) {
return "Output: "+outputValue(weight, bias).toFixed(2);
} function sigmoid(z) {
return 1/(1+Math.exp(-z));
} function arrow(point1, point2, width, color) {
if (typeof width === 'undefined') {width=1};
if (typeof color === 'undefined') {color='black'};
delta = point1 - point2;
n = delta/delta.length;
nperp = new Point(-n.y, n.x);
line = new Path(point1, point2);
line.strokeColor = color;
line.strokeWidth = width;
arrow_stroke_1 = new Path(point2, point2+(n+nperp)*6);
arrow_stroke_1.strokeWidth = width;
arrow_stroke_1.strokeColor = color;
arrow_stroke_2 = new Path(point2, point2+(n-nperp)*6);
arrow_stroke_2.strokeWidth = width;
arrow_stroke_2.strokeColor = color;
}
http://neuralnetworksanddeeplearning.com/js/saturation4.js
// This is a paper.js widget to show a single neuron learning. In
// particular, the widget is used to show the learning slowdown that
// occurs when the output is saturated.
//
// The same basic widget is used several times, in slightly different
// configurations. paper.js makes it somewhat complex to reuse the
// code, so I have simply duplicated the code. This can give rise to
// bugs if one is not careful to keep the code in sync, so I have
// separated the code into two pieces.
//
// The first piece is the header code. This changes between widgets.
// It sets up things like the starting weight, bias, cost function,
// and so on -- things which may vary betweens widgets.
//
// The second piece is the body code. This is almost exactly the same
// for the different widgets. Note, however, that the costGraphX and
// epochX variables change name, due to a bug in the way paperjs
// handles scope.
//
// We can make these changes by searching on costGraph1 and replacing
// with costGraph2, costGraph3 etc, and by replacing epoch1 with
// epoch2, epoch3 etc.
//
// This separation makes it easy to maintain the duplicated code. // HEADER CODE var startingWeight = 2.0;
var startingBias = 2.0;
var eta = 0.005;
var numFrames = 300; quadratic_cost = {
fn: function(a) {return a*a/2;},
derivative: function(a) {return a*a*(1-a);},
scaling: 240 // used to scale on the graph
} cross_entropy_cost = {
fn: function(a) {return -Math.log(1-a);},
derivative: function(a) {return 1/(1-a);},
scaling: 30
} cost4 = cross_entropy_cost; // A path for the graph.
costGraph4 = new Path();
costGraph4.strokeColor = "#2A6EA6"; // BODY CODE // STATIC ELEMENTS
//
// Note that this includes some paper.js items which will later be
// modified, e.g., the variables output and weightText. This section
// merely sets the static parts of the elements. var input = new PointText(new Point(8, 40));
input.fontSize = 18;
input.content = "Input: 1.0"; arrow(new Point(100, 35), new Point(230, 35), 0.8); // input arrow var neuron = new Path.Circle(new Point(260, 35), 30);
neuron.strokeColor = "black"; arrow(new Point(290, 35), new Point(380, 35), 0.8); // output arrow // The output text's content will be set dynamically, later
var output = new PointText(new Point(390, 40));
output.fontSize = 18; // The weight text and bar
var weightText = new PointText(new Point(120, 52));
weightText.fontSize=14;
var weightBar = new Path.Rectangle(new Rectangle(120, 57, 90, 9));
weightBar.strokeColor = "grey";
weightBar.strokeWidth = 1;
var weightTick = new Path(new Point(165, 57), new Point(165, 71));
weightTick.strokeColor = "black";
var weightSlider = new Path.Line(
new Point(165, 61.5), new Point(165+weight*20, 61.5));
weightSlider.strokeColor = "#2A6EA6";
weightSlider.strokeWidth = 9; // The bias text and bar
var biasText = new PointText(new Point(230, 82));
biasText.fontSize = 14;
var biasBar = new Path.Rectangle(new Rectangle(230, 88, 90, 9));
biasBar.strokeColor = "grey";
biasBar.strokeWidth = 1;
var biasTick = new Path(new Point(275, 88), new Point(275, 102));
biasTick.strokeColor = "black";
var biasSlider = new Path.Line(
new Point(275, 92.5), new Point(275+bias*20, 92.5));
biasSlider.strokeColor = "#2A6EA6";
biasSlider.strokeWidth = 9; // Axes for the graph
arrow(new Point(100, 250), new Point(100, 120));
arrow(new Point(100, 250), new Point(130+numFrames/2, 250)); // Labels on the axes
var costText = new PointText(new Point(60, 145));
costText.fontSize = 18;
costText.content = "Cost"; var epoch4LabelText = new PointText(new Point(140+numFrames/2, 255));
epoch4LabelText.fontSize = 18;
epoch4LabelText.content = "Epoch"; // Marker for the current epoch
var epoch4Tick = new Path(new Point(100, 250), new Point(100, 255));
epoch4Tick.strokeColor = "black"; var epoch4Number = new PointText(new Point(100, 267));
epoch4Number.fontSize = 14;
epoch4Number.justification = "center"; // We group the epochTick and epochNumber, to make it easy to move
epoch4 = new Group([epoch4Tick, epoch4Number]); // Initialize the dynamic elements. It's convenient to do this in a
// function, so that function can also be called upon a (re)start of
// the widget. var weight, bias;
initDynamicElements(); function initDynamicElements() {
weight = startingWeight;
bias = startingBias;
weightText.content = paramContent("w = ", weight);
weightSlider.segments[1].point.x = 165+weight*20;
biasText.content = paramContent("b = ", bias);
biasSlider.segments[1].point.x = 275+bias*20;
output.content = outputContent(weight, bias);
epoch4.position.x = 100;
epoch4Number.content = "0";
costGraph4.removeSegments();
} function paramContent(s, x) {
sign = (x >= 0)? "+": "";
return s+sign+x.toFixed(2);
} // The run button var runBox = new Path.Rectangle(new Rectangle(430, 230, 60, 30), 5);
runBox.fillColor = "#dddddd"; var runText = new PointText(new Point(460, 250));
runText.justification = "center";
runText.fontSize = 18;
runText.content = "Run"; var runIcon = new Group([runBox, runText]); runIcon.onMouseEnter = function(event) {
runBox.fillColor = "#aaaaaa";
} runIcon.onMouseLeave = function(event) {
runBox.fillColor = "#dddddd";
} var playing = false;
var count = 0; runIcon.onClick = function(event) {
initDynamicElements();
this.visible = false;
weight = startingWeight;
bias = startingBias;
playing = true;
} // The actual procedure function onFrame(event) {
if (playing) {
a = outputValue(weight, bias);
delta = cost4.derivative(a);
weight += -eta*delta;
bias += -eta*delta;
weightText.content = paramContent("w = ", weight);
weightSlider.segments[1].point.x = 165+weight*20;
biasText.content = paramContent("b = ", bias);
biasSlider.segments[1].point.x = 275+bias*20;
output.content = outputContent(weight, bias);
if (count % 2 === 0) {epoch4.position.x += 1;}
costGraph4.add(new Point(epoch4.position.x, 250-cost4.scaling*cost4.fn(a)));
epoch4Number.content = count;
count += 1;
if (count > numFrames) {
count = 0;
runIcon.visible = true;
playing = false;
}
}
} function outputValue(weight, bias) {
return sigmoid(weight+bias);
} function outputContent(weight, bias) {
return "Output: "+outputValue(weight, bias).toFixed(2);
} function sigmoid(z) {
return 1/(1+Math.exp(-z));
} function arrow(point1, point2, width, color) {
if (typeof width === 'undefined') {width=1};
if (typeof color === 'undefined') {color='black'};
delta = point1 - point2;
n = delta/delta.length;
nperp = new Point(-n.y, n.x);
line = new Path(point1, point2);
line.strokeColor = color;
line.strokeWidth = width;
arrow_stroke_1 = new Path(point2, point2+(n+nperp)*6);
arrow_stroke_1.strokeWidth = width;
arrow_stroke_1.strokeColor = color;
arrow_stroke_2 = new Path(point2, point2+(n-nperp)*6);
arrow_stroke_2.strokeWidth = width;
arrow_stroke_2.strokeColor = color;
}
output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.的更多相关文章
- 使用神经网络识别手写数字Using neural nets to recognize handwritten digits
The human visual system is one of the wonders of the world. Consider the following sequence of handw ...
- chapter1:using neural nets to recognize handwritten digits
two important types of artificial neuron :the perceptron and the sigmoid neuron Perceptrons 感知机的输入个数 ...
- 提高神经网络的学习方式Improving the way neural networks learn
When a golf player is first learning to play golf, they usually spend most of their time developing ...
- 神经网络和Deep Learning
参考资料: 在线免费书籍 http://neuralnetworksanddeeplearning.com/chap1.html Chapter 1 1. perceptron 感知机 it's a ...
- (六)6.16 Neurons Networks linear decoders and its implements
Sparse AutoEncoder是一个三层结构的网络,分别为输入输出与隐层,前边自编码器的描述可知,神经网络中的神经元都采用相同的激励函数,Linear Decoders 修改了自编码器的定义,对 ...
- [LeetCode] Similar RGB Color 相似的红绿蓝颜色
In the following, every capital letter represents some hexadecimal digit from 0 to f. The red-green- ...
- CS229 6.16 Neurons Networks linear decoders and its implements
Sparse AutoEncoder是一个三层结构的网络,分别为输入输出与隐层,前边自编码器的描述可知,神经网络中的神经元都采用相同的激励函数,Linear Decoders 修改了自编码器的定义,对 ...
- Digital Adjustment of DC-DC Converter Output Voltage in Portable Applications
http://pdfserv.maximintegrated.com/en/an/AN818.pdf http://www.maximintegrated.com/app-notes/index.mv ...
- Simple Addition Permits Voltage Control Of DC-DC Converter's Output
http://electronicdesign.com/power/simple-addition-permits-voltage-control-dc-dc-converters-output In ...
随机推荐
- SparkSQL的3种Join实现
引言 Join是SQL语句中的常用操作,良好的表结构能够将数据分散在不同的表中,使其符合某种范式,减少表冗余.更新容错等.而建立表和表之间关系的最佳方式就是Join操作. 对于Spark来说有3中Jo ...
- Android中IntentService详解
简单说,IntentService是继承于Service并处理异步请求的一个类,在IntentService内有一个工作线程来处理耗时操作,启动IntentService的方式和启动传统Service ...
- Windows网页上碰到无法完全显示的图片怎么办
如图所示,下一幅图片只能显示一半. 我们选中并在新标签中打开 一般即可正常显示,如果还不行,留意地址栏,这就是这个图片的真实地址,我们完全可以用迅雷直接把这幅图片下载下来. 我们甚至可以猜测,去掉后缀 ...
- 端口监听与telnet
例一: 明明端口已经监听了,为什么远程的telnet连接不上. 远程机器 telnet 10.10.1.85 53 就是进不去. Linux 防火墙一关,ok service iptables sto ...
- 计算两个有序数组的第K大数(转)
传统解法,最直观的解法是O(m+n).直接merge两个数组,然后求第K大的数字. 如果想要时间复杂度将为O(log(m+n)).我们可以考虑从K入手.如果我们每次能够删除一个一定在第K个元素之前的元 ...
- Android 冷兵器 之 tools
代码地址如下:http://www.demodashi.com/demo/12612.html 前言 Android开发在所难免的就是UI的预览和调整,一般情况下都是直接run看效果,或者是使用AS的 ...
- java检测http请求的ip地址 Java问题通用解决代码
以后再用到的话,至少能起个参考作用 java实现,struts2的Action中,依赖HttpServletRequest package net.dookoo.web.action; i ...
- asp.net core mvc视频A:笔记2-1.控制器定义
方式一:以Controller结尾 方式二:不以Controller结尾 思考 默认路由规则为 运行示例(这里不需要写testcontroller,只写test就可以了) 同理测试test类中的控制器 ...
- python 读写 json文件
json的优势: 1. 数据体积方面. JSON相对于XML来讲,数据的体积小,传递的速度更快些. 2. 传输速度方面. JSON的速度要远远快于XML 3. 数据格式 数据格式比较简单, 易于读写, ...
- matlab-非线性方程求根函数及函数曲线绘制
Matlab中提供了很多求解非线性方程(y=f(x))的函数,刚開始使用,真的很困惑.全部.这里依据matlab的help文档对这些函数做一些小小的总结 fsolve函数 用来求解非线性方程组:F(x ...