output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.

http://neuralnetworksanddeeplearning.com/chap1.html

. Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.

http://neuralnetworksanddeeplearning.com/chap3.html

// This is a paper.js widget to show a single neuron learning.  In

// particular, the widget is used to show the learning slowdown that

// occurs when the output is saturated.

//

// The same basic widget is used several times, in slightly different

// configurations.  paper.js makes it somewhat complex to reuse the

// code, so I have simply duplicated the code.  This can give rise to

// bugs if one is not careful to keep the code in sync, so I have

// separated the code into two pieces.

//

// The first piece is the header code.  This changes between widgets.

// It sets up things like the starting weight, bias, cost function,

// and so on -- things which may vary betweens widgets.

//

// The second piece is the body code.  This is almost exactly the same

// for the different widgets.  Note, however, that the costGraphX and

// epochX variables change name, due to a bug in the way paperjs

// handles scope.

//

// We can make these changes by searching on costGraph1 and replacing

// with costGraph2, costGraph3 etc, by replacing epoch1 with epoch2,

// epoch3 etc, and by replcacing cost1 with cost2, cost3 etc.

//

// This separation makes it easy to maintain the duplicated code.

// HEADER CODE

var startingWeight = 0.6;

var startingBias = 0.9;

var eta = 0.15;

var numFrames = 300;

quadratic_cost = {

    fn: function(a) {return a*a/2;},

    derivative: function(a) {return a*a*(1-a);},

    scaling: 240 // used to scale on the graph

}

cross_entropy_cost = {

    fn: function(a) {return -Math.log(1-a);},

    derivative: function(a) {return 1/(1-a);},

    scaling: 30

}

cost1 = quadratic_cost;

// A path for the graph.

costGraph1 = new Path();

costGraph1.strokeColor = "#2A6EA6";

// BODY CODE

// STATIC ELEMENTS

//

// Note that this includes some paper.js items which will later be

// modified, e.g., the variables output and weightText.  This section

// merely sets the static parts of the elements.

var input = new PointText(new Point(8, 40));

input.fontSize = 18;

input.content = "Input: 1.0";

arrow(new Point(100, 35), new Point(230, 35), 0.8); // input arrow

var neuron = new Path.Circle(new Point(260, 35), 30);

neuron.strokeColor = "black";

arrow(new Point(290, 35), new Point(380, 35), 0.8); // output arrow

// The output text's content will be set dynamically, later

var output = new PointText(new Point(390, 40));

output.fontSize = 18;

// The weight text and bar

var weightText = new PointText(new Point(120, 52));

weightText.fontSize=14;

var weightBar = new Path.Rectangle(new Rectangle(120, 57, 90, 9));

weightBar.strokeColor = "grey";

weightBar.strokeWidth = 1;

var weightTick = new Path(new Point(165, 57), new Point(165, 71));

weightTick.strokeColor = "black";

var weightSlider = new Path.Line(

    new Point(165, 61.5), new Point(165+weight*20, 61.5));

weightSlider.strokeColor = "#2A6EA6";

weightSlider.strokeWidth = 9;

// The bias text and bar

var biasText = new PointText(new Point(230, 82));

biasText.fontSize = 14;

var biasBar = new Path.Rectangle(new Rectangle(230, 88, 90, 9));

biasBar.strokeColor = "grey";

biasBar.strokeWidth = 1;

var biasTick = new Path(new Point(275, 88), new Point(275, 102));

biasTick.strokeColor = "black";

var biasSlider = new Path.Line(

    new Point(275, 92.5), new Point(275+bias*20, 92.5));

biasSlider.strokeColor = "#2A6EA6";

biasSlider.strokeWidth = 9;

// Axes for the graph

arrow(new Point(100, 250), new Point(100, 120));

arrow(new Point(100, 250), new Point(130+numFrames/2, 250));

// Labels on the axes

var costText = new PointText(new Point(60, 145));

costText.fontSize = 18;

costText.content = "Cost";

var epoch1LabelText = new PointText(new Point(140+numFrames/2, 255));

epoch1LabelText.fontSize = 18;

epoch1LabelText.content = "Epoch";

// Marker for the current epoch

var epoch1Tick = new Path(new Point(100, 250), new Point(100, 255));

epoch1Tick.strokeColor = "black";

var epoch1Number = new PointText(new Point(100, 267));

epoch1Number.fontSize = 14;

epoch1Number.justification = "center";

// We group the epochTick and epochNumber, to make it easy to move

epoch1 = new Group([epoch1Tick, epoch1Number]);

// Initialize the dynamic elements.  It's convenient to do this in a

// function, so that function can also be called upon a (re)start of

// the widget.

var weight, bias;

initDynamicElements();

function initDynamicElements() {

    weight = startingWeight;

    bias = startingBias;

    weightText.content = paramContent("w = ", weight);

    weightSlider.segments[1].point.x = 165+weight*20;

    biasText.content = paramContent("b = ", bias);

    biasSlider.segments[1].point.x = 275+bias*20;

    output.content = outputContent(weight, bias);

    epoch1.position.x = 100;

    epoch1Number.content = "0";

    costGraph1.removeSegments();

}

function paramContent(s, x) {

    sign = (x >= 0)? "+": "";

    return s+sign+x.toFixed(2);

}

// The run button

var runBox = new Path.Rectangle(new Rectangle(430, 230, 60, 30), 5);

runBox.fillColor = "#dddddd";

var runText = new PointText(new Point(460, 250));

runText.justification = "center";

runText.fontSize = 18;

runText.content = "Run";

var runIcon = new Group([runBox, runText]);

runIcon.onMouseEnter = function(event) {

    runBox.fillColor = "#aaaaaa";

}

runIcon.onMouseLeave = function(event) {

    runBox.fillColor = "#dddddd";

}

var playing = false;

var count = 0;

runIcon.onClick = function(event) {

    initDynamicElements();

    this.visible = false;

    weight = startingWeight;

    bias = startingBias;

    playing = true;

}

// The actual procedure

function onFrame(event) {

    if (playing) {

	a = outputValue(weight, bias);

	delta = cost1.derivative(a);

	weight += -eta*delta;

	bias += -eta*delta;

	weightText.content = paramContent("w = ", weight);

	weightSlider.segments[1].point.x = 165+weight*20;

	biasText.content = paramContent("b = ", bias);

	biasSlider.segments[1].point.x = 275+bias*20;

	output.content = outputContent(weight, bias);

	if (count % 2 === 0) {epoch1.position.x += 1;}

	costGraph1.add(new Point(epoch1.position.x, 250-cost1.scaling*cost1.fn(a)));

	epoch1Number.content = count;

	count += 1;

	if (count > numFrames) {

	    count = 0;

	    runIcon.visible = true;

	    playing = false;

	}

	}

}

function outputValue(weight, bias) {

    return sigmoid(weight+bias);

}

function outputContent(weight, bias) {

    return "Output: "+outputValue(weight, bias).toFixed(2);

}

function sigmoid(z) {

    return 1/(1+Math.exp(-z));

}

function arrow(point1, point2, width, color) {

    if (typeof width === 'undefined') {width=1};

    if (typeof color === 'undefined') {color='black'};

    delta = point1 - point2;

    n = delta/delta.length;

    nperp = new Point(-n.y, n.x);

    line = new Path(point1, point2);

    line.strokeColor = color;

    line.strokeWidth = width;

    arrow_stroke_1 = new Path(point2, point2+(n+nperp)*6);

    arrow_stroke_1.strokeWidth = width;

    arrow_stroke_1.strokeColor = color;

    arrow_stroke_2 = new Path(point2, point2+(n-nperp)*6);

    arrow_stroke_2.strokeWidth = width;

    arrow_stroke_2.strokeColor = color;

}

http://neuralnetworksanddeeplearning.com/js/saturation4.js

// This is a paper.js widget to show a single neuron learning.  In

// particular, the widget is used to show the learning slowdown that

// occurs when the output is saturated.

//

// The same basic widget is used several times, in slightly different

// configurations.  paper.js makes it somewhat complex to reuse the

// code, so I have simply duplicated the code.  This can give rise to

// bugs if one is not careful to keep the code in sync, so I have

// separated the code into two pieces.

//

// The first piece is the header code.  This changes between widgets.

// It sets up things like the starting weight, bias, cost function,

// and so on -- things which may vary betweens widgets.

//

// The second piece is the body code.  This is almost exactly the same

// for the different widgets.  Note, however, that the costGraphX and

// epochX variables change name, due to a bug in the way paperjs

// handles scope.

//

// We can make these changes by searching on costGraph1 and replacing

// with costGraph2, costGraph3 etc, and by replacing epoch1 with

// epoch2, epoch3 etc.

//

// This separation makes it easy to maintain the duplicated code.

// HEADER CODE

var startingWeight = 2.0;

var startingBias = 2.0;

var eta = 0.005;

var numFrames = 300;

quadratic_cost = {

    fn: function(a) {return a*a/2;},

    derivative: function(a) {return a*a*(1-a);},

    scaling: 240 // used to scale on the graph

}

cross_entropy_cost = {

    fn: function(a) {return -Math.log(1-a);},

    derivative: function(a) {return 1/(1-a);},

    scaling: 30

}

cost4 = cross_entropy_cost;

// A path for the graph.

costGraph4 = new Path();

costGraph4.strokeColor = "#2A6EA6";

// BODY CODE

// STATIC ELEMENTS

//

// Note that this includes some paper.js items which will later be

// modified, e.g., the variables output and weightText.  This section

// merely sets the static parts of the elements.

var input = new PointText(new Point(8, 40));

input.fontSize = 18;

input.content = "Input: 1.0";

arrow(new Point(100, 35), new Point(230, 35), 0.8); // input arrow

var neuron = new Path.Circle(new Point(260, 35), 30);

neuron.strokeColor = "black";

arrow(new Point(290, 35), new Point(380, 35), 0.8); // output arrow

// The output text's content will be set dynamically, later

var output = new PointText(new Point(390, 40));

output.fontSize = 18;

// The weight text and bar

var weightText = new PointText(new Point(120, 52));

weightText.fontSize=14;

var weightBar = new Path.Rectangle(new Rectangle(120, 57, 90, 9));

weightBar.strokeColor = "grey";

weightBar.strokeWidth = 1;

var weightTick = new Path(new Point(165, 57), new Point(165, 71));

weightTick.strokeColor = "black";

var weightSlider = new Path.Line(

    new Point(165, 61.5), new Point(165+weight*20, 61.5));

weightSlider.strokeColor = "#2A6EA6";

weightSlider.strokeWidth = 9;

// The bias text and bar

var biasText = new PointText(new Point(230, 82));

biasText.fontSize = 14;

var biasBar = new Path.Rectangle(new Rectangle(230, 88, 90, 9));

biasBar.strokeColor = "grey";

biasBar.strokeWidth = 1;

var biasTick = new Path(new Point(275, 88), new Point(275, 102));

biasTick.strokeColor = "black";

var biasSlider = new Path.Line(

    new Point(275, 92.5), new Point(275+bias*20, 92.5));

biasSlider.strokeColor = "#2A6EA6";

biasSlider.strokeWidth = 9;

// Axes for the graph

arrow(new Point(100, 250), new Point(100, 120));

arrow(new Point(100, 250), new Point(130+numFrames/2, 250));

// Labels on the axes

var costText = new PointText(new Point(60, 145));

costText.fontSize = 18;

costText.content = "Cost";

var epoch4LabelText = new PointText(new Point(140+numFrames/2, 255));

epoch4LabelText.fontSize = 18;

epoch4LabelText.content = "Epoch";

// Marker for the current epoch

var epoch4Tick = new Path(new Point(100, 250), new Point(100, 255));

epoch4Tick.strokeColor = "black";

var epoch4Number = new PointText(new Point(100, 267));

epoch4Number.fontSize = 14;

epoch4Number.justification = "center";

// We group the epochTick and epochNumber, to make it easy to move

epoch4 = new Group([epoch4Tick, epoch4Number]);

// Initialize the dynamic elements.  It's convenient to do this in a

// function, so that function can also be called upon a (re)start of

// the widget.

var weight, bias;

initDynamicElements();

function initDynamicElements() {

    weight = startingWeight;

    bias = startingBias;

    weightText.content = paramContent("w = ", weight);

    weightSlider.segments[1].point.x = 165+weight*20;

    biasText.content = paramContent("b = ", bias);

    biasSlider.segments[1].point.x = 275+bias*20;

    output.content = outputContent(weight, bias);

    epoch4.position.x = 100;

    epoch4Number.content = "0";

    costGraph4.removeSegments();

}

function paramContent(s, x) {

    sign = (x >= 0)? "+": "";

    return s+sign+x.toFixed(2);

}

// The run button

var runBox = new Path.Rectangle(new Rectangle(430, 230, 60, 30), 5);

runBox.fillColor = "#dddddd";

var runText = new PointText(new Point(460, 250));

runText.justification = "center";

runText.fontSize = 18;

runText.content = "Run";

var runIcon = new Group([runBox, runText]);

runIcon.onMouseEnter = function(event) {

    runBox.fillColor = "#aaaaaa";

}

runIcon.onMouseLeave = function(event) {

    runBox.fillColor = "#dddddd";

}

var playing = false;

var count = 0;

runIcon.onClick = function(event) {

    initDynamicElements();

    this.visible = false;

    weight = startingWeight;

    bias = startingBias;

    playing = true;

}

// The actual procedure

function onFrame(event) {

    if (playing) {

	a = outputValue(weight, bias);

	delta = cost4.derivative(a);

	weight += -eta*delta;

	bias += -eta*delta;

	weightText.content = paramContent("w = ", weight);

	weightSlider.segments[1].point.x = 165+weight*20;

	biasText.content = paramContent("b = ", bias);

	biasSlider.segments[1].point.x = 275+bias*20;

	output.content = outputContent(weight, bias);

	if (count % 2 === 0) {epoch4.position.x += 1;}

	costGraph4.add(new Point(epoch4.position.x, 250-cost4.scaling*cost4.fn(a)));

	epoch4Number.content = count;

	count += 1;

	if (count > numFrames) {

	    count = 0;

	    runIcon.visible = true;

	    playing = false;

	}

	}

}

function outputValue(weight, bias) {

    return sigmoid(weight+bias);

}

function outputContent(weight, bias) {

    return "Output: "+outputValue(weight, bias).toFixed(2);

}

function sigmoid(z) {

    return 1/(1+Math.exp(-z));

}

function arrow(point1, point2, width, color) {

    if (typeof width === 'undefined') {width=1};

    if (typeof color === 'undefined') {color='black'};

    delta = point1 - point2;

    n = delta/delta.length;

    nperp = new Point(-n.y, n.x);

    line = new Path(point1, point2);

    line.strokeColor = color;

    line.strokeWidth = width;

    arrow_stroke_1 = new Path(point2, point2+(n+nperp)*6);

    arrow_stroke_1.strokeWidth = width;

    arrow_stroke_1.strokeColor = color;

    arrow_stroke_2 = new Path(point2, point2+(n-nperp)*6);

    arrow_stroke_2.strokeWidth = width;

    arrow_stroke_2.strokeColor = color;

}

output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.的更多相关文章

使用神经网络识别手写数字Using neural nets to recognize handwritten digits
The human visual system is one of the wonders of the world. Consider the following sequence of handw ...
chapter1:using neural nets to recognize handwritten digits
two important types of artificial neuron :the perceptron and the sigmoid neuron Perceptrons 感知机的输入个数 ...
提高神经网络的学习方式Improving the way neural networks learn
When a golf player is first learning to play golf, they usually spend most of their time developing ...
神经网络和Deep Learning
参考资料: 在线免费书籍 http://neuralnetworksanddeeplearning.com/chap1.html Chapter 1 1. perceptron 感知机 it's a ...
（六）6.16 Neurons Networks linear decoders and its implements
Sparse AutoEncoder是一个三层结构的网络,分别为输入输出与隐层,前边自编码器的描述可知,神经网络中的神经元都采用相同的激励函数,Linear Decoders 修改了自编码器的定义,对 ...
[LeetCode] Similar RGB Color 相似的红绿蓝颜色
In the following, every capital letter represents some hexadecimal digit from 0 to f. The red-green- ...
CS229 6.16 Neurons Networks linear decoders and its implements
Sparse AutoEncoder是一个三层结构的网络,分别为输入输出与隐层,前边自编码器的描述可知,神经网络中的神经元都采用相同的激励函数,Linear Decoders 修改了自编码器的定义,对 ...
Digital Adjustment of DC-DC Converter Output Voltage in Portable Applications
http://pdfserv.maximintegrated.com/en/an/AN818.pdf http://www.maximintegrated.com/app-notes/index.mv ...
Simple Addition Permits Voltage Control Of DC-DC Converter's Output
http://electronicdesign.com/power/simple-addition-permits-voltage-control-dc-dc-converters-output In ...

随机推荐

第一章初识shiro
shiro学习教程来自开涛大神的博客:http://jinnianshilongnian.iteye.com/blog/2018936 第一章初识shiro 简单了解shiro主要记住三张图即可. ...
javascript 中event是全局变量
The only thing I can think of is that event is in fact window.event and it makes itself available wh ...
POJ 1040 Transportation
链接:http://poj.org/problem?id=1040 Transportation Time Limit: 1000MS Memory Limit: 10000K Total Submi ...
iOS实录：GCD使用小结（一）
导语:在iOS中,多线程方案有四种:pthread.NSThread.NSOperation & NSOperationQueue 和 GCD,但是开发中GCD使用得最多,本文主要总结一下我使 ...
Pixhawk---fatal: Not a git repository (or any of the parent directories)
当从github.com上面下载下了Firmware后.无意中删除了Firmware文件夹下的.git文件夹,再去编译就会出现: fatal: Not a git repository (or ...
Php函数之end
Php函数之end end()函数 (PHP 4, PHP 5, PHP 7) end - 将数组的内部指针指向最后一个单元说明 mixed end ( array &$array ) en ...
jquery load方式浏览器断点调试
jquery load的方式引入的,如果需要在浏览器中断点调试,需要在代码中使用debugger进行断点
redhat 用yum安装的apache、mysql一般默认安装在哪个目录下?
使用yum安装成功后,使用rpm -qa | grep httpd和rpm -qa | grep mysql查看是否安装成功然后使用rpm -ql httpd和rpm -ql mysql查看安装文件都 ...
Oracle 11g 递归+ exists运行计划的改变
有一个递归查询在10g上执行非常快,但在11g上执行不出来. SQL> select * from v$version; BANNER ----------------------------- ...
UVA 617 - Nonstop Travel（数论+暴力枚举）
题目链接:617 - Nonstop Travel 题意:给定一些红绿灯.如今速度能在30-60km/h之内,求多少个速度满足一路不遇到红灯. 思路:暴力每个速度,去推断可不能够,最后注意下输出格式就 ...

output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.

output value . Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.的更多相关文章

随机推荐

热门专题