tensorflow 卷积/反卷积-池化/反池化操作详解
- Plese see this answer for a detailed example of how
tf.nn.conv2d_backprop_input
andtf.nn.conv2d_backprop_filter
in an example.
In tf.nn
, there are 4 closely related 2d conv functions:
tf.nn.conv2d
tf.nn.conv2d_backprop_filter
tf.nn.conv2d_backprop_input
tf.nn.conv2d_transpose
def conv2d(input, filter, strides, padding, use_cudnn_on_gpu=True, data_format="NHWC", name=None):
r"""Computes a 2-D convolution given 4-D `input` and `filter` tensors. Given an input tensor of shape `[batch, in_height, in_width, in_channels]`
and a filter / kernel tensor of shape
`[filter_height, filter_width, in_channels, out_channels]`, this op
performs the following: 1. Flattens the filter to a 2-D matrix with shape
`[filter_height * filter_width * in_channels, output_channels]`.
2. Extracts image patches from the input tensor to form a *virtual*
tensor of shape `[batch, out_height, out_width,
filter_height * filter_width * in_channels]`.
3. For each patch, right-multiplies the filter matrix and the image patch
vector. In detail, with the default NHWC format, output[b, i, j, k] =
sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *
filter[di, dj, q, k] Must have `strides[0] = strides[3] = 1`. For the most common case of the same
horizontal and vertices strides, `strides = [1, stride, stride, 1]`.
Given out = conv2d(x, w)
and the output gradient d_out
:
- Use
tf.nn.conv2d_backprop_filter
to compute the filter gradientd_w
- Use
tf.nn.conv2d_backprop_input
to compute the filter gradientd_x
tf.nn.conv2d_backprop_input
can be implemented bytf.nn.conv2d_transpose
- All 4 functions above can be implemented by
tf.nn.conv2d
- Actually, use TF's autodiff is the fastest way to compute gradients
Long Answer
Now, let's give an actual working code example of how to use the 4 functions above to compute d_x
and d_w
given d_out
. This shows how conv2d
, conv2d_backprop_filter
, conv2d_backprop_input
, and conv2d_transpose
are related to each other. Please find the full scripts here.
Computing d_x
in 4 different ways:
# Method 1: TF's autodiff
d_x = tf.gradients(f, x)[0]
# Method 2: manually using conv2d
d_x_manual = tf.nn.conv2d(input=tf_pad_to_full_conv2d(d_out, w_size),
filter=tf_rot180(w),
strides=strides,
padding='VALID')
# Method 3: conv2d_backprop_input
d_x_backprop_input = tf.nn.conv2d_backprop_input(input_sizes=x_shape,
filter=w,
out_backprop=d_out,
strides=strides,
padding='VALID')
# Method 4: conv2d_transpose
d_x_transpose = tf.nn.conv2d_transpose(value=d_out,
filter=w,
output_shape=x_shape,
strides=strides,
padding='VALID')
Computing d_w
in 3 different ways:
# Method 1: TF's autodiff
d_w = tf.gradients(f, w)[0]
# Method 2: manually using conv2d
d_w_manual = tf_NHWC_to_HWIO(tf.nn.conv2d(input=x,
filter=tf_NHWC_to_HWIO(d_out),
strides=strides,
padding='VALID'))
# Method 3: conv2d_backprop_filter
d_w_backprop_filter = tf.nn.conv2d_backprop_filter(input=x,
filter_sizes=w_shape,
out_backprop=d_out,
strides=strides,
padding='VALID')
Please see the full scripts for the implementation of tf_rot180
, tf_pad_to_full_conv2d
, tf_NHWC_to_HWIO
. In the scripts, we check that the final output values of different methods are the same; a numpy implementation is also available.
- 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用
- http://www.cnblogs.com/pinard/p/6494810.html :卷积神经网络(CNN)反向传播算法
- http://blog.csdn.net/yunpiao123456/article/details/52437794
tensorflow 卷积/反卷积-池化/反池化操作详解的更多相关文章
- 超详细的Tensorflow模型的保存和加载(理论与实战详解)
1.Tensorflow的模型到底是什么样的? Tensorflow模型主要包含网络的设计(图)和训练好的各参数的值等.所以,Tensorflow模型有两个主要的文件: a) Meta graph: ...
- jdbc连接池中c3p0的配置文件的详解以及在在java中如何使用
<c3p0-config> <!-- 默认配置,如果没有指定则使用这个配置 --> <default-config> <property name=" ...
- opencv-python图像二值化函数cv2.threshold函数详解及参数cv2.THRESH_OTSU使用
cv2.threshold()函数的作用是将一幅灰度图二值化,基本用法如下: #ret:暂时就认为是设定的thresh阈值,mask:二值化的图像 ret,mask = cv2.threshold(i ...
- CentOS Minimal版最小化安装后VMware联网详解
最近想搞个mailman邮件列表,又不想在我常用的CentOS 6.4上做实验,怕破坏了环境,于是就想装个试验机,又嫌它占空间太大,于是找了半天发现CentOS 6.0的minimal版本最适合了,装 ...
- Windows里安装wireshark或者ethereal工具(包括汉化破解)(图文详解)
不多说,直接上干货! https://www.wireshark.org/download.html 我这里,读取的是,来自于https://www.ll.mit.edu/ideval/data/19 ...
- Java学习笔记 线程池使用及详解
有点笨,参考了好几篇大佬们写的文章才整理出来的笔记.... 字面意思上解释,线程池就是装有线程的池,我们可以把要执行的多线程交给线程池来处理,和连接池的概念一样,通过维护一定数量的线程池来达到多个线程 ...
- 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用
反卷积是指,通过测量输出和已知输入重构未知输入的过程.在神经网络中,反卷积过程并不具备学习的能力,仅仅是用于可视化一个已经训练好的卷积神经网络,没有学习训练的过程.反卷积有着许多特别的应用,一般可以用 ...
- 『TensorFlow』卷积层、池化层详解
一.前向计算和反向传播数学过程讲解
- 深度学习原理与框架-Tensorflow卷积神经网络-卷积神经网络mnist分类 1.tf.nn.conv2d(卷积操作) 2.tf.nn.max_pool(最大池化操作) 3.tf.nn.dropout(执行dropout操作) 4.tf.nn.softmax_cross_entropy_with_logits(交叉熵损失) 5.tf.truncated_normal(两个标准差内的正态分布)
1. tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME') # 对数据进行卷积操作 参数说明:x表示输入数据,w表示卷积核, stride ...
随机推荐
- [ 转载 ] Android JNI(一)——NDK与JNI基础
Android JNI(一)——NDK与JNI基础 隔壁老李头 关注 4.4 2018.05.09 17:15* 字数 5481 阅读 11468评论 8喜欢 140 本系列文章如下: Androi ...
- Django模版语言inclusion_tag的用法。
inclusion_tag.它多用于一个HTML片段的.例如我写的一个BBS项目中. 一个博主的主页面的左侧栏和查看博主某篇文章的页面的左栅栏的一样的.为了不用重复写同样的代码.且提高页面的扩 ...
- bzoj4001: [TJOI2015]概率论
题目链接 bzoj4001: [TJOI2015]概率论 题解 生成函数+求导 设\(g(n)\)表示有\(n\)个节点的二叉树的个数,\(g(0) = 1\) 设\(f(x)\)表示\(n\)个节点 ...
- hdu 5203 && BC Round #37 1002
代码参考自:xyz111 题意: 众所周知,萌萌哒六花不擅长数学,所以勇太给了她一些数学问题做练习,其中有一道是这样的:勇太有一根长度为n的木棍,这个木棍是由n个长度为1的小木棍拼接而成,当然由于时间 ...
- bzoj 4004 向量拟阵
题解RT. eps = 1e-10 WrongAnswer eps = 1e-5 Accepted /************************************************* ...
- hdu 5735 Born Slippy 暴力
Born Slippy 题目连接: http://acm.hdu.edu.cn/showproblem.php?pid=5735 Description Professor Zhang has a r ...
- CRC 概述
Acquired from: ftp.adelaide.edu.au:/pub/rocksoft/crc_v3.txt or ftp://ftp.rocksoft.com/papers/crc_v3. ...
- [转]小心C# 5.0 中的await and async模式造成的死锁
原文链接 https://www.cnblogs.com/OpenCoder/p/4434574.html 内容 UI Example Consider the example below. A bu ...
- mysql slave 主从 指定表 通配符
slave配置 slave端有可能,只复制部分表,有一些表不需要备份配置如下: 有一些表你可能做水平或则垂直的处理.如果表的前几位一样,就可以用通配符%匹配 replicate-wild-ignore ...
- dos下 和 批处理中的 for 语句的基本用法
for 语句的基本用法 : 最复杂的for 语句,也有其基本形态,它的模样是这样的: 在cmd 窗口中:for %I in (command1) do command2 在批处理文件中:for % ...