• Plese see this answer for a detailed example of how tf.nn.conv2d_backprop_input and tf.nn.conv2d_backprop_filter in an example.

In tf.nn, there are 4 closely related 2d conv functions:

  • tf.nn.conv2d
  • tf.nn.conv2d_backprop_filter
  • tf.nn.conv2d_backprop_input
  • tf.nn.conv2d_transpose
def conv2d(input, filter, strides, padding, use_cudnn_on_gpu=True, data_format="NHWC", name=None):
r"""Computes a 2-D convolution given 4-D `input` and `filter` tensors. Given an input tensor of shape `[batch, in_height, in_width, in_channels]`
and a filter / kernel tensor of shape
`[filter_height, filter_width, in_channels, out_channels]`, this op
performs the following: 1. Flattens the filter to a 2-D matrix with shape
`[filter_height * filter_width * in_channels, output_channels]`.
2. Extracts image patches from the input tensor to form a *virtual*
tensor of shape `[batch, out_height, out_width,
filter_height * filter_width * in_channels]`.
3. For each patch, right-multiplies the filter matrix and the image patch
vector. In detail, with the default NHWC format, output[b, i, j, k] =
sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *
filter[di, dj, q, k] Must have `strides[0] = strides[3] = 1`. For the most common case of the same
horizontal and vertices strides, `strides = [1, stride, stride, 1]`.

Given out = conv2d(x, w) and the output gradient d_out:

  • Use tf.nn.conv2d_backprop_filter to compute the filter gradient d_w
  • Use tf.nn.conv2d_backprop_input to compute the filter gradient d_x
  • tf.nn.conv2d_backprop_input can be implemented by tf.nn.conv2d_transpose
  • All 4 functions above can be implemented by tf.nn.conv2d
  • Actually, use TF's autodiff is the fastest way to compute gradients

Long Answer

Now, let's give an actual working code example of how to use the 4 functions above to compute d_x and d_w given d_out. This shows how conv2dconv2d_backprop_filterconv2d_backprop_input, and conv2d_transpose are related to each other. Please find the full scripts here.

Computing d_x in 4 different ways:

# Method 1: TF's autodiff
d_x = tf.gradients(f, x)[0] # Method 2: manually using conv2d
d_x_manual = tf.nn.conv2d(input=tf_pad_to_full_conv2d(d_out, w_size),
filter=tf_rot180(w),
strides=strides,
padding='VALID') # Method 3: conv2d_backprop_input
d_x_backprop_input = tf.nn.conv2d_backprop_input(input_sizes=x_shape,
filter=w,
out_backprop=d_out,
strides=strides,
padding='VALID') # Method 4: conv2d_transpose
d_x_transpose = tf.nn.conv2d_transpose(value=d_out,
filter=w,
output_shape=x_shape,
strides=strides,
padding='VALID')

Computing d_w in 3 different ways:

# Method 1: TF's autodiff
d_w = tf.gradients(f, w)[0] # Method 2: manually using conv2d
d_w_manual = tf_NHWC_to_HWIO(tf.nn.conv2d(input=x,
filter=tf_NHWC_to_HWIO(d_out),
strides=strides,
padding='VALID')) # Method 3: conv2d_backprop_filter
d_w_backprop_filter = tf.nn.conv2d_backprop_filter(input=x,
filter_sizes=w_shape,
out_backprop=d_out,
strides=strides,
padding='VALID')

Please see the full scripts for the implementation of tf_rot180tf_pad_to_full_conv2dtf_NHWC_to_HWIO. In the scripts, we check that the final output values of different methods are the same; a numpy implementation is also available.

tensorflow 卷积/反卷积-池化/反池化操作详解的更多相关文章

  1. 超详细的Tensorflow模型的保存和加载(理论与实战详解)

    1.Tensorflow的模型到底是什么样的? Tensorflow模型主要包含网络的设计(图)和训练好的各参数的值等.所以,Tensorflow模型有两个主要的文件: a) Meta graph: ...

  2. jdbc连接池中c3p0的配置文件的详解以及在在java中如何使用

    <c3p0-config> <!-- 默认配置,如果没有指定则使用这个配置 --> <default-config> <property name=" ...

  3. opencv-python图像二值化函数cv2.threshold函数详解及参数cv2.THRESH_OTSU使用

    cv2.threshold()函数的作用是将一幅灰度图二值化,基本用法如下: #ret:暂时就认为是设定的thresh阈值,mask:二值化的图像 ret,mask = cv2.threshold(i ...

  4. CentOS Minimal版最小化安装后VMware联网详解

    最近想搞个mailman邮件列表,又不想在我常用的CentOS 6.4上做实验,怕破坏了环境,于是就想装个试验机,又嫌它占空间太大,于是找了半天发现CentOS 6.0的minimal版本最适合了,装 ...

  5. Windows里安装wireshark或者ethereal工具(包括汉化破解)(图文详解)

    不多说,直接上干货! https://www.wireshark.org/download.html 我这里,读取的是,来自于https://www.ll.mit.edu/ideval/data/19 ...

  6. Java学习笔记 线程池使用及详解

    有点笨,参考了好几篇大佬们写的文章才整理出来的笔记.... 字面意思上解释,线程池就是装有线程的池,我们可以把要执行的多线程交给线程池来处理,和连接池的概念一样,通过维护一定数量的线程池来达到多个线程 ...

  7. 第十四节,TensorFlow中的反卷积,反池化操作以及gradients的使用

    反卷积是指,通过测量输出和已知输入重构未知输入的过程.在神经网络中,反卷积过程并不具备学习的能力,仅仅是用于可视化一个已经训练好的卷积神经网络,没有学习训练的过程.反卷积有着许多特别的应用,一般可以用 ...

  8. 『TensorFlow』卷积层、池化层详解

    一.前向计算和反向传播数学过程讲解

  9. 深度学习原理与框架-Tensorflow卷积神经网络-卷积神经网络mnist分类 1.tf.nn.conv2d(卷积操作) 2.tf.nn.max_pool(最大池化操作) 3.tf.nn.dropout(执行dropout操作) 4.tf.nn.softmax_cross_entropy_with_logits(交叉熵损失) 5.tf.truncated_normal(两个标准差内的正态分布)

    1. tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME')  # 对数据进行卷积操作 参数说明:x表示输入数据,w表示卷积核, stride ...

随机推荐

  1. c# HttpWebRequest 和HttpWebResponse 登录网站或论坛(校内网登陆)

    这是登录校内网的代码呵呵自己注册一个试试吧我的账号和密码就不给了 不过可以加我为好友      冯洪春  貌似校内上就我一个 Form1.cs代码: using System;using System ...

  2. Linux驱动之串口(UART)

    <uart驱动程序概述> 在嵌入式Linux系统中,串口被看成终端设备,终端设备(tty)的驱动程序分为3部分: tty_core tty_disicipline  tty_driver ...

  3. python3 开发面试题(面向对象)6.6

    """ 封装.继承.多态 1. 谈谈你对面向对象的理解? 2. Python面向对象中的继承有什么特点? 3. 面向对象深度优先和广度优先是什么? 4. 面向对象中sup ...

  4. android studio 使用总结

    网站1:http://stormzhang.com/posts.html 网站2:http://blog.csdn.net/hyr83960944/article/details/38388429

  5. CocoaPods第三方库管理工具

    http://code4app.com/article/cocoapods-install-usage

  6. 解决Windows x86网易云音乐不能将音乐下载到SD卡的BUG

    由于我个人最常用的电脑是Surface pro4 256G版本,装了不少生产力空间还挺吃紧的,音乐之类的必然都存单独的SD卡里.用UWP版本的网易云音乐倒是没问题,最近问题来了,UWP版本的网易云音乐 ...

  7. STM32 F4 Clock Sources

    STM32 F4 Clock Sources Goal: routing clock sources to the microcontroller output pin (MCO1)    High- ...

  8. HOWTO: Use STM32 SPI half duplex mode

    HOWTO: Use STM32 SPI half duplex mode I’ve got my hands onto some STM32F030F4P6 ARM-Cortex M0 proces ...

  9. Xamarin.iOS,AOT,JIT,Limitations

    Since applications on the iPhone using Xamarin.iOS are compiled to static code, it is not possible t ...

  10. Java---16---多线程---死锁

    死锁: 概念: 所谓死锁: 是指两个或两个以上的进程在执行过程中,因争夺资源而造成的一种互相等待的现象,若无外力作用.它们都将无法推进下去.此时称系统处于死锁状态或系统产生了死锁,这些永远在互相等待的 ...