Huang H., Wang Y., Erfani S., Gu Q., Bailey J. and Ma X. Exploring architectural ingredients of adversarially robust deep neural networks. In Advances in Neural Information Processing Systems (NIPS), 2021

概

本文是对现有的残差网络结构的探索, grid search一个鲁棒的结构.

主要内容

大家普遍认为越大的模型鲁棒性能会越好, 某种程度上如此, 但是现有的WRN(Wide ResNet)是为干净精度设计的, 对于鲁棒性并不是最优的.

现在的WRN有三个stage:

其越到后面越宽(即卷积核个数越多).

比如标准的WRN-34-10, 每个stage有5个block, 均乘上了factor=10.

本文便是探究block数量(即网络深度), 以及factor(即宽度)的影响.

深度

由上图可知, 削弱最后一个stage能够有效提升鲁棒性.

宽度

同样的, 削弱最后一个stage能够有效提升鲁棒性.

结合二者, 作者发现, 宽度比深度更有效, 维持10-10-4的比例的模型是最优的.

若进一步改为20-20-8(同比例scale), 鲁棒性接近饱和.

感觉给人的启示是, 最后一stage不能有太强的表达能力, 为什么?

我感觉还是残差连接的原因啊.

代码

原文代码

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks的更多相关文章

  1. [Box] Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint

    目录 概 主要内容 LSGD Box 初始化 Box for Resnet 代码 Cyr E C, Gulian M, Patel R G, et al. Robust Training and In ...

  2. （转）Understanding, generalisation, and transfer learning in deep neural networks

    Understanding, generalisation, and transfer learning in deep neural networks FEBRUARY 27, 2017   Thi ...

  3. [C4] Andrew Ng - Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

    About this Course This course will teach you the "magic" of getting deep learning to work ...

  4. Classifying plankton with deep neural networks

    Classifying plankton with deep neural networks The National Data Science Bowl, a data science compet ...

  5. Why are Eight Bits Enough for Deep Neural Networks?

    Why are Eight Bits Enough for Deep Neural Networks? Deep learning is a very weird technology. It evo ...

  6. 论文翻译：2018_Source localization using deep neural networks in a shallow water environment

    论文地址:https://asa.scitation.org/doi/abs/10.1121/1.5036725 深度神经网络在浅水环境中的源定位 摘要: 深度神经网络(DNNs)在表征复杂的非线性关 ...

  7. Training Deep Neural Networks

    http://handong1587.github.io/deep_learning/2015/10/09/training-dnn.html  //转载于 Training Deep Neural ...

  8. On Explainability of Deep Neural Networks

    On Explainability of Deep Neural Networks « Learning F# Functional Data Structures and Algorithms is ...

  9. Introduction to Deep Neural Networks

    Introduction to Deep Neural Networks Neural networks are a set of algorithms, modeled loosely after ...

随机推荐

  1. Java 数据类型转化

    目录 Java类型转化 基本数据类型自动类型转换 自动类型提升 强制类型转换 - 自动类型提升的逆运算 int与long int类型与String类型 int类型转换成String类型 方法1:+ 拼 ...

  2. 学习java 7.18

    学习内容: Lambda表达式的格式:(形式参数)  ->  {代码块} 如果有多个参数,参数之间用逗号隔开 new Thread(  ()   ->   { System.out.pri ...

  3. keybd_event模拟键盘按键，mouse_event怎么用

    从 模仿UP主,用Python实现一个弹幕控制的直播间! - 蛮三刀酱 - 博客园 (cnblogs.com) 知道了 PyAutoGUI: * Moving the mouse and clicki ...

  4. accommodate ~ ache

    accommodate The accommodation reflex [反射] (or accommodation-convergence [会聚] reflex) is a reflex act ...

  5. linux 内存变量的分布

    我们知道,linux通过虚拟内存管理进程的内存(进程的地址空间),而进程的地址空间分布如下 : 从进程的空间中可以看出,内存中的变量有的来自可执行elf文件,在elf文件中已经分配好存储空间,有的是在 ...

  6. css相关，position定位详解

    CSS 有两个最重要的基本属性,前端开发必须掌握:display 和 position. display属性指定网页的布局.两个重要的布局,弹性布局flex和网格布局grid. 本文介绍非常有用的po ...

  7. Advanced C++ | Conversion Operators

    In C++, the programmer abstracts real world objects using classes as concrete types. Sometimes it is ...

  8. Classs类

    Classs类如何获得 获得Class对象 方式一: 通过Object类中的getClass()方法 方式二: 通过 类名.class 获取到字节码文件对象( 方式三: 通过Class类中的方法(将类 ...

  9. 3.使用Spring Data ElasticSearch操作ElasticSearch(5.6.8版本)

    1.引入maven坐标 <!--spring-data-elasticsearch--><dependency> <groupId>org.springframew ...

  10. ssm+mysql+jsp打造在线考试系统WeKnow-学生端

    一.登陆模块 前台提交账号和密码传到后台处理控制层 1.1 首先是控制器 @RequestMapping(value="/studentLogin", method=Request ...