Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
概
本文是对现有的残差网络结构的探索, grid search一个鲁棒的结构.
主要内容
大家普遍认为越大的模型鲁棒性能会越好, 某种程度上如此, 但是现有的WRN(Wide ResNet)是为干净精度设计的, 对于鲁棒性并不是最优的.
现在的WRN有三个stage:
其越到后面越宽(即卷积核个数越多).
比如标准的WRN-34-10, 每个stage有5个block, 均乘上了factor=10.
本文便是探究block数量(即网络深度), 以及factor(即宽度)的影响.
深度
由上图可知, 削弱最后一个stage能够有效提升鲁棒性.
宽度
同样的, 削弱最后一个stage能够有效提升鲁棒性.
结合二者, 作者发现, 宽度比深度更有效, 维持10-10-4的比例的模型是最优的.
若进一步改为20-20-8(同比例scale), 鲁棒性接近饱和.
感觉给人的启示是, 最后一stage不能有太强的表达能力, 为什么?
我感觉还是残差连接的原因啊.
代码
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks的更多相关文章
- [Box] Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint
目录 概 主要内容 LSGD Box 初始化 Box for Resnet 代码 Cyr E C, Gulian M, Patel R G, et al. Robust Training and In ...
- （转）Understanding, generalisation, and transfer learning in deep neural networks
Understanding, generalisation, and transfer learning in deep neural networks FEBRUARY 27, 2017 Thi ...
- [C4] Andrew Ng - Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
About this Course This course will teach you the "magic" of getting deep learning to work ...
- Classifying plankton with deep neural networks
Classifying plankton with deep neural networks The National Data Science Bowl, a data science compet ...
- Why are Eight Bits Enough for Deep Neural Networks?
Why are Eight Bits Enough for Deep Neural Networks? Deep learning is a very weird technology. It evo ...
- 论文翻译：2018_Source localization using deep neural networks in a shallow water environment
论文地址:https://asa.scitation.org/doi/abs/10.1121/1.5036725 深度神经网络在浅水环境中的源定位 摘要: 深度神经网络(DNNs)在表征复杂的非线性关 ...
- Training Deep Neural Networks
http://handong1587.github.io/deep_learning/2015/10/09/training-dnn.html //转载于 Training Deep Neural ...
- On Explainability of Deep Neural Networks
On Explainability of Deep Neural Networks « Learning F# Functional Data Structures and Algorithms is ...
- Introduction to Deep Neural Networks
Introduction to Deep Neural Networks Neural networks are a set of algorithms, modeled loosely after ...
随机推荐
- cvc-complex-type.2.3: Element 'servlet' cannot have character [children], because the type's content
错误原因:粘贴代码 <servlet> <servlet-name>barServlet</servlet-name> <servlet-class>S ...
- 学习java 7.14
学习内容: 标准输入输出流 输出语言的本质:是一个标准的输出流 字节打印流 字符打印流 对象序列化流 明天内容: 进程和线程 遇到问题: 用对象序列化流序列化一个对象后,假如我们修改了对象所属的类文件 ...
- 大数据学习day38----数据仓库01-----区域字典的生成
更多内容见文档 1. 区域字典的生成 mysql中有如下表格数据 现要将这类数据转换成(GEOHASH码, 省,市,区)如下所示 (1)第一步:在mysql中使用sql语句对表格数据进行整理(此处使用 ...
- [转]sizeof计算空间大小的总结
原文链接:http://www.cnblogs.com/houjun/p/4907622.html 关于sizeof的总结 1.sizeof的使用形式:sizeof(var_name)或者sizeof ...
- 复制virtualbox虚拟硬盘
D:\VirtualBox\VBoxManage.exe clonevdi F:\virtualbox\rac1\rac1.vdi F:\virtualbox\rac2\rac2.vdi 虚拟机软件安 ...
- 【Linux】【Basis】文件
refer to: https://en.wikipedia.org/wiki/POSIX refer to: https://en.wikipedia.org/wiki/Unix_file_type ...
- 【Linux】【Services】【KVM】安装与简单配置
1. 环境: 1.1. OS: Red Hat Enterprise Linux Server release 7.4 (Maipo) 1.2. Kernel: 3.10.0-693.el7.x86_ ...
- javascript将平行的拥有上下级关系的数据转换成树形结构
转换函数 var Littlehow = {}; /** * littlehow 2019-05-15 * 平行数据树形转换器 * @type {{format: tree.format, sort: ...
- Spring Batch Event Listeners
Learn to create and configure Spring batch's JobExecutionListener (before and after job), StepExecut ...
- JSP 文字乱码、${}引用无效
问题: 代码:<form action="/test/requestPost.do" method="post"> <input type=& ...