目录 概 主要内容 "代码" Katharopoulos A, Fleuret F. Not All Samples Are Created Equal: Deep Learning with Importance Sampling[J]. arXiv: Learning, 2018. @article{katharopoulos2018not, title={Not All Samples Are Created Equal: Deep Learning with Importanc…

目录 概 相关工作 主要内容 代码 Accelerating Deep Learning by Focusing on the Biggest Losers 概 思想很简单, 在训练网络的时候, 每个样本都会产生一个损失\(\mathcal{L}(f(x_i),y_i)\), 训练的模式往往是批训练, 将一个批次\(\sum_i \mathcal{L}(f(x_i),y_i)\)所产生的损失的梯度都传回去, 然后更新参数. 本文认为, 有些样本\((x_i,y_i)\)由于重复度高, 网络很高能…

The major advancements in Deep Learning in 2016 Pablo Tue, Dec 6, 2016 in MACHINE LEARNING DEEP LEARNING GAN Deep Learning has been the core topic in the Machine Learning community the last couple of years and 2016 was not the exception. In this arti…

Introduction Deep learning is a recent trend in machine learning that models highly non-linear representations of data. In the past years, deep learning has gained a tremendous momentum and prevalence for a variety of applications (Wikipedia 2016a).…

Summary on deep learning framework --- PyTorch  Updated on 2018-07-22 21:25:42  import osos.environ["CUDA_VISIBLE_DEVICES"]="4" 1. install the pytorch version 0.1.11  ## Version 0.1.11 ## python2.7 and cuda 8.0 sudo pip install http://…

About this Course If you want to break into cutting-edge AI, this course will help you do so. Deep learning engineers are highly sought after, and mastering deep learning will give you numerous new career opportunities. Deep learning is also a new "s…

前言 本文是基于Exercise:PCA and Whitening的练习. 理论知识见:UFLDL教程. 实验内容:从10张512*512自然图像中随机选取10000个12*12的图像块(patch),然后对这些patch进行99%的方差保留的PCA计算,最后对这些patch做PCA Whitening和ZCA Whitening,并进行比较. 实验步骤及结果 1.加载图像数据,得到10000个图像块为原始数据x,它是144*10000的矩阵,随机显示200个图像块,其结果如下: 2.把它的每…

Main Menu Fortune.com       E-mail Tweet Facebook Linkedin Share icons By Roger Parloff Illustration by Justin Metz SEPTEMBER 28, 2016, 5:00 PM EDT WHY DEEP LEARNING IS SUDDENLY CHANGING YOUR LIFE Decades-old discoveries are now electrifying the comp…

Adit Deshpande CS Undergrad at UCLA ('19) Blog About Resume Deep Learning Research Review Week 1: Generative Adversarial Nets Starting this week, I’ll be doing a new series called Deep Learning Research Review. Every couple weeks or so, I’ll be summa…

Adit Deshpande CS Undergrad at UCLA ('19) Blog About The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3) Introduction Link to Part 1Link to Part 2 In this post, we’ll go into summarizing a lot of the new and important develo…

What's the most effective way to get started with deep learning?       29 Answers     Yoshua Bengio, My lab has been one of the three that started the deep learning approach, back in 2006, along with Hinton's... Answered Jan 20, 2016   Originally Ans…

Learning Deep Learning with Keras Piotr Migdał - blog Projects Articles Publications Resume About Photos Learning Deep Learning with Keras 30 Apr 2017 • Piotr Migdał • [machine-learning] [deep-learning] [overview] I teach deep learning both for a liv…

如何提高深度学习性能 20 Tips, Tricks and Techniques That You Can Use ToFight Overfitting and Get Better Generalization How can you get better performance from your deep learning model? It is one of the most common questions I get asked. It might be asked as: H…

In recent years, there’s been a resurgence in the field of Artificial Intelligence. It’s spread beyond the academic world with major players like Google, Microsoft, and Facebook creating their own research teams and making some impressive acquisition…

AlexNet / VGG-F network visualized by mNeuron. Project 6: Deep LearningIntroduction to Computer Vision Brief Due date: Tuesday, December 6th, 11:55pm Project materials including starter code, training and testing data, and html writeup template: proj…

In this lesson, Andrew Trask, the author of Grokking Deep Learning, will walk you through using neural networks for sentiment analysis. In particular, you'll build a network that classifies movie reviews as positive or negative just based on their te…

Awesome Deep Learning  Table of Contents Free Online Books Courses Videos and Lectures Papers Tutorials Researchers WebSites Datasets Frameworks Miscellaneous Contributing Free Online Books Deep Learning by Yoshua Bengio, Ian Goodfellow and Aaron Cou…

  Deep Learning Research Review Week 2: Reinforcement Learning 转载自: https://adeshpande3.github.io/adeshpande3.github.io/Deep-Learning-Research-Review-Week-2-Reinforcement-Learning This is the 2nd installment of a new series called Deep Learning Resea…

前言: 本文主要是bengio的deep learning tutorial教程主页中最后一个sample:rnn-rbm in polyphonic music. 即用RNN-RBM来model复调音乐,训练过程中采用的是midi格式的音频文件,接着用建好的model来产生复调音乐.对音乐建模的难点在与每首乐曲中帧间是高度时间相关的(这样样本的维度会很高),用普通的网络模型是不能搞定的(普通设计网络模型没有考虑时间维度,图模型中的HMM有这方面的能力),这种情况下可以采用RNN来处理,这里的R…

前言 训练神经网络模型时,如果训练样本较少,为了防止模型过拟合,Dropout可以作为一种trikc供选择.Dropout是hintion最近2年提出的,源于其文章Improving neural networks by preventing co-adaptation of feature detectors.中文大意为:通过阻止特征检测器的共同作用来提高神经网络的性能.本篇博文就是按照这篇论文简单介绍下Dropout的思想,以及从用一个简单的例子来说明该如何使用dropout. 基础知识:…

Deep Learning in a Nutshell: History and Training This series of blog posts aims to provide an intuitive and gentle introduction to deep learning that does not rely heavily on math or theoretical constructs. The first part in this series provided an…

前言 理论知识:UFLDL教程.Deep learning:二十六(Sparse coding简单理解).Deep learning:二十七(Sparse coding中关于矩阵的范数求导).Deep learning:二十九(Sparse coding练习) 实验环境:win7, matlab2015b,16G内存,2T机械硬盘 本节实验比较不好理解也不好做,我看很多人最后也没得出好的结果,所以得花时间仔细理解才行. 实验内容:Exercise:Sparse Coding.从10张512*51…

前言 实验内容:Exercise:Learning color features with Sparse Autoencoders.即:利用线性解码器,从100000张8*8的RGB图像块中提取颜色特征,这些特征会被用于下一节的练习 理论知识:线性解码器和http://www.cnblogs.com/tornadomeet/archive/2013/04/08/3007435.html 实验基础说明: 1.为什么要用线性解码器,而不用前面用过的栈式自编码器等?即:线性解码器的作用? 这一点,Ng…

1前言 本人写技术博客的目的,其实是感觉好多东西,很长一段时间不动就会忘记了,为了加深学习记忆以及方便以后可能忘记后能很快回忆起自己曾经学过的东西. 首先,在网上找了一些资料,看见介绍说UFLDL很不错,很适合从基础开始学习,Adrew Ng大牛写得一点都不装B,感觉非常好,另外对我们英语不好的人来说非常感谢,此教程的那些翻译者们!如余凯等.因为我先看了一些深度学习的文章,但是感觉理解得不够,一般要自己编程或者至少要看懂别人的程序才能理解深刻,所以我根据该教程的练习,一步一步做起,当然我也参考了…

HOME ABOUT CONTACT SUBSCRIBE VIA RSS   DEEP LEARNING FOR ENTERPRISE Distributed Deep Learning, Part 1: An Introduction to Distributed Training of Neural Networks Oct 3, 2016 3:00:00 AM / by Alex Black and Vyacheslav Kokorin Tweet inShare27   This pos…

Pedestrian Detection aided by Deep Learning Semantic Tasks CVPR 2015 本文考虑将语义任务(即:行人属性和场景属性)和行人检测相结合,以语义信息协助进行行人检测.先来看一下大致的检测结果(TA-CNN为本文检测结果): 可以看出,由于有了属性信息的协助,其行人检测的精确度有了较大的提升.具体网络架构如下图所示: 首先从各个数据集上进行行人数据集的收集和整理,即:从Caltech上收集行人正样本和负样本,然后从其他数据集上收集 ha…

Deep Learning: Assuming a deep neural network is properly regulated, can adding more layers actually make the performance degrade? I found this to be really puzzling. A deeper NN is supposed to be more powerful or at least equal to a shallower NN. I…

Understanding Convolution in Deep Learning Convolution is probably the most important concept in deep learning right now. It was convolution and convolutional nets that catapulted deep learning to the forefront of almost any machine learning task the…

The Brain vs Deep Learning Part I: Computational Complexity — Or Why the Singularity Is Nowhere Near July 27, 2015July 27, 2015 Tim Dettmers Deep Learning, NeuroscienceDeep Learning, dendritic spikes, high performance computing, neuroscience, singula…

Deep Learning and the Triumph of Empiricism By Zachary Chase Lipton, July 2015 Deep learning is now the standard-bearer for many tasks in supervised machine learning. It could also be argued that deep learning has yielded the most practically useful…