Coursera, Deep Learning 5, Sequence Models, week4, Transformer Network

【Coursera, Deep Learning 5, Sequence Models, week4, Transformer Network】的更多相关文章

Coursera, Deep Learning 5, Sequence Models, week3, Sequence models & Attention mechanism

Sequence to Sequence models basic sequence-to-sequence model: basic image-to-sequence or called image captioning model: but there are some differences between how you write a model like this to generate a sequence, compared to how you were synthesizi…

Coursera, Deep Learning 5, Sequence Models, week1 Recurrent Neural Networks

有哪些sequence model Notation: RNN - Recurrent Neural Network 传统NN 在解决sequence input 时有什么问题? RNN就没有上面的问题. 注意这里还提到了BRNN 双向RNN的概念. 激活函数 g1 经常用的是tanh, 也有用relu的但是不常用 Backpropagation through time Difference types of RNNs Language model and sequence generatio…

Coursera, Deep Learning 5, Sequence Models, week2, Natural Language Processing & Word Embeddings

Word embeding 给word 加feature,用来区分word 之间的不同,或者识别word之间的相似性. 用于学习 Embeding matrix E 的数据集非常大,比如 1B - 100B 的word corpos. 所以即使你输入的是没见过的 durian cutivator 也知道和 orange farmer 很相近. 这是transfter learning 的一个case. 因为t-SNE 做了non-liner 的转化,所以在原来的300维空间的平行的向量在转化过后…

Predicting effects of noncoding variants with deep learning–based sequence model | 基于深度学习的序列模型预测非编码区变异的影响

Predicting effects of noncoding variants with deep learning–based sequence model PDF Interpreting noncoding variants- 非常好的学习资料这篇文章的第一个亮点就是直接从序列开始分析,第二就是使用深度学习获得了很好的预测效果. This is, to our knowledge, the first approach for prioritization of functional…

Coursera, Deep Learning 4, Convolutional Neural Networks - week4,

Face recognition One Shot Learning 只看一次图片,就能以后识别, 传统deep learning 很难做到这个. 而且如果要加一个人到数据库里面,就要重新train model 显然不合理,所以就引出了 One Shot Learning 的概念. 怎么得出这个similarity function d(img1, img2) 呢?下面的介绍的 Siamese network.可以实现这个目标. 怎么定义object function 来满足上面的的条件呢?可以…

Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1, Assignment(Regularization)

声明:所有内容来自coursera,作为个人学习笔记记录在这里. Regularization Welcome to the second assignment of this week. Deep Learning models have so much flexibility and capacity that overfitting can be a serious problem, if the training dataset is not big enough. Sure it do…

Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week2, Optimization algorithms

Gradient descent Batch Gradient Decent, Mini-batch gradient descent, Stochastic gradient descent 还有很多比gradient decent 更优化的算法,在了解这些算法前,需要先理解 Exponentially weighted averages 这个概念 Exponentially weighted average 是一种计算平均值的方法,非常省storage 和 memory, 但是不是很精确.…

Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1, Assignment(Gradient Checking)

声明:所有内容来自coursera,作为个人学习笔记记录在这里. Gradient Checking Welcome to the final assignment for this week! In this assignment you will learn to implement and use gradient checking. You are part of a team working to make mobile payments available globally, and…

Coursera, Deep Learning 1, Neural Networks and Deep Learning - week1, Introduction to deep learning

整个deep learing 系列课程主要包括哪些内容 Intro to Deep learning…

Coursera, Deep Learning 4, Convolutional Neural Networks - week1

CNN 主要解决 computer vision 问题,同时解决input X 维度太大的问题. Edge detection 下面演示了convolution 的概念下图的 vertical edge 看起来有点厚,但是如果图片远比6x6像素大的话,就会看到效果非常不错. 除了前面讲过的第一种filter, 还有两种 (Sobel filter, Scharr filter) 接下来会讲到 CNN 的两个重要的buiding block - padding, strided convolut…