machine learning学习笔记

看到Max Welling教授主页上有不少学习notes，收藏一下吧，其最近出版了一本书呢还，还没看过。

http://www.ics.uci.edu/~welling/classnotes/classnotes.html

Statistical Estimation [ps]
- bayesian estimation
- maximum a posteriori (MAP) estimation
- maximum likelihood (ML) estimation
- Bias/Variance tradeoff & minimum description length (MDL)

Expectation Maximization (EM) Algorithm [ps]
- detailed derivation plus some examples

Supervised Learning (Function Approximation) [ps]
- mixture of experts (MoE)
- cluster weighted modeling (CWM)

Clustering [ps]
- mixture of gaussians (MoG)
- vector quantization (VQ) with k-means.

Linear Models [ps]
- factor analysis (FA)
- probabilistic principal component analysis (PPCA)
- principal component analysis (PCA)

Independent Component Analysis (ICA) [ps]
- noiseless ICA
- noisy ICA
- variational ICA

Mixture of Factor Analysers (MoFA) [ps]
- derivation of learning algorithm

Hidden Markov Models (HMM) [ps]
- viterbi decoding algorithm
- Baum-Welch learning algorithm

Kalman Filters (KF) [ps]
- kalman filter algorithm (very detailed derivation)
- kalman smoother algorithm (very detailed derivation)

Approximate Inference Algorithms [ps]
- variational EM
- laplace approximation
- importance sampling
- rejection sampling
- markov chain monte carlo (MCMC) sampling
- gibbs sampling
- hybrid monte carlo sampling (HMC)

Belief Propagation (BP) [ps]
- Introduction to BP and GBP: powerpoint presentation [ppt]
- converting directed acyclic graphical models (DAG) into junction trees (JT)
- Shafer-Shenoy belief propagation on junction trees
- some examples

Boltzmann Machine (BM) [ps]
- derivation of learning algorithm

Generative Topographic Mapping (GTM) [ps]
- derivation of learning algorithm

Introduction to Kernel Methods: powerpoint presentation [ppt]

Kernel Principal Components Analysis [pdf]

Kernel Canonical Correlation Analysis [pdf]

Kernel Support Vector Machines [pdf]

Kernel Ridge-Regression [pdf]

Kernel Support Vector Regression [pdf]

Convex Optimization [pdf]
A brief introduction based on Stephan Boyd’s book, chapter 5.

Fisher Linear Discriminant Analysis [pdf]