原文地址:http://www.jianshu.com/p/9bf9e2add795 AdaBoost 问题描述 程序实现 # coding:utf-8 import math import numpy as np import matplotlib.pyplot as plt def ReadData(dataFile): with open(dataFile, 'r') as f: lines = f.readlines() data_list = [] for line in lines:…

从这一节开始学习机器学习技法课程中的SVM, 这一节主要介绍标准形式的SVM: Linear SVM 引入SVM 首先回顾Percentron Learning Algrithm(感知器算法PLA)是如何分类的,如下图,找到一条线,将两类训练数据点分开即可: PLA的最后的直线可能有很多条,那到底哪条好呢?好坏的标准则是其泛化性能,即在测试数据集上的正确率,如下,下面三条直线都能正确的分开训练数据,那到底哪个好呢?SVM就是解决这个问题的. SVM求解 直觉告诉我们最右的要好一些,因为测试数据的…

Roadmap Kernel Ridge Regression Support Vector Regression Primal Support Vector Regression Dual Summary of Kernel Models Map of Linear Models Map of Kernel Models possible kernels: polynomial, Gaussian,..., your design (with Mercer’s condition), coup…

前一篇,我们将SVM与logistic regression联系起来,这一次我们将SVM与ridge regression(之前的linear regression)联系起来. (一)kernel ridge regression 之前我们之前在做的是linear regression,现在我们希望在regression中使用kernel trick. 下面是linear versus kernel: 至此,kernel ridge regression结束.但是,这里的β与kernel log…

作者:桂. 时间:2017-05-23  15:52:51 链接:http://www.cnblogs.com/xingshansi/p/6895710.html 一.理论描述 Kernel ridge regression (KRR)是对Ridge regression的扩展,看一下Ridge回归的准则函数: 求解 一些文章利用矩阵求逆,其实求逆只是表达方便,也可以直接计算.看一下KRR的理论推导,注意到 左乘,并右乘,得到 利用Ridge回归中的最优解 对于xxT的形式可以利用kernel的…

Roadmap Soft-Margin SVM as Regularized Model SVM versus Logistic Regression SVM for Soft Binary Classification Kernel Logistic Regression Summary…

Roadmap Kernel Trick Polynomial Kernel Gaussian Kernel Comparison of Kernels Summary…

原文地址:https://www.jianshu.com/p/6bf801bdc644 特征变换 问题描述 程序实现 # coding: utf-8 import numpy as np from cvxopt import matrix, solvers from sklearn import svm def gen_data(): X = [[1, 0], [0, 1], [0, -1], [-1, 0], [0, 2], [0, -2], [-2, 0]] X = np.array(X)…

原文地址:https://www.jianshu.com/p/1db700f866ee 问题描述 程序实现 # kNN_RBFN.py # coding:utf-8 import numpy as np import matplotlib.pyplot as plt def ReadData(dataFile): with open(dataFile, 'r') as f: lines = f.readlines() data_list = [] for line in lines: line…

原文地址:https://www.jianshu.com/p/7ff6fd6fc99f 问题描述 程序实现 13-15 # coding:utf-8 # decision_tree.py import numpy as np def ReadData(dataFile): with open(dataFile, 'r') as f: lines = f.readlines() data_list = [] for line in lines: line = line.strip().split(…

Roadmap Adaptive Boosted Decision Tree Optimization View of AdaBoost Gradient Boosting Summary of Aggregation Models Summary…

Roadmap Motivation of Boosting Diversity by Re-weighting Adaptive Boosting Algorithm Adaptive Boosting in Action Summary…

Roadmap Linear Network Hypothesis Basic Matrix Factorization Stochastic Gradient Descent Summary of Extraction Models Summary…

Roadmap Feature Exploitation Techniques Error Optimization Techniques Overfitting Elimination Techniques Machine Learning in Practice Summary…

Roadmap Motivation Neural Network Hypothesis Neural Network Learning Optimization and Regularization Summary…

Roadmap RBF Network Hypothesis RBF Network Learning k-Means Algorithm k-Means and RBF Network in Action Summary…

Roadmap Decision Tree Hypothesis Decision Tree Algorithm Decision Tree Heuristics in C&RT Decision Tree in Action Summary…

Roadmap Random Forest Algorithm Out-Of-Bag Estimate Feature Selection Random Forest in Action Summary…

Roadmap Motivation of Aggregation Uniform Blending Linear and Any Blending Bagging (Bootstrap Aggregation) Summary…

Roadmap Motivation and Primal Problem Dual Problem Messages behind Soft-Margin SVM Model Selection Summary…

Roadmap Motivation of Dual SVM Lagrange Dual SVM Solving Dual SVM Messages behind Dual SVM Summary…

Roadmap Deep Neural Network Autoencoder Denoising Autoencoder Principal Component Analysis Summary…

Roadmap Course Introduction Large-Margin Separating Hyperplane Standard Large-Margin Problem Support Vector Machine Reasons behind Large-Margin Hyperplane Summary…

原文地址:https://www.jianshu.com/p/58259cdde0e1 Roadmap Motivation of Dual SVM Lagrange Dual SVM Solving Dual SVM Messages behind Dual SVM Summary…

回顾一下岭回归,岭回归的目的是学习得到特征和因变量之间的映射关系,由于特征可能很高维,所以需要正则化 岭回归的目标函数是 $$ \sum_{i=1}^n \left\|y-X\beta\right\|^2+\lambda\beta^T\beta $$ 由于数据可能是非线性的,单纯的线性回归效果可能不是很好,因此可以把数据映射到一个核空间,使得数据在这个核空间里面线性可分. 设核函数为$\Phi_i=\Phi(x_i)$,$\Phi_i$是一个$d$维空间中的向量,通常$d$比原来的维数高,甚至可…

Ridge Regression and Ridge Regression Kernel Reference: 1. scikit-learn linear_model ridge regression 2. Machine learning for quantum mechanics in a nutshell Authors 3. sample plot ridge path code from #Fabian Pedregosa -- Ridge regression Ridge regr…

Roadmap Kernel Ridge Regression Support Vector Regression Primal Support Vector Regression Dual Summary of Kernel Models Map of Linear Models Map of Kernel Models possible kernels: polynomial, Gaussian, : : :, your design (with Mercer's condition), c…

这节课主要讲述了RBF这类的神经网络+Kmeans聚类算法,以及二者的结合使用. 首先回归的了Gaussian SVM这个模型: 其中的Gaussian kernel又叫做Radial Basis Function kernel 1)radial:表示输入点与center点的距离 2)basis function:表示‘combined’ 从这个角度来看,Gaussian Kernel SVM可以看成许多小的radial hypotheses的线性组合(前面的系数就是SV的alphan和yn)…

上节课讲了Kernel的技巧如何应用到Logistic Regression中.核心是L2 regularized的error形式的linear model是可以应用Kernel技巧的. 这一节,继续沿用representer theorem,延伸到一般的regression问题. 首先想到的就是ridge regression,它的cost函数本身就是符合representer theorem的形式. 由于optimal solution一定可以表示成输入数据的线性组合,再配合Kernel T…

前言 最近在看Peter Harrington写的"机器学习实战",这是我的学习笔记,这次是第7章 - 利用AdaBoost元算法提高分类性能. 核心思想 在使用某个特定的算法是,有时会发现生成的算法\(f(x)\)的错误率比较高,只使用这个算法达不到要求. 这时\(f(x)\)就是一个弱算法. 在以前学习算法的过程中,我们认识到算法的参数很重要,所以把公式改写成这样: \[ f(x,arguments) \\ where \\ \qquad x \text{ : calculated…