====================================================================

This article came from here.

Thanks for zhizhihu.

====================================================================

Kernel Functions

Below is a list of some kernel functions available from the existing literature. As was the case with previous articles, every LaTeX notation for the formulas below are readily available from their alternate text html tag. I can not guarantee all of them are perfectly correct, thus use them at your own risk. Most of them have links to articles where they have been originally used or proposed.

1. Linear Kernel

The Linear kernel is the simplest kernel function. It is given by the inner product <x,y> plus an optional constant c. Kernel algorithms using a linear kernel are often equivalent to their non-kernel counterparts, i.e. KPCA with linear kernel is the same as standard PCA.

2. Polynomial Kernel

The Polynomial kernel is a non-stationary kernel. Polynomial kernels are well suited for problems where all the training data is normalized.


Adjustable parameters are the slope alpha, the constant term c and the polynomial degree d.

3. Gaussian Kernel

The Gaussian kernel is an example of radial basis function kernel.

Alternatively, it could also be implemented using

The adjustable parameter sigma plays a major role in the performance of the kernel, and should be carefully tuned to the problem at hand. If overestimated, the exponential will behave almost linearly and the higher-dimensional projection will start to lose its non-linear power. In the other hand, if underestimated, the function will lack regularization and the decision boundary will be highly sensitive to noise in training data.

4. Exponential Kernel

The exponential kernel is closely related to the Gaussian kernel, with only the square of the norm left out. It is also a radial basis function kernel.

5. Laplacian Kernel

The Laplace Kernel is completely equivalent to the exponential kernel, except for being less sensitive for changes in the sigma parameter. Being equivalent, it is also a radial basis function kernel.

It is important to note that the observations made about the sigma parameter for the Gaussian kernel also apply to the Exponential and Laplacian kernels.

6. ANOVA Kernel

The ANOVA kernel is also a radial basis function kernel, just as the Gaussian and Laplacian kernels. It is said to perform well in multidimensional regression problems (Hofmann, 2008).

7. Hyperbolic Tangent (Sigmoid) Kernel

The Hyperbolic Tangent Kernel is also known as the Sigmoid Kernel and as the Multilayer Perceptron (MLP) kernel. The Sigmoid Kernel comes from the Neural Networks field, where the bipolar sigmoid function is often used as an activation function for artificial neurons.

It is interesting to note that a SVM model using a sigmoid kernel function is equivalent to a two-layer, perceptron neural network. This kernel was quite popular for support vector machines due to its origin from neural network theory. Also, despite being only conditionally positive definite, it has been found to perform well in practice.

There are two adjustable parameters in the sigmoid kernel, the slope alpha and the intercept constant c. A common value for alpha is 1/N, where N is the data dimension. A more detailed study on sigmoid kernels can be found in the works by Hsuan-Tien and Chih-Jen.

8. Rational Quadratic Kernel

The Rational Quadratic kernel is less computationally intensive than the Gaussian kernel and can be used as an alternative when using the Gaussian becomes too expensive.

9. Multiquadric Kernel

The Multiquadric kernel can be used in the same situations as the Rational Quadratic kernel. As is the case with the Sigmoid kernel, it is also an example of an non-positive definite kernel.

10. Inverse Multiquadric Kernel

The Inverse Multi Quadric kernel. As with the Gaussian kernel, it results in a kernel matrix with full rank (Micchelli, 1986) and thus forms a infinite dimension feature space.

11. Circular Kernel

The circular kernel comes from a statistics perspective. It is an example of an isotropic stationary kernel and is positive definite in R2.


12. Spherical Kernel

The spherical kernel is similar to the circular kernel, but is positive definite in R3.

13. Wave Kernel

The Wave kernel is also symmetric positive semi-definite (Huang, 2008).

14. Power Kernel

The Power kernel is also known as the (unrectified) triangular kernel. It is an example of scale-invariant kernel (Sahbi and Fleuret, 2004) and is also only conditionally positive definite.

15. Log Kernel

The Log kernel seems to be particularly interesting for images, but is only conditionally positive definite.

16. Spline Kernel

The Spline kernel is given as a piece-wise cubic polynomial, as derived in the works by Gunn (1998).

However, what it actually mean is:

With

17. B-Spline (Radial Basis Function) Kernel

The B-Spline kernel is defined on the interval [−1, 1]. It is given by the recursive formula:

In the work by Bart Hamers it is given by:

Alternatively, Bn can be computed using the explicit expression (Fomel, 2000):

Where x+ is defined as the truncated power function:

18. Bessel Kernel

The Bessel kernel is well known in the theory of function spaces of fractional smoothness. It is given by:

where J is the Bessel function of first kind. However, in the Kernlab for R documentation, the Bessel kernel is said to be:

19. Cauchy Kernel

The Cauchy kernel comes from the Cauchy distribution (Basak, 2008). It is a long-tailed kernel and can be used to give long-range influence and sensitivity over the high dimension space.

20. Chi-Square Kernel

The Chi-Square kernel comes from the Chi-Square distribution.

21. Histogram Intersection Kernel

The Histogram Intersection Kernel is also known as the Min Kernel and has been proven useful in image classification.

22. Generalized Histogram Intersection

The Generalized Histogram Intersection kernel is built based on the Histogram Intersection Kernelfor image classification but applies in a much larger variety of contexts (Boughorbel, 2005). It is given by:

23. Generalized T-Student Kernel

The Generalized T-Student Kernel has been proven to be a Mercel Kernel, thus having a positive semi-definite Kernel matrix (Boughorbel, 2004). It is given by:

24. Bayesian Kernel

The Bayesian kernel could be given as:

where

However, it really depends on the problem being modeled. For more information, please see the work by Alashwal, Deris and Othman, in which they used a SVM with Bayesian kernels in the prediction of protein-protein interactions.

25. Wavelet Kernel

The Wavelet kernel (Zhang et al, 2004) comes from Wavelet theory and is given as:

Where a and c are the wavelet dilation and translation coefficients, respectively (the form presented above is a simplification, please see the original paper for details). A translation-invariant version of this kernel can be given as:

Where in both h(x) denotes a mother wavelet function. In the paper by Li Zhang, Weida Zhou, and Licheng Jiao, the authors suggests a possible h(x) as:

Which they also prove as an admissible kernel function.

SVM Kernel Functions的更多相关文章

  1. HDU 5095 Linearization of the kernel functions in SVM(模拟)

    主题链接:http://acm.hdu.edu.cn/showproblem.php? pid=5095 Problem Description SVM(Support Vector Machine) ...

  2. HDU 5095--Linearization of the kernel functions in SVM【模拟】

    Linearization of the kernel functions in SVM Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: ...

  3. Linearization of the kernel functions in SVM(多项式模拟)

    Description SVM(Support Vector Machine)is an important classification tool, which has a wide range o ...

  4. 模拟 HDOJ 5095 Linearization of the kernel functions in SVM

    题目传送门 /* 题意:表达式转换 模拟:题目不难,也好理解题意,就是有坑!具体的看测试样例... */ #include <cstdio> #include <algorithm& ...

  5. Kernel Functions for Machine Learning Applications

    In recent years, Kernel methods have received major attention, particularly due to the increased pop ...

  6. Kernel Functions-Introduction to SVM Kernel & Examples - DataFlair

    Kernel Functions-Introduction to SVM Kernel & Examples - DataFlairhttps://data-flair.training/bl ...

  7. HDU 5095 Linearization of the kernel functions in SVM (坑水)

    比较坑的水题,首项前面的符号,-1,+1,只有数字项的时候要输出0. 感受一下这些数据 160 0 0 0 0 0 0 0 0 -10 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 ...

  8. hdu 5095 Linearization of the kernel functions in SVM(模拟,分类清楚就行)

    题意: INPUT: The input of the first line is an integer T, which is the number of test data (T<120). ...

  9. ML- 核函数(Kernel) 的 SVM

    Why 核函数 目的是为了解决线性不可分问题. 核心思想是升维. 当样本点在低维空间不能很好地分开的时候, 可以考虑将样本通过某种映射(就是左乘一个矩阵) 到高维空间中, 然后在高维空间就容易求解一个 ...

随机推荐

  1. PAT甲题题解-1052. Linked List Sorting (25)-排序

    三个注意点: 1.给出的n个节点并不一定都在链表中 2.最后一组样例首地址即为-1 3.输出地址的时候一直忘记前面要补0... #include <iostream> #include & ...

  2. 1083. List Grades (25)-简单的排序

    给定区间[L,R],给出在这区间之内的学生,并且按照他们的成绩非升序的顺序输出. #include <iostream> #include <cstdio> #include ...

  3. Linux内核分析——第二周学习笔记20135308

    第二周 操作系统是如何工作的 第一节 函数调用堆栈 存储程序计算机:是所有计算机基础的框架 堆栈:计算机中基础的部分,在计算机只有机器语言.汇编语言时,就有了堆栈.堆栈机制是高级语言可以运行的基础. ...

  4. 谈vs2013单元测试感想

    (1)安装篇:这个就不用多说啦,百度一个安装包进行安装. 之前下载过vs2013当时是抱着玩玩的心态,也没有安装成功,现在作为作业重新安装,并且进行单元测试.下面就是安装vs2013的具体过程以及我遇 ...

  5. ElasticSearch 2 (2) - Setup

    ElasticSearch 2.1.1 (2) - Setup Installation Elasticsearch can be started using: $ bin/elasticsearc ...

  6. 『编程题全队』alpha阶段项目复审

    小组的名字和链接 优点 缺点,bug 报告 最终名次 Gakki赛高 (1)支持注册账号和账号管理(2) 支持自动登录,提供便捷性(3)题目不重复且题目答案准确(4)支持排行榜统计功能(5)自己设计算 ...

  7. Jquery ajax 完整实例子1

    $ajax请求--------------------------------- var $personWifePs=$("#wife-money tbody tr"); var ...

  8. BeanFactory 简介以及它 和FactoryBean的区别(阿里面试)

    BeanFacotry是spring中比较原始的Factory.如XMLBeanFactory就是一种典型的BeanFactory.原始的BeanFactory无法支持spring的许多插件,如AOP ...

  9. Spring之使用表达式配置切入点

    使用表达式配置切入点

  10. Several ports (8005, 8080, 8009) required by Tomcat

    转载:http://blog.csdn.net/tomoto_zh/article/details/51931945 先找到Java项目中  Servers找到Server.xml然后 把8005, ...