Mahout 系列之----共轭梯度
无预处理共轭梯度
要求解线性方程组 ,稳定双共轭梯度法从初始解
开始按以下步骤迭代:
- 任意选择向量
使得
,例如,
- 对
- 若
足够精确则退出
预处理共轭梯度
预处理通常被用来加速迭代方法的收敛。要使用预处理子 来求解线性方程组
,预处理稳定双共轭梯度法从初始解
开始按以下步骤迭代:
- 任意选择向量
使得
,例如,
- 对
- 若
足够精确则退出
这个形式等价于将无预处理的稳定双共轭梯度法应用于显式预处理后的方程组
,
其中 ,
,
。换句话说,左预处理和右预处理都可以通过这个形式实施。
Mahout 分布式共轭梯度实现:
package org.apache.mahout.math.solver;
import org.apache.mahout.math.CardinalityException;
import org.apache.mahout.math.DenseVector;
import org.apache.mahout.math.Vector;
import org.apache.mahout.math.VectorIterable;
import org.apache.mahout.math.function.Functions;
import org.apache.mahout.math.function.PlusMult;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* <p>Implementation of a conjugate gradient iterative solver for linear systems. Implements both
* standard conjugate gradient and pre-conditioned conjugate gradient.
*
* <p>Conjugate gradient requires the matrix A in the linear system Ax = b to be symmetric and positive
* definite. For convenience, this implementation allows the input matrix to be be non-symmetric, in
* which case the system A'Ax = b is solved. Because this requires only one pass through the matrix A, it
* is faster than explictly computing A'A, then passing the results to the solver.
*
* <p>For inputs that may be ill conditioned (often the case for highly sparse input), this solver
* also accepts a parameter, lambda, which adds a scaled identity to the matrix A, solving the system
* (A + lambda*I)x = b. This obviously changes the solution, but it will guarantee solvability. The
* ridge regression approach to linear regression is a common use of this feature.
*
* <p>If only an approximate solution is required, the maximum number of iterations or the error threshold
* may be specified to end the algorithm early at the expense of accuracy. When the matrix A is ill conditioned,
* it may sometimes be necessary to increase the maximum number of iterations above the default of A.numCols()
* due to numerical issues.
*
* <p>By default the solver will run a.numCols() iterations or until the residual falls below 1E-9.
*
* <p>For more information on the conjugate gradient algorithm, see Golub & van Loan, "Matrix Computations",
* sections 10.2 and 10.3 or the <a href="http://en.wikipedia.org/wiki/Conjugate_gradient">conjugate gradient
* wikipedia article</a>.
*/
public class ConjugateGradientSolver {
public static final double DEFAULT_MAX_ERROR = 1.0e-9;
private static final Logger log = LoggerFactory.getLogger(ConjugateGradientSolver.class);
private static final PlusMult PLUS_MULT = new PlusMult(1.0);
private int iterations;
private double residualNormSquared;
public ConjugateGradientSolver() {
this.iterations = 0;
this.residualNormSquared = Double.NaN;
}
/**
* Solves the system Ax = b with default termination criteria. A must be symmetric, square, and positive definite.
* Only the squareness of a is checked, since testing for symmetry and positive definiteness are too expensive. If
* an invalid matrix is specified, then the algorithm may not yield a valid result.
*
* @param a The linear operator A.
* @param b The vector b.
* @return The result x of solving the system.
* @throws IllegalArgumentException if a is not square or if the size of b is not equal to the number of columns of a.
*
*/
public Vector solve(VectorIterable a, Vector b) {
return solve(a, b, null, b.size(), DEFAULT_MAX_ERROR);
}
/**
* Solves the system Ax = b with default termination criteria using the specified preconditioner. A must be
* symmetric, square, and positive definite. Only the squareness of a is checked, since testing for symmetry
* and positive definiteness are too expensive. If an invalid matrix is specified, then the algorithm may not
* yield a valid result.
*
* @param a The linear operator A.
* @param b The vector b.
* @param precond A preconditioner to use on A during the solution process.
* @return The result x of solving the system.
* @throws IllegalArgumentException if a is not square or if the size of b is not equal to the number of columns of a.
*
*/
public Vector solve(VectorIterable a, Vector b, Preconditioner precond) {
return solve(a, b, precond, b.size(), DEFAULT_MAX_ERROR);
}
/**
* Solves the system Ax = b, where A is a linear operator and b is a vector. Uses the specified preconditioner
* to improve numeric stability and possibly speed convergence. This version of solve() allows control over the
* termination and iteration parameters.
*
* @param a The matrix A.
* @param b The vector b.
* @param preconditioner The preconditioner to apply.
* @param maxIterations The maximum number of iterations to run.
* @param maxError The maximum amount of residual error to tolerate. The algorithm will run until the residual falls
* below this value or until maxIterations are completed.
* @return The result x of solving the system.
* @throws IllegalArgumentException if the matrix is not square, if the size of b is not equal to the number of
* columns of A, if maxError is less than zero, or if maxIterations is not positive.
*/
// 共轭梯度实现的主题部分。很明显该方法是既可以用预处理的方式,也可以不用预处理的方式。Mahout中提供了单机模式的雅克比预处理,但是没有提供分布式处理的雅克比预处理,这个需要自己写。很简单,只要将对角线元素去倒数,组成一个对角阵即可。
public Vector solve(VectorIterable a,
Vector b,
Preconditioner preconditioner,
int maxIterations,
double maxError) {
if (a.numRows() != a.numCols()) {
throw new IllegalArgumentException("Matrix must be square, symmetric and positive definite.");
}
if (a.numCols() != b.size()) {
throw new CardinalityException(a.numCols(), b.size());
}
if (maxIterations <= 0) {
throw new IllegalArgumentException("Max iterations must be positive.");
}
if (maxError < 0.0) {
throw new IllegalArgumentException("Max error must be non-negative.");
}
Vector x = new DenseVector(b.size());
iterations = 0;
Vector residual = b.minus(a.times(x));
residualNormSquared = residual.dot(residual);
log.info("Conjugate gradient initial residual norm = {}", Math.sqrt(residualNormSquared));
double previousConditionedNormSqr = 0.0;
Vector updateDirection = null;
while (Math.sqrt(residualNormSquared) > maxError && iterations < maxIterations) {
Vector conditionedResidual;
double conditionedNormSqr;
if (preconditioner == null) {
conditionedResidual = residual;
conditionedNormSqr = residualNormSquared;
} else {
conditionedResidual = preconditioner.precondition(residual);
conditionedNormSqr = residual.dot(conditionedResidual);
}
++iterations;
if (iterations == 1) {
updateDirection = new DenseVector(conditionedResidual);
} else {
double beta = conditionedNormSqr / previousConditionedNormSqr;
// updateDirection = residual + beta * updateDirection
updateDirection.assign(Functions.MULT, beta);
updateDirection.assign(conditionedResidual, Functions.PLUS);
}
Vector aTimesUpdate = a.times(updateDirection);
double alpha = conditionedNormSqr / updateDirection.dot(aTimesUpdate);
// x = x + alpha * updateDirection
PLUS_MULT.setMultiplicator(alpha);
x.assign(updateDirection, PLUS_MULT);
// residual = residual - alpha * A * updateDirection
PLUS_MULT.setMultiplicator(-alpha);
residual.assign(aTimesUpdate, PLUS_MULT);
previousConditionedNormSqr = conditionedNormSqr;
residualNormSquared = residual.dot(residual);
log.info("Conjugate gradient iteration {} residual norm = {}", iterations, Math.sqrt(residualNormSquared));
}
return x;
}
/**
* Returns the number of iterations run once the solver is complete.
*
* @return The number of iterations run.
*/
public int getIterations() {
return iterations;
}
/**
* Returns the norm of the residual at the completion of the solver. Usually this should be close to zero except in
* the case of a non positive definite matrix A, which results in an unsolvable system, or for ill conditioned A, in
* which case more iterations than the default may be needed.
*
* @return The norm of the residual in the solution.
*/
public double getResidualNorm() {
return Math.sqrt(residualNormSquared);
}
}
DistributedConjugateGradientSolver 是上CG的扩展,DCG和CG的区别在于,DCG矩阵和向量相乘时需要MR实现矩阵相乘。
Mahout 系列之----共轭梯度的更多相关文章
- Mahout系列之----共轭梯度预处理
对于大型矩阵,预处理是很重要的.常用的预处理方法有: (1) 雅克比预处理 (2)块状雅克比预处理 (3)半LU 分解 (4)超松弛法
- 共轭梯度算法求最小值-scipy
# coding=utf-8 #共轭梯度算法求最小值 import numpy as np from scipy import optimize def f(x, *args): u, v = x a ...
- 机器学习: 共轭梯度算法(PCG)
今天介绍数值计算和优化方法中非常有效的一种数值解法,共轭梯度法.我们知道,在解大型线性方程组的时候,很少会有一步到位的精确解析解,一般都需要通过迭代来进行逼近,而 PCG 就是这样一种迭代逼近算法. ...
- Mahout系列之----kmeans 聚类
Kmeans是最经典的聚类算法之一,它的优美简单.快速高效被广泛使用. Kmeans算法描述 输入:簇的数目k:包含n个对象的数据集D. 输出:k个簇的集合. 方法: 从D中任意选择k个对象作为初始簇 ...
- Mahout 系列之--canopy 算法
Canopy 算法,流程简单,容易实现,一下是算法 (1)设样本集合为S,确定两个阈值t1和t2,且t1>t2. (2)任取一个样本点p属于S,作为一个Canopy,记为C,从S中移除p. (3 ...
- Mahout系列之-----相似度
Mahout推荐系统中有许多相似度实现,这些组件实现了计算不能User之间或Item之间的相似度.对于数据量以及数据类型不同的数据源,需要不同的相似度计算方法来提高推荐性能,在mahout提供了大量用 ...
- Mahout系列之----距离度量
x = (x1,...,xn) 和y = (y1,...,yn) 之间的距离为 (1)欧氏距离 EuclideanDistanceMeasure (2)曼哈顿距离 ManhattanDis ...
- mahout系列----Dirichlet 分布
Dirichlet分布可以看做是分布之上的分布.如何理解这句话,我们可以先举个例子:假设我们有一个骰子,其有六面,分别为{1,2,3,4,5,6}.现在我们做了10000次投掷的实验,得到的实验结果是 ...
- mahout系列之---谱聚类
1.构造亲和矩阵W 2.构造度矩阵D 3.拉普拉斯矩阵L 4.计算L矩阵的第二小特征值(谱)对应的特征向量Fiedler 向量 5.以Fiedler向量作为kmean聚类的初始中心,用kmeans聚类 ...
随机推荐
- Docker创建 tomcat/weblogic 集群
安装 tomcat 镜像 准备好需要的 jdk.tomcat 等软件放到 home 目录下面,启动一个容器 docker run -t -i -v /home:/opt/data --name mk_ ...
- 阿里巴巴Java开发规约插件
就在今天 10月14日上午9:00 阿里巴巴于在杭州云栖大会<研发效能峰会>上,正式发布<阿里巴巴Java开发手册>扫描插件,该插件在扫描代码后,将不符合<手册>的 ...
- NDK编程的一个坑—Arm平台下的类型转换
最近在做DNN定点化相关的工作,DNN定点化就是把float表示的模型压缩成char表示,虽然会损失精度,但是由于DNN训练的模型值比较接近且范围较小,实际上带来的性能损失非常小.DNN定点化的好处是 ...
- 自定义View总结2
自定义控件: 1.组合控件:将系统原生控件组合起来,加上动画效果,形成一种特殊的UI效果 2.纯粹自定义控件:继承自系统的View,自己去实现view效果 优酷菜单: 1.系统原生的旋转和位置动画并没 ...
- proc文件系统探索 之 根目录下的文件[二]
包括对proc根目录下stat,uptime,swaps三个文件的解析. /proc/stat 文件包含了系统启动后的一些系统统计信息. Cat /proc/stat: cpu 77781 1077 ...
- CentOS7下安装GitLab
三步在CentOS7系统下,完成GitLab的安装. 1.安装和配置必须的依赖 sudo yum install curl policycoreutils openssh-server openssh ...
- [boost] Windows下编译
编译命令 32位 编译 bjam variant=release link=static threading=multi runtime-link=static -a -q bjam variant= ...
- Leetcode解题-树(5.0.0)基础类
与第二章类似,LeetCode指定了TreeNode实现.为了方便后续习题的编写和测试,创建一个基础父类,包含TreeNode实现,以及create()和print()创建和打印树的方法.其中crea ...
- 2014 BDTC 参会有感
中国大数据技术大会(Big Data Technology Conference,BDTC)是目前国内最具影响.规模最大的大数据领域的技术盛会.大会的前身是Hadoop中国云计算大会(Hadoop i ...
- 看见的力量 – (II) 影响地图
本文转自台湾的李智桦老师的博客,原文地址 Impact Mapping 真是令人惊艳的可视化工具.等你看完这篇文章,你会爱上它的. 典故 继2011年6月Example of specificatio ...