Understanding about numerical stability, convergence and consistency
In a computer simulation of the real world, physical quantities, which usually have continuous distributions governed by partial differential equations (PDEs), can be solved by numerical methods such as finite element method (FEM) and boundary element method (BEM). Whether the obtained solution is a good approximation of the reality and whether the numerical schemes can proceed properly under perturbations of different error sources, such as numerical quadrature error and round-off error, should be clarified before any code implementation. To answer these questions, this post will introduce the fundamental concepts of numerical stability, convergence and consistency according to the following figure.
Let \(u\) be the real solution of the following general variational problem for a PDE:
\[
\text{Solve $u \in U$: } a(u, v) = (f, v) \quad (\forall v \in W),
\]
where both \(U\) and \(W\) are Hilbert spaces, \(a(\cdot, \cdot): U \times W \rightarrow \mathbb{K}\) with \(\mathbb{K} \in \{\mathbb{R}, \mathbb{C}\}\) is a sesquilinear or bilinear form and \(f: W \rightarrow \mathbb{K}\) is a continuous linear functional on \(W\). The solution \(u\) belongs to the space \(U\) of continuous functions with infinite dimension. For ease of further analysis, priori assumption is usually adopted for such function space thus we have the assumed function space \(V\). For example, the countably normed spaces \(V = B_{\varrho}(\Gamma)\) used in the \(hp\)-BEM is defined as
\[
B_{\varrho}(\Gamma) = \{ v \in L^2(\Gamma): v \circ \kappa_K \in B_{\varrho}(K_0) \},
\]
where
- \(\Gamma\) is the boundary manifold of the solution domain, which is covered by the mesh \(\{K_i\}_{i=1}^{N_M}\) with \(N_M\) as the number of mesh elements;
- \(K_0\) is the reference cell and \(K\) is the real cell which may be curved;
- \(\kappa_K: K_0 \rightarrow K\) is the mapping from the reference cell to the real cell;
- \(B_{\varrho}(K_0)\) is the countably normed space restricted on the reference cell, which has constraints on the norm of all the derivatives of \(v\). Its formulation is given as below:
\[
B_{\varrho}(K_0) = \big\{ v \in L^2(K_0): \Norm{r_X^{k - \varrho} \left( \Pd{}{r_X} \right)^k \left( \vartheta (\alpha_X - \vartheta_X) \right)^{l - \varrho} \left( \Pd{}{\vartheta_X} \right)^l v}_{L^2(U_X)} \leq C d^{k+l+1} k! l!\big\},
\]
for which I do not provide more explanation in this post, but just give you an impression that the construction of the assumed solution function space can be quite complicated.
To solve the PDE on a computer, a finite dimensional subspace \(V^L\) of \(V\) must be constructed, in which a solution is to be sought as an approximation of the real solution by using some sort of numerical method. Then the stability condition means, for any function \(u\) in the real space \(U\) or the assumed space \(V\) of infinite dimension, whether there exists a function \(v\) in the finite dimensional space \(V^L\), such that the norm of their difference can be controlled to be arbitrarily small as \(N_L\), the dimension of space \(V^L\), increases. For example, in the \(hp\)-BEM, a subspace \(V^L\) can be constructed to have the following exponential stability condition:
\[
\begin{equation}
\label{eq:stability-condition}
\forall u \in B_{\varrho}(\Gamma): \inf_{v \in V^L} \norm{u - v}_{L^2(\Gamma)} \leq C \exp(-b N_L^{1/4}).
\end{equation}
\]
Once the solution \(u^L \in V^L\) for the finite dimensional problem is obtained from a general method such as the Galerkin method, i.e.
\[
\text{Solve $u^L \in V^L$: } a(u^L, v) = (f, v) \quad (\forall v \in V^L),
\]
the concept of convergence comes into play, which ensures that the difference between this \(u^L\) and the real solution \(u\) can be controlled. For example, if the following condition can be satisfied:
\[
\Norm{P_L A u^L} \geq C_s \Norm{u^L} \quad (\forall u^L \in V^L),
\]
where \(P_L: V \rightarrow V^L\) is the projection operator, \(A: V^L \rightarrow (V^L)'\) is the associated operator of the sesquilinear or bilinear form \(a(\cdot, \cdot)\) and \(C_s > 0\) is a constant, it can be proved that the solution obtained from the Galerkin method satisfies
\[
\begin{equation}
\label{eq:convergence-condition}
\norm{u - u^L} \leq C \inf_{v \in V^L} \Norm{u - v}.
\end{equation}
\]
This means the real solution can be properly approximated by the Galerkin solution with the error norm controlled by the approximation capability of the adopted finite dimensional space \(V^L\), and we say the method is convergent. In addition, combing equation \eqref{eq:stability-condition} and \eqref{eq:convergence-condition}, we know the solution has the exponential convergence property:
\[
\begin{equation}
\label{eq:exponential-convergence}
\norm{u - u^L} \leq C \exp(-b N_L^{1/4}).
\end{equation}
\]
Finally, we introduce the concept of consistency. During the discretization of the problem, the sesquilinear or bilinear form \(a(\cdot, \cdot)\), or rather, its associated operator \(A\), is to be approximated by its discrete version, i.e. the stiffness matrix \(A^L\). The evaluation of \(A^L\)'s coefficients usually needs numerical quadrature techniques, which introduces additional numerical error. Even though there is an analytical formula for integration, round-off error limited by the finite computer byte length is unavoidable. Hence, an operator \(\tilde{A}^L\) is obtained being different from \(A^L\). The error between \(A^L\) and \(\tilde{A}^L\) will perturb the adopted numerical method. If the error between the real and numerical solutions \(\Norm{u - \tilde{u}^L}\) can still be controlled, we say the method is consistent. For example, in the \(hp\)-BEM, if the stiffness matrix coefficient error satisfies the following consistent condition
\[
\abs{A^L_{ij} - \tilde{A}^L_{ij}} < \Phi(L) \quad (i,j = 1, \cdots, N_L)
\]
with
\[
\lim_{L \rightarrow \infty} N_L \Phi(L) = 0 \; \text{and} \; \Phi(L) = N_L^{-1} L \sigma^{\varrho L},
\]
the exponential convergence as shown in \eqref{eq:exponential-convergence} can be preserved.
Understanding about numerical stability, convergence and consistency的更多相关文章
- Softmax vs. Softmax-Loss VS cross-entropy损失函数 Numerical Stability(转载)
http://freemind.pluskid.org/machine-learning/softmax-vs-softmax-loss-numerical-stability/ 卷积神经网络系列之s ...
- Softmax vs. Softmax-Loss: Numerical Stability
http://freemind.pluskid.org/machine-learning/softmax-vs-softmax-loss-numerical-stability/ softmax 在 ...
- Understanding Convolution in Deep Learning
Understanding Convolution in Deep Learning Convolution is probably the most important concept in dee ...
- [C4] Andrew Ng - Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
About this Course This course will teach you the "magic" of getting deep learning to work ...
- 【转】Artificial Neurons and Single-Layer Neural Networks
原文:written by Sebastian Raschka on March 14, 2015 中文版译文:伯乐在线 - atmanic 翻译,toolate 校稿 This article of ...
- AP(affinity propagation)研究
待补充…… AP算法,即Affinity propagation,是Brendan J. Frey* 和Delbert Dueck于2007年在science上提出的一种算法(文章链接,维基百科) 现 ...
- 提高神经网络的学习方式Improving the way neural networks learn
When a golf player is first learning to play golf, they usually spend most of their time developing ...
- 【Caffe 测试】Training LeNet on MNIST with Caffe
Training LeNet on MNIST with Caffe We will assume that you have Caffe successfully compiled. If not, ...
- MR for Baum-Welch algorithm
The Baum-Welch algorithm is commonly used for training a Hidden Markov Model because of its superior ...
随机推荐
- OpenSIPS 1.11.1安装记录
说明:操作系统Centos 6.5 64位 安装步骤: 1.安装依赖包 : yum -y install gcc make gdb wget yum -y install flex bison nc ...
- git bash的命令
git bash cd /f 该命令可以把当前目录切换到f盘 git clone git上的项目的url
- 基于官方mysql镜像构建自己的mysql镜像
参考文章:https://www.jb51.net/article/115422.htm搭建步骤 1.首先创建Dckerfile: 1 2 3 4 5 6 7 8 9 10 11 12 FROM my ...
- mixins,generics(ApiView)
#生成序列化对象class BookModelSerizter(serializers.ModelSerializer): class Meta: model=Book fields='__all__ ...
- Android 设备的CPU类型(通常称为”ABIs”)
armeabiv-v7a: 第7代及以上的 ARM 处理器.2011年15月以后的生产的大部分Android设备都使用它. arm64-v8a: 第8代.64位ARM处理器,很少设备,三星 Galax ...
- ios中input获取焦点时的问题
1.获取焦点时,input会变大 解决办法是:font-size设置为32px以上 还有就是要在header里面加这一行代码:<meta name="viewport" co ...
- textarea的高度随内容变化而变化
<li class="text"> <span>参赛宣言*</span> <textarea name="txt" i ...
- iOS -- Effective Objective-C 阅读笔记 (2)
1: 多用类型常量, 少用 #define 预处理指令 #define 预处理指令会把碰到的所有 指定名称 一律换位 定义的内容, 这样的话, 假设此指令在某个头文件中, 那么所有引入这个头文件的代码 ...
- Swift 学习- 01 -- 基础部分
print("世界,你好") var myVariable = 42 myVariable = 50 let myConstant = 42 let implicitinteger ...
- Confluence 6 关于统一插件管理器
所有的组件通过 统一插件管理器(Universal Plugin Manager)进行管理,这个也被称为 UPM.UPM 可以在几乎所有的 Atlassian 应用中找到,能够提供完整同意的插件安装管 ...