在CentOS上安装Singularity高性能容器
什么是singularity容器
Singularity是劳伦斯伯克利国家实验室专门为大规模、跨节点HPC和DL工作负载而开发的容器化技术。具备轻量级、快速部署、方便迁移等诸多优势，且支持从Docker镜像格式转换为Singularity镜像格式。除了兼容docker的镜像之外，singularity还有一个不太明显的优势：可以通过非root帐号来拉起容器，这样对于某些安全性要求比较高的场景来说还是有用途的。
环境准备
目前singularity支持了redhat系列Linux发行版的二进制安装，这里我们使用的基础系统是CentOS：
[root@centos /]# cat /etc/redhat-release
CentOS Linux release 8.3.2011
安装配置epel扩展源
首先我们需要更新所有的系统软件：
[root@centos /]# yum update -y
CentOS Linux 8 - AppStream 1.1 MB/s | 6.3 MB 00:05
CentOS Linux 8 - BaseOS 1.7 MB/s | 2.3 MB 00:01
Dependencies resolved.
Nothing to do.
Complete!
安装epel扩展源：
[root@centos /]# yum install -y epel-release
Last metadata expiration check: 0:00:22 ago on Wed 13 Jan 2021 09:24:36 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
Package Architecture Version Repository Size
============================================================================================================================================================================================================================================
Installing:
epel-release noarch 8-8.el8 extras 23 k
Transaction Summary
============================================================================================================================================================================================================================================
Install 1 Package
Total download size: 23 k
Installed size: 32 k
Downloading Packages:
epel-release-8-8.el8.noarch.rpm 140 kB/s | 23 kB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 34 kB/s | 23 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : epel-release-8-8.el8.noarch 1/1
Running scriptlet: epel-release-8-8.el8.noarch 1/1
Verifying : epel-release-8-8.el8.noarch 1/1
Installed:
epel-release-8-8.el8.noarch
Complete!
安装好epel源之后，需要再次更新系统软件：
[root@centos /]# yum update -y
Extra Packages for Enterprise Linux Modular 8 - x86_64 9.2 kB/s | 527 kB 00:57
Extra Packages for Enterprise Linux 8 - x86_64 10 kB/s | 8.8 MB 14:46
Last metadata expiration check: 0:00:19 ago on Wed 13 Jan 2021 09:26:09 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
Package Architecture Version Repository Size
============================================================================================================================================================================================================================================
Upgrading:
epel-release noarch 8-10.el8 epel 22 k
Transaction Summary
============================================================================================================================================================================================================================================
Upgrade 1 Package
Total download size: 22 k
Downloading Packages:
epel-release-8-10.el8.noarch.rpm 7.8 kB/s | 22 kB 00:02
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 6.1 kB/s | 22 kB 00:03
warning: /var/cache/dnf/epel-05da96c052a128d8/packages/epel-release-8-10.el8.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
Extra Packages for Enterprise Linux 8 - x86_64 1.6 MB/s | 1.6 kB 00:00
Importing GPG key 0x2F86D6A1:
Userid : "Fedora EPEL (8) <epel@fedoraproject.org>"
Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: epel-release-8-10.el8.noarch 1/1
Upgrading : epel-release-8-10.el8.noarch 1/2
Cleanup : epel-release-8-8.el8.noarch 2/2
Running scriptlet: epel-release-8-8.el8.noarch 2/2
Verifying : epel-release-8-10.el8.noarch 1/2
Verifying : epel-release-8-8.el8.noarch 2/2
Upgraded:
epel-release-8-10.el8.noarch
Complete!
配置epel国内镜像源
使用默认的源地址去下载软件的话，有可能会导致下载速度缓慢甚至下载失败。这里推荐使用华为的国内镜像源地址，配置方法如下：
[root@centos /]# sed -i "s/#baseurl/baseurl/g" /etc/yum.repos.d/epel.repo
[root@centos /]# sed -i "s/metalink/#metalink/g" /etc/yum.repos.d/epel.repo
[root@centos /]# sed -i "s@https\?://download.fedoraproject.org/pub@https://mirrors.huaweicloud.com@g" /etc/yum.repos.d/epel.repo
再次更新所有源：
[root@centos /]# yum update -y
Extra Packages for Enterprise Linux 8 - x86_64 5.9 MB/s | 8.8 MB 00:01
Last metadata expiration check: 0:00:02 ago on Wed 13 Jan 2021 09:55:51 AM UTC.
Dependencies resolved.
Nothing to do.
Complete!
使用yum安装singularity
到这里为止，基本的环境配置就完成了，可以正式开始singularity的安装：
[root@centos /]# yum install -y singularity
Last metadata expiration check: 0:00:59 ago on Wed 13 Jan 2021 09:55:51 AM UTC.
Dependencies resolved.
============================================================================================================================================================================================================================================
Package Architecture Version Repository Size
============================================================================================================================================================================================================================================
Installing:
singularity x86_64 3.7.0-1.el8 epel 42 M
Transaction Summary
============================================================================================================================================================================================================================================
Install 1 Package
Total download size: 42 M
Installed size: 138 M
Downloading Packages:
singularity-3.7.0-1.el8.x86_64.rpm 3.4 MB/s | 42 MB 00:12
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 3.4 MB/s | 42 MB 00:12
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : singularity-3.7.0-1.el8.x86_64 1/1
Running scriptlet: singularity-3.7.0-1.el8.x86_64 1/1
Verifying : singularity-3.7.0-1.el8.x86_64 1/1
Installed:
singularity-3.7.0-1.el8.x86_64
Complete!
可以通过如下方式来验证singularity是否安装成功，并且查看版本号：
[root@centos /]# singularity --version
singularity version 3.7.0-1.el8
singualrity的基本使用方法
首先我们可以通过软件自身的help来查看软件支持的功能：
[root@centos /]# singularity --help
Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)
Usage:
singularity [global options...]
Description:
Singularity containers provide an application virtualization layer enabling
mobility of compute via both application and environment portability. With
Singularity one is capable of building a root file system that runs on any
other Linux system where Singularity is installed.
Options:
-c, --config string specify a configuration file (for root or
unprivileged installation only) (default
"/etc/singularity/singularity.conf")
-d, --debug print debugging information (highest verbosity)
-h, --help help for singularity
--nocolor print without color output (default False)
-q, --quiet suppress normal output
-s, --silent only print errors
-v, --verbose print additional information
--version version for singularity
Available Commands:
build Build a Singularity image
cache Manage the local cache
capability Manage Linux capabilities for users and groups
config Manage various singularity configuration (root user only)
delete Deletes requested image from the library
exec Run a command within a container
help Help about any command
inspect Show metadata for an image
instance Manage containers running as services
key Manage OpenPGP keys
oci Manage OCI containers
plugin Manage Singularity plugins
pull Pull an image from a URI
push Upload image to the provided URI
remote Manage singularity remote endpoints, keyservers and OCI/Docker registry credentials
run Run the user-defined default command within a container
run-help Show the user-defined help for an image
search Search a Container Library for images
shell Run a shell within a container
sif siftool is a program for Singularity Image Format (SIF) file manipulation
sign Attach digital signature(s) to an image
test Run the user-defined tests within a container
verify Verify cryptographic signatures attached to an image
version Show the version for Singularity
Examples:
$ singularity help <command> [<subcommand>]
$ singularity help build
$ singularity help instance start
For additional help or support, please visit https://www.sylabs.io/docs/
这里可以看到跟docker的接口还是有较大区别的，但是容器的本质还是基于namespace和cgroup的隔离方案，这点上都是大同小异。由于本文的主要目的在于介绍singularity的安装，这里不详细展开singularity的使用介绍，仅简单介绍一个使用的案例：适用singularity搭建一个ubuntu的容器环境。
- 首先我们创建一个容器沙箱，这里用的基础镜像还是从dockerhub获取的ubuntu基础镜像
[root@centos /]# singularity build --sandbox ubuntu docker://ubuntu
INFO: Starting build...
Getting image source signatures
Copying blob da7391352a9b done
Copying blob 14428a6d4bcd skipped: already exists
Copying blob 2c2d948710f2 [--------------------------------------] 0.0b / 0.0b
Copying config aa23411143 done
Writing manifest to image destination
Storing signatures
2021/01/14 01:20:39 info unpack layer: sha256:da7391352a9bb76b292a568c066aa4c3cbae8d494e6a3c68e3c596d34f7c75f8
2021/01/14 01:20:39 info unpack layer: sha256:14428a6d4bcdba49a64127900a0691fb00a3f329aced25eb77e3b65646638f8d
2021/01/14 01:20:39 info unpack layer: sha256:2c2d948710f21ad82dce71743b1654b45acb5c059cf5c19da491582cef6f2601
INFO: Creating sandbox directory...
INFO: Build complete: ubuntu
- 在执行结束后可以在当期目录下生成一个名为
ubuntu的目录，通过拉起该目录为容器，可以修改镜像配置，制作自己的容器镜像
[root@centos /]# singularity shell -w ubuntu
- 修改完配置之后，可以将该沙箱
build成一个镜像文件
[root@centos /]# singularity build ubuntu-base.sif ubuntu/
INFO: Starting build...
INFO: Creating SIF file...
INFO: Build complete: ubuntu-base.sif
- 创建好
sif镜像之后，会在本地生成一个sif文件，可通过该sif文件拉起一个容器，这里我们可以用一个非root帐号来拉起
[dechin@centos /]# singularity shell ubuntu.sif
这里我们不过多赘述容器的使用方法，后续会出一些源码安装的方案以及更多的使用和配置场景。
参考链接
在CentOS上安装Singularity高性能容器的更多相关文章
- 在Centos上安装RabbitMQ流程(转)
在Centos上安装RabbitMQ流程------------------------ 1. 需求 由于项目中要用到消息队列,经过ActiveMQ与RabbitMQ的比较,最终选择了RabbbitM ...
- 什么是EPEL 及 Centos上安装EPEL
RHEL以及他的衍生发行版如CentOS为了稳定,官方的rpm repository提供的rpm包为了服务器安全稳定更新往往是很滞后的,很多时候需要自己编译那太辛苦了,而EPEL恰恰可以解决这两方面的 ...
- 什么是EPEL 及 Centos上安装EPEL（转）
什么是EPEL 及 Centos上安装EPEL 转自:http://www.unxmail.com/read.php?67 RHEL以及他的衍生发行版如CentOS.Scientific Linux为 ...
- NoSql1 在Linux(CentOS)上安装memcached及使用
前言: 今天是初五,生活基本要从过年的节奏中回归到正常的生活了,所以想想也该想想与工作有关的事情了.我之前在工作中会经常使用memcached和redis,但是自己一直没有时间系统的好好看 ...
- 在Ubuntu|CentOS上安装Shutter截图工具及快捷键设置
简介 Shutter前身叫GScrot,它是一款相当棒的截图软件. 通过Shutter,你可以截取包括选定区域.全屏幕.窗口.窗口内的控件甚至网页的图像.通过内置的强大插件机制,你可以在截图后,对图像 ...
- 在CentOS上安装rabbitmq-server
***在 CentOS 6.4上安装python*** 注意啊,自己手动安装python2.7.5,不要动系统上面其他的版本 1,先安装GCC,用如下命令yum install gcc gcc-c++ ...
- CentOS上安装软件错误提示：configure: error: no acceptable C compiler found in $PATH
CentOS上安装软件错误提示:configure: error: no acceptable C compiler found in $PATH 因为是centos linux,默认可以采用yum方 ...
- 如何在centos上安装epel源
一.EPEL是什么? EPEL (Extra Packages for Enterprise Linux,企业版Linux的额外软件包) 是Fedora小组维护的一个软件仓库项目,为RHEL/Cent ...
- 在CentOS上安装ZooKeeper集群
一共准备3个CentOS虚拟机 172.16.9.194 172.16.9.195 172.16.9.196 上传zookeeper-3.3.6.tar.gz到服务器并解压,3台服务器的目录结构如下 ...
随机推荐
- CF484B Maximum Value
这个东西他一开始也是草稿 题解 我们考虑在值域上做,设值域为 \(m\) . 我们可以考虑数论分块,对于一对 \(a_i\) 和 \(a_j\) ,$\left \lfloor \frac{a_i}{ ...
- JUC（二）：CAS及ABA
CAS是什么? 比较并交换. CAS示例 package com.chinda.java.audition; import java.util.concurrent.atomic.AtomicInte ...
- IDEA的基本操作——导入导出jar包
在使用Jmeter工具测试时,有时也需要导出jar包,测试对应功能,或者自己二次开发Jmeter工具,也是需要导出jar包的.既然经常用,所以就总结了下导入导出jar包的方法. 导入jar包 先打开i ...
- Java CPU占用过高之诊断
Redhat, java 先用top 命令(默认为cpu占用排序), 确认java占用CPU kill -3 <java pid> ps Hh -eo pid,tid,pcpu | sor ...
- 大白话详解大数据hive知识点，老刘真的很用心(3)
前言:老刘不敢说写的有多好,但敢保证尽量用大白话把自己复习的内容详细解释出来,拒绝资料上的生搬硬套,做到有自己的了解! 1. hive知识点(3) 从这篇文章开始决定进行一些改变,老刘在博客上主要分享 ...
- 从零到一快速搭建个人博客网站（域名自动跳转www，二级域名使用）（二）
前言 本篇文章是对上篇文章从零到一快速搭建个人博客网站(域名备案 + https免费证书)(一)的完善,比如域名自动跳转www.二级域名使用等. 域名自动跳转www 这里对上篇域名访问进行优化,首先支 ...
- Spring Cloud Alibaba基础教程-Nacos(二)
在Spring Cloud Alibaba基础教程-Nacos(一)当中学习了,如何从 nacos当中 通过Java的方式获取值,以及连接数据库,下面我们开始第二篇的学习 ,如果对你有帮助,方便下次寻 ...
- python解压可迭代对象赋值给多个变量
如果一个可迭代对象的元素个数超过了变量的个数,会出现"解压值太多"的异常,python提供*号表达式(码夫破石的理解)来解决. 1,求一组平均值,就以类似学生成绩为例,要求是去掉第 ...
- C#未能找到路径“\bin\roslyn\csc.exe”的一部分。
主要原因是因为两个库存在,需要生成一个 roslyn文件,但是这个项目是从 vs2017中,打开的,所以,没有必要存在它. 那么就删除这两个关联的库,就可以达到目的 S2017 打开 程序包管理控制 ...
- 在Linux下面端口映射socat自动脚本
这个sh脚本可以方面的端口映射,在使用本功能之前请确保socat已经放到了/usr/bin/socat #!/bin/bash cd `dirname $0` let listenport=`base ...