[k8s] k8s基于csi使用rbd存储
描述
ceph-csi扩展各种存储类型的卷的管理能力，实现第三方存储ceph的各种操作能力与k8s存储系统的结合。通过 ceph-csi 使用 ceph rbd块设备，它动态地提供rbd以支持 Kubernetes 持久化存储,并将这些rbd映射给 pod做为块设备持久化数据使用。 Ceph 将pod存在块设备的数据以副本机制的方式存储在多个osd上，实现pod数据具有更高的可靠性。
部署环境信息
OS： CentOS Linux release 7.9.2009 (Core)
Kubectl Version：v1.20.2
Ceph Versions：14.2.2
需要部署环境部署可以看我以前的文章基于kolla容器化部署ceph和基于ansible自动化部署k8s
配置ceph
# 创建名为kubernetes存储池。
pg与pgs要根据实际情况修改。
$ ceph osd pool create kubernetes 64 64
$ ceph osd pool application enable kubernetes rbd
# 新创建的池使用前进行rbd初始化。
$ rbd pool init kubernetes
# 为csi创建一个新用户kubernetes访问kubernetes池。执行以下命令并记录生成的密钥。
$ ceph auth get-or-create client.kubernetes mon 'profile rbd' osd 'profile rbd pool=kubernetes' mgr 'profile rbd'
[client.kubernetes]
key = AQDStCFiN0JMFxAAK8EHEnEIRIN+SbACY0T2lw==
下面配置cephx secret对象时，会要用到userID=kubernetes和userKey=AQDStCFiN0JMFxAAK8EHEnEIRIN+SbACY0T2lw==
# csi需要一个存储在 Kubernetes 中的ConfigMap对象, 对象定义 的是Ceph 集群的 Ceph 监视器地址和fsid。收集 Ceph 集群唯一fsid和监视器地址。
$ ceph mon dump
.....
fsid 4a9e463a-4853-4237-a5c5-9ae9d25bacda
0: [v2:172.20.163.52:3300/0,v1:172.20.163.52:6789/0] mon.172.20.163.52
ceph-csi目前只支持旧版 V1 协议
配置ceph-csi configmap
# 创建ConfigMap对象，将fsid替换为"clusterID"，并将监视器地址替换为"monitors"。
$ cat > csi-config-map.yaml << EOF
---
apiVersion: v1
kind: ConfigMap
data:
config.json: |-
[
{
"clusterID": "4a9e463a-4853-4237-a5c5-9ae9d25bacda",
"monitors": [
"172.20.163.52:6789",
"172.20.163.52:6789",
"172.20.163.52:6789"
]
}
]
metadata:
name: ceph-csi-config
EOF
$ kubectl apply -f csi-config-map.yaml
# 新版本的csi还需要一个额外的ConfigMap对象来定义密钥管理服务 (KMS) 提供者的详细信息, 空配置即可以。
$ cat > csi-kms-config-map.yaml << EOF
---
apiVersion: v1
kind: ConfigMap
data:
config.json: |-
{}
metadata:
name: ceph-csi-encryption-kms-config
EOF
$ kubectl apply -f csi-kms-config-map.yaml
# 查看ceph.conf文件
$ docker exec ceph_mon cat /etc/ceph/ceph.conf
[global]
log file = /var/log/kolla-ceph/$cluster-$name.log
log to syslog = false
err to syslog = false
log to stderr = false
err to stderr = false
fsid = 4a9e463a-4853-4237-a5c5-9ae9d25bacda
mon initial members = 172.20.163.52
mon host = 172.20.163.52
mon addr = 172.20.163.52:6789
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 1
osd pool default min size = 1
setuser match path = /var/lib/ceph/$type/$cluster-$id
osd crush update on start = false
# 通过ConfigMap对象来定义 Ceph 配置，以添加到 CSI 容器内的 ceph.conf 文件中。ceph.conf文件内容替换以下的内容。
$ cat > ceph-config-map.yaml << EOF
---
apiVersion: v1
kind: ConfigMap
data:
ceph.conf: |
[global]
log file = /var/log/kolla-ceph/$cluster-$name.log
log to syslog = false
err to syslog = false
log to stderr = false
err to stderr = false
fsid = 4a9e463a-4853-4237-a5c5-9ae9d25bacda
mon initial members = 172.20.163.52
mon host = 172.20.163.52
mon addr = 172.20.163.52:6789
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 1
osd pool default min size = 1
setuser match path = /var/lib/ceph/$type/$cluster-$id
osd crush update on start = false
# keyring is a required key and its value should be empty
keyring: |
metadata:
name: ceph-config
EOF
$ kubectl apply -f ceph-config-map.yaml
配置ceph-csi cephx secret
# 创建secret对象, csi需要cephx凭据才能与ceph集群通信。
$ cat > csi-rbd-secret.yaml << EOF
---
apiVersion: v1
kind: Secret
metadata:
name: csi-rbd-secret
namespace: default
stringData:
userID: kubernetes
userKey: AQDStCFiN0JMFxAAK8EHEnEIRIN+SbACY0T2lw==
EOF
$ kubectl apply -f csi-rbd-secret.yaml
配置ceph-csi插件
# 创建所需的ServiceAccount和 RBAC ClusterRole/ClusterRoleBinding Kubernetes 对象。
点击查看csi-provisioner-rbac代码
$ cat > csi-provisioner-rbac.yaml << EOF
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-csi-provisioner
# replace with non-default namespace name
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-external-provisioner-runner
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "update", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["get", "list", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update", "patch"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get"]
- apiGroups: [""]
resources: ["serviceaccounts"]
verbs: ["get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-csi-provisioner-role
subjects:
- kind: ServiceAccount
name: rbd-csi-provisioner
# replace with non-default namespace name
namespace: default
roleRef:
kind: ClusterRole
name: rbd-external-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
# replace with non-default namespace name
namespace: default
name: rbd-external-provisioner-cfg
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch", "create", "update", "delete"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-csi-provisioner-role-cfg
# replace with non-default namespace name
namespace: default
subjects:
- kind: ServiceAccount
name: rbd-csi-provisioner
# replace with non-default namespace name
namespace: default
roleRef:
kind: Role
name: rbd-external-provisioner-cfg
apiGroup: rbac.authorization.k8s.io
EOF
$ $ kubectl apply -f csi-provisioner-rbac.yaml
点击查看csi-nodeplugin-rbac代码
$ cat > csi-nodeplugin-rbac.yaml << EOF
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-csi-nodeplugin
# replace with non-default namespace name
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-csi-nodeplugin
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get"]
# allow to read Vault Token and connection options from the Tenants namespace
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get"]
- apiGroups: [""]
resources: ["serviceaccounts"]
verbs: ["get"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-csi-nodeplugin
subjects:
- kind: ServiceAccount
name: rbd-csi-nodeplugin
# replace with non-default namespace name
namespace: default
roleRef:
kind: ClusterRole
name: rbd-csi-nodeplugin
apiGroup: rbac.authorization.k8s.io
EOF
$ kubectl apply -f csi-nodeplugin-rbac.yaml
# 创建所需的ceph-csi容器
点击查看csi-rbdplugin-provisioner代码
$ cat > csi-rbdplugin-provisioner.yaml << EOF
---
kind: Service
apiVersion: v1
metadata:
name: csi-rbdplugin-provisioner
# replace with non-default namespace name
namespace: default
labels:
app: csi-metrics
spec:
selector:
app: csi-rbdplugin-provisioner
ports:
- name: http-metrics
port: 8080
protocol: TCP
targetPort: 8681
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-rbdplugin-provisioner
# replace with non-default namespace name
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: csi-rbdplugin-provisioner
template:
metadata:
labels:
app: csi-rbdplugin-provisioner
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-rbdplugin-provisioner
topologyKey: "kubernetes.io/hostname"
serviceAccountName: rbd-csi-provisioner
priorityClassName: system-cluster-critical
hostNetwork: true
containers:
- name: csi-provisioner
image: antidebug/csi-provisioner:v3.0.0
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--timeout=150s"
- "--retry-interval-start=500ms"
- "--leader-election=true"
# set it to true to use topology based provisioning
- "--feature-gates=Topology=false"
# if fstype is not specified in storageclass, ext4 is default
- "--default-fstype=ext4"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-snapshotter
image: antidebug/csi-snapshotter:v4.1.1
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--timeout=150s"
- "--leader-election=true"
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-attacher
image: antidebug/csi-attacher:v3.2.1
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--retry-interval-start=500ms"
env:
- name: ADDRESS
value: /csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-resizer
image: antidebug/csi-resizer:v1.2.0
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--timeout=150s"
- "--leader-election"
- "--retry-interval-start=500ms"
- "--handle-volume-inuse-error=false"
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-rbdplugin
# for stable functionality replace canary with latest release version
image: quay.io/cephcsi/cephcsi:v3.5.1
args:
- "--nodeid=$(NODE_ID)"
- "--type=rbd"
- "--controllerserver=true"
- "--endpoint=$(CSI_ENDPOINT)"
- "--csi-addons-endpoint=$(CSI_ADDONS_ENDPOINT)"
- "--v=5"
- "--drivername=rbd.csi.ceph.com"
- "--pidlimit=-1"
- "--rbdhardmaxclonedepth=8"
- "--rbdsoftmaxclonedepth=4"
- "--enableprofiling=false"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: KMS_CONFIGMAP_NAME
# value: encryptionConfig
- name: CSI_ENDPOINT
value: unix:///csi/csi-provisioner.sock
- name: CSI_ADDONS_ENDPOINT
value: unix:///csi/csi-addons.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- mountPath: /dev
name: host-dev
- mountPath: /sys
name: host-sys
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- name: ceph-csi-config
mountPath: /etc/ceph-csi-config/
- name: ceph-csi-encryption-kms-config
mountPath: /etc/ceph-csi-encryption-kms-config/
- name: keys-tmp-dir
mountPath: /tmp/csi/keys
- name: ceph-config
mountPath: /etc/ceph/
- name: csi-rbdplugin-controller
# for stable functionality replace canary with latest release version
image: quay.io/cephcsi/cephcsi:v3.5.1
args:
- "--type=controller"
- "--v=5"
- "--drivername=rbd.csi.ceph.com"
- "--drivernamespace=$(DRIVER_NAMESPACE)"
env:
- name: DRIVER_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: ceph-csi-config
mountPath: /etc/ceph-csi-config/
- name: keys-tmp-dir
mountPath: /tmp/csi/keys
- name: ceph-config
mountPath: /etc/ceph/
- name: liveness-prometheus
image: quay.io/cephcsi/cephcsi:v3.5.1
args:
- "--type=liveness"
- "--endpoint=$(CSI_ENDPOINT)"
- "--metricsport=8681"
- "--metricspath=/metrics"
- "--polltime=60s"
- "--timeout=3s"
env:
- name: CSI_ENDPOINT
value: unix:///csi/csi-provisioner.sock
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: socket-dir
mountPath: /csi
imagePullPolicy: "IfNotPresent"
volumes:
- name: host-dev
hostPath:
path: /dev
- name: host-sys
hostPath:
path: /sys
- name: lib-modules
hostPath:
path: /lib/modules
- name: socket-dir
emptyDir: {
medium: "Memory"
}
- name: ceph-config
configMap:
name: ceph-config
- name: ceph-csi-config
configMap:
name: ceph-csi-config
- name: ceph-csi-encryption-kms-config
configMap:
name: ceph-csi-encryption-kms-config
- name: keys-tmp-dir
emptyDir: {
medium: "Memory"
}
EOF
$ kubectl apply -f csi-rbdplugin-provisioner.yaml
点击csi-rbdplugin代码
$ cat > csi-rbdplugin.yaml << EOF
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-rbdplugin
# replace with non-default namespace name
namespace: default
spec:
selector:
matchLabels:
app: csi-rbdplugin
template:
metadata:
labels:
app: csi-rbdplugin
spec:
serviceAccountName: rbd-csi-nodeplugin
hostNetwork: true
hostPID: true
priorityClassName: system-node-critical
# to use e.g. Rook orchestrated cluster, and mons' FQDN is
# resolved through k8s service, set dns policy to cluster first
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: driver-registrar
# This is necessary only for systems with SELinux, where
# non-privileged sidecar containers cannot access unix domain socket
# created by privileged CSI driver container.
securityContext:
privileged: true
image: antidebug/csi-node-driver-registrar:v2.2.0
args:
- "--v=5"
- "--csi-address=/csi/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/plugins/rbd.csi.ceph.com/csi.sock"
env:
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: registration-dir
mountPath: /registration
- name: csi-rbdplugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
# for stable functionality replace canary with latest release version
image: quay.io/cephcsi/cephcsi:v3.5.1
args:
- "--nodeid=$(NODE_ID)"
- "--pluginpath=/var/lib/kubelet/plugins"
- "--stagingpath=/var/lib/kubelet/plugins/kubernetes.io/csi/pv/"
- "--type=rbd"
- "--nodeserver=true"
- "--endpoint=$(CSI_ENDPOINT)"
- "--csi-addons-endpoint=$(CSI_ADDONS_ENDPOINT)"
- "--v=5"
- "--drivername=rbd.csi.ceph.com"
- "--enableprofiling=false"
# If topology based provisioning is desired, configure required
# node labels representing the nodes topology domain
# and pass the label names below, for CSI to consume and advertise
# its equivalent topology domain
# - "--domainlabels=failure-domain/region,failure-domain/zone"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# - name: KMS_CONFIGMAP_NAME
# value: encryptionConfig
- name: CSI_ENDPOINT
value: unix:///csi/csi.sock
- name: CSI_ADDONS_ENDPOINT
value: unix:///csi/csi-addons.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- mountPath: /dev
name: host-dev
- mountPath: /sys
name: host-sys
- mountPath: /run/mount
name: host-mount
- mountPath: /etc/selinux
name: etc-selinux
readOnly: true
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- name: ceph-csi-config
mountPath: /etc/ceph-csi-config/
- name: ceph-csi-encryption-kms-config
mountPath: /etc/ceph-csi-encryption-kms-config/
- name: plugin-dir
mountPath: /var/lib/kubelet/plugins
mountPropagation: "Bidirectional"
- name: mountpoint-dir
mountPath: /var/lib/kubelet/pods
mountPropagation: "Bidirectional"
- name: keys-tmp-dir
mountPath: /tmp/csi/keys
- name: ceph-logdir
mountPath: /var/log/ceph
- name: ceph-config
mountPath: /etc/ceph/
- name: liveness-prometheus
securityContext:
privileged: true
image: quay.io/cephcsi/cephcsi:v3.5.1
args:
- "--type=liveness"
- "--endpoint=$(CSI_ENDPOINT)"
- "--metricsport=8680"
- "--metricspath=/metrics"
- "--polltime=60s"
- "--timeout=3s"
env:
- name: CSI_ENDPOINT
value: unix:///csi/csi.sock
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: socket-dir
mountPath: /csi
imagePullPolicy: "IfNotPresent"
volumes:
- name: socket-dir
hostPath:
path: /var/lib/kubelet/plugins/rbd.csi.ceph.com
type: DirectoryOrCreate
- name: plugin-dir
hostPath:
path: /var/lib/kubelet/plugins
type: Directory
- name: mountpoint-dir
hostPath:
path: /var/lib/kubelet/pods
type: DirectoryOrCreate
- name: ceph-logdir
hostPath:
path: /var/log/ceph
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry/
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-sys
hostPath:
path: /sys
- name: etc-selinux
hostPath:
path: /etc/selinux
- name: host-mount
hostPath:
path: /run/mount
- name: lib-modules
hostPath:
path: /lib/modules
- name: ceph-config
configMap:
name: ceph-config
- name: ceph-csi-config
configMap:
name: ceph-csi-config
- name: ceph-csi-encryption-kms-config
configMap:
name: ceph-csi-encryption-kms-config
- name: keys-tmp-dir
emptyDir: {
medium: "Memory"
}
---
# This is a service to expose the liveness metrics
apiVersion: v1
kind: Service
metadata:
name: csi-metrics-rbdplugin
# replace with non-default namespace name
namespace: default
labels:
app: csi-metrics
spec:
ports:
- name: http-metrics
port: 8080
protocol: TCP
targetPort: 8680
selector:
app: csi-rbdplugin
EOF
此处csi-rbdplugin-provisioner
代码与官方不同之处: pod采用hostNetwork: true。
使用ceph块设备
# 创建一个存储类
kubernetes storageclass定义了一个存储类。 可以创建多个storageclass对象以映射到不同的服务质量级别（即 NVMe 与基于 HDD 的池）和功能。例如，要创建一个映射到上面创建的kubernetes池的storageclass ，确保"clusterID"与您的ceph集群的fsid一致。
$ cat > csi-rbd-sc.yaml << EOF
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-rbd-sc
provisioner: rbd.csi.ceph.com
parameters:
clusterID: 4a9e463a-4853-4237-a5c5-9ae9d25bacda
pool: kubernetes
imageFeatures: layering
csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/controller-expand-secret-name: csi-rbd-secret
csi.storage.k8s.io/controller-expand-secret-namespace: default
csi.storage.k8s.io/node-stage-secret-name: csi-rbd-secret
csi.storage.k8s.io/node-stage-secret-namespace: default
reclaimPolicy: Delete
allowVolumeExpansion: true
mountOptions:
- discard
EOF
$ kubectl apply -f csi-rbd-sc.yaml
# 创建pvc
使用上面创建的storageclass创建PersistentVolumeClaim
cat > raw-block-pvc.yaml << EOF
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: raw-block-pvc
spec:
accessModes:
- ReadWriteOnce
volumeMode: Block
resources:
requests:
storage: 1Gi
storageClassName: csi-rbd-sc
EOF
$ kubectl apply -f raw-block-pvc.yaml
# 查看pvc的状态为Bound即正常。
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
raw-block-pvc Bound pvc-d57ba3b8-c916-4182-966f-eb8680955cb7 2Gi RWO csi-rbd-sc 132m
参考文献
[1] ceph.com 作者 202203
[k8s] k8s基于csi使用rbd存储的更多相关文章
- 基于openshift+华为对象存储的CSI开发
目录 需求来源 环境准备 代码修改 镜像下载 镜像生成 修改部署文件 部署CSI插件 CSI原理 核心原理 生命周期: 组件介绍 FAQ 参考: 需求来源 项目上目前使用的是openshift 3.1 ...
- TKE用户故事 | 作业帮检索服务基于Fluid的计算存储分离实践
作者 吕亚霖,2019年加入作业帮,作业帮基础架构-架构研发团队负责人,在作业帮期间主导了云原生架构演进.推动实施容器化改造.服务治理.GO微服务框架.DevOps的落地实践. 张浩然,2019年加入 ...
- [k8s]k8s内容索引
我会陆陆续续小结下. dns相关 dns策略 1.集群内 2.指定内网 subdomain 3.访问互联网 resovel.conf kube-dns架构图解及策略 nginx-ingress: DN ...
- [k8s]k8s 1.9(on the fly搭建) 1.9_cni-flannel部署排错 ipvs模式
角色 节点名 节点ip master n1 192.168.14.11 节点1 n2 192.168.14.12 节点2 n3 192.168.14.13 https://raw.githubuser ...
- 【WP8.1开发】基于应用的联系人存储
上一篇文章所吹的牛是访问系统(手机)上的联系人,当然那只是读不能改,这是自然的,要是让你能随便修改用户的联系人信息的话,那后果很严重,有些恶意开发者就有可能把”你的户口改成猪“. 但是,API也允许应 ...
- 基于catalog 创建RMAN存储脚本
--============================== -- 基于catalog 创建RMAN存储脚本 --============================== 简言之,将rman的 ...
- [大牛翻译系列]Hadoop（19）MapReduce 文件处理：基于压缩的高效存储（二）
5.2 基于压缩的高效存储(续) (仅包括技术27) 技术27 在MapReduce,Hive和Pig中使用可分块的LZOP 如果一个文本文件即使经过压缩后仍然比HDFS的块的大小要大,就需要考虑选择 ...
- [大牛翻译系列]Hadoop（18）MapReduce 文件处理：基于压缩的高效存储（一）
5.2 基于压缩的高效存储 (仅包括技术25,和技术26) 数据压缩可以减小数据的大小,节约空间,提高数据传输的效率.在处理文件中,压缩很重要.在处理Hadoop的文件时,更是如此.为了让Hadoop ...
- 服务注册发现consul之四： 分布式锁之四：基于Consul的KV存储和分布式信号量实现分布式锁
一.基于key/value实现 我们在构建分布式系统的时候,经常需要控制对共享资源的互斥访问.这个时候我们就涉及到分布式锁(也称为全局锁)的实现,基于目前的各种工具,我们已经有了大量的实现方式,比如: ...
随机推荐
- Spring源码-IOC部分-Spring是如何解决Bean循环依赖的【6】
实验环境:spring-framework-5.0.2.jdk8.gradle4.3.1 Spring源码-IOC部分-容器简介[1] Spring源码-IOC部分-容器初始化过程[2] Spring ...
- stram流char[]保存，支持中文，Filestram需要先转byte[]才能使用，但是性能更好《转载》
学习流的使用时(stream类),逐步遇到新的理解,记录一下 1.FileStream流是处理byte[],默认UTF8类型 当你使用wirte方法时将非byte类型的输入内容,先将内容通过转换为字节 ...
- C# 将OFD转为PDF
OFD格式的文档是一种我国独有的国家标准版式的文档,在不同场景需求中,可以通过格式转换的方法将PDF转为OFD,或者将OFD转为PDF.本次内容,将通过C#程序介绍如何实现由OFD到PDF的转换,并附 ...
- AQS源码一窥-JUC系列
AQS源码一窥 考虑到AQS的代码量较大,涉及信息量也较多,计划是先使用较常用的ReentrantLock使用代码对AQS源码进行一个分析,一窥内部实现,然后再全面分析完AQS,最后把以它为基础的同步 ...
- simpholders 官方网址 https://www.simpholders.com/
SimPholders可让你快速直接地访问iPhone模拟器应用的app文档.你可以通过SimPholders找到数据库文件.永久存储以及缓存,它是一个非常实用的app debug工具,同时还可以离线 ...
- PHP版的猴子选大王算法
猴子选大王 这个算法可能是目前我看到的最简洁都算法吧,而且很好理解.它不同于其他算法,其他算法都是判断这个猴子能不能被选中,而他只是找出不能被选中的猴子,然后将其塞到数组模拟的环状队列中,参与下次选. ...
- webpack引入css文件
需要配置 postcss 详见 官网 https://www.postcss.com.cn/
- C语言中的单引号和双引号的区别
首先肯定地说,二者是有区别的,不是说用谁都一样. 1.实质区别,代表的含义不同 'A'代表的是一个整数,而且这个整数对应的是编译器所采用的字符集中的字符序列对应的数值.所以'A'跟ASCII中的65意 ...
- 如何在 pyqt 中实现全局事件总线
前言 在 Qt 中可以使用信号和槽机制很方便地实现部件之间的通信,考虑下面这样的场景: 我想要点击任意一个专辑卡并通知主界面跳转到专辑界面,那么一种实现方式如上图所示:点击任意一个蓝色方框所示的专辑卡 ...
- Solution -「AGC 003D」「AT 2004」Anticube
\(\mathcal{Description}\) Link. 给定 \(n\) 个数 \(a_i\),要求从中选出最多的数,满足任意两个数之积都不是完全立方数. \(n\le10^5\) ...