一、环境介绍

主机名 IP地址
master 192.168.0.100
node1 192.168.0.101
node2 192.168.0.102

1.1、操作系统: CensOS8.4.2105

[root@kubernetes-master /]# cat /etc/redhat-release
CentOS Linux release 8.4.2105

1.2、Kubernetes版本 v1.19.1

     kube-apiserver              v1.19.1
    kube-controller-manager     v1.19.1
    kube-proxy                 v1.19.1
    kube-scheduler             v1.19.1
    etcd                       3.3.10
    pause                       3.1
    coredns                     1.3.1

二:Linux服务器安装及系统配置

2.1:Linux安装注意

安装VMware虚拟机软件,下载Centos镜像,安装的时候一点切记,为了使用方便,保证局域网内其他主机能访问到我们的虚拟机,所以这里的网络模式我们使用桥接模式,把虚拟机当做一个真实的物理机来使用,安装完成之后再配置其静态IP,配置静态IP的方式如下。

找到文件  /etc/sysconfig/network-scripts/ifcfg-ens33,修改为以下配置

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=7ab3916f-b1cf-4d45-a39f-f8a7d2447349
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.0.100
PREFIX=24
GATEWAY=192.168.0.1
DNS1=114.114.114.114
IPV6_PRIVACY=no
NETMASK=255.255.255.0

2.2:准备工作

更新镜像源

yum -y update

由于k8s对时间的要求特别的高,所以我们必须保证集群中的每台机器的完全一致,时间同步命令如下:

systemctl start chronyd

服务器文件同步脚本

创建脚本: mkdir xsync

脚本中添加一下内容

#!/bin/bash
#1 获取输入参数个数,如果没有参数,直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi

#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname

#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir

#4 获取当前用户名称
user=`whoami`
host_prefix='192.168.0.'

#5 循环
for((host=100; host<103; host++))
do
rsync -rvl $pdir/$fname $user@$host_prefix$host:$pdir
done

2.3:关闭防火墙,开机不自启动

systemctl stop firewalld
systemctl disable firewalld

2.4 禁用selinux

vi /etc/selinux/config
修改以下配置
SELINUX=disabled

2.5:禁用swap

注释掉 # /dev/mapper/cl-swap     swap                    swap    defaults        0 0

2.6: 配置ipvs功能

接下来还需要确保各个节点上已经安装了ipset软件包
# yum -y install ipset
为了便于查看ipvs的代理规则,最好安装一下管理工具ipvsadm
# yum -y install ipvsadm

如果以上前提条件如果不满足,则即使kube-proxy的配置开启了ipvs模式,也会退回到iptables模式

添加需要加载的模块写入脚本文件

cat <<EOF>  /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

为脚本文件添加执行权限

chmod +x /etc/sysconfig/modules/ipvs.modules

执行脚本文件

/bin/bash /etc/sysconfig/modules/ipvs.modules

查看对应的模块是否加载成功

lsmod | grep -e ip_vs -e nf_conntrack_ipv4

2.7:重启服务器,检查之前关闭的各种配置是否生效S

getenforce
free -m

三:Kubernetes安装

3.1:docker安装

切换镜像源

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O/etc/yum.repos.d/docker-ce.repo

查看镜像源中支持的docker版本

 yum list docker-ce --showduplicates

安装

yum install docker-ce

3.2:添加配置文件

Docker在黑默认情况下使用的Cgroup Driver为cgroupfs,而kubernetes推荐使用systemd来代替cgroupfs

mkdir /etc/docker

cat <<EOF > /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://kn0t2bca.mirror.aliyuncs.com"]
}
EOF

3.3:启动docker

# 启动docker
systemctl start docker
# 开机启动
systemctl enable docker

安装Kubernetes组件 由于Kubernetes的镜像源在国外,速度比较慢,这里切换成国内的镜像源 编辑/etc/yum.repos.d/kubernetes.repo,添加下面的配置

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装kubeadm,kubelet和kubectl

yum -y install kubeadm-1.19.1 kubectl-1.19.1 kubelet-1.19.1

更换版本

yum remove kubeadm.x86_64 kubectl.x86_64 kubelet.x86_64 -y

配置kubelet的cgroup

# 编辑/etc/sysconfig/kubelet,添加下面的配置
KUBELET_CGROUP_ARGS="--cgroup-driver=systemd"
KUBE_PROXY_MODE="ipvs"

设置kubelet开机自启

systemctl enable kubelet

3.4:初始化集群

在master 上操作


# 由于默认拉取镜像地址 k8s.gcr.io 国内无法访问,这里指定阿里云镜像仓库地址
# apiserver-advertise-address 需要写自己的ip
kubeadm init \
--apiserver-advertise-address=192.168.0.100 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version=v1.19.1 \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.96.0.0/12

#使用 kubectl 工具
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

加入master指令,在执行完kubeadm init 之后会自动在控制台打印

kubeadm join 192.168.0.100:6443 --token awk15p.t6bamck54w69u4s8 \
--discovery-token-ca-cert-hash sha256:a94fa09562466d32d29523ab6cff122186f1127599fa4dcd5fa0152694f17117

然后我们需要将node 节点加入集群中,在 node 服务器 上执行上述红框的命令加入到master,执行完毕后可在master节点查看,指令如下

kubectl get nodes

3.5:安装网络插件,只在master节点操作即可

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

由于外网不好访问,如果出现无法访问的情况,可以直接用下面的 记得文件名是kube-flannel.yml,位置:/root/

kube-flannel.yml内容

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:
- configMap
- secret
- emptyDir
- hostPath
allowedHostPaths:
- pathPrefix: "/etc/cni/net.d"
- pathPrefix: "/etc/kube-flannel"
- pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false
# Users and groups
runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
# Privilege Escalation
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
# Capabilities
allowedCapabilities: ['NET_ADMIN']
defaultAddCapabilities: []
requiredDropCapabilities: []
# Host namespaces
hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:
- min: 0
max: 65535
# SELinux
seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
rules:
- apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-amd64
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- amd64
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-amd64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-amd64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-arm64
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- arm64
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-arm64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-arm64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-arm
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- arm
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-arm
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-arm
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-ppc64le
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- ppc64le
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-ppc64le
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-ppc64le
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-s390x
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- s390x
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-s390x
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay-mirror.qiniu.com/coreos/flannel:v0.12.0-s390x
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg

使用配置文件启动fannel

kubectl apply -f kube-flannel.yml

等待它安装完毕 发现已经是 集群的状态已经是Ready

四:集群测试

创建一个nginx服务

kubectl create deployment nginx --image=nginx:1.14-alpine

暴露端口

kubectl expose deploy nginx --port=80 --target-port=80 --type=NodePort

查看服务

kubectl get svc   或者 kubectl get service

查看pod

kubectl get pod

浏览器测试,集群的每个节点都可以访问

至此,kubernetes环境搭建完成,第一步已完成,在了解k8s的路上又近了一步。

Kubernetes集群安装的更多相关文章

  1. Kubernetes集群部署史上最详细(一)Kubernetes集群安装

    适用部署结构以及版本 本系列中涉及的部署方式和脚本适用于1.13.x和1.14,而且采取的是二进制程序部署方式. 脚本支持的部署模式 最小部署模式 3台主机,1台为k8s的master角色,其余2台为 ...

  2. Kubernetes 集群安装部署

    etcd集群配置 master节点配置 1.安装kubernetes etcd [root@k8s ~]# yum -y install kubernetes-master etcd 2.配置 etc ...

  3. 【Kubernetes学习之二】Kubernetes集群安装

    环境 centos 7 Kubernetes有三种安装方式:yum.二进制.kubeadm,这里演示kubeadm. 一.准备工作1.软件版本 软件 版本 kubernetes v1.15.3 Cen ...

  4. Kubernetes集群安装(自己搭过,已搭好)

    k8s安装目录 1. 组件版本 && 集群环境 组件版本 etcd 集群 && k8s master 机器 && k8s node 机器 集群环境变量 ...

  5. kubernetes 集群安装etcd集群,带证书

    install etcd 准备证书 https://www.kubernetes.org.cn/3096.html 在master1需要安装CFSSL工具,这将会用来建立 TLS certificat ...

  6. centos7.5下kubeadm安装kubernetes集群安装

    文章是按https://blog.csdn.net/Excairun/article/details/88962769,来进行操作并记录相关结果 版本:k8s V14.0,docker-ce 18.0 ...

  7. [转] Kubernetes集群安装文档-v1.6版本

    [From] https://www.kubernetes.org.cn/1870.html http://jimmysong.io/kubernetes-handbook

  8. 基于Containerd安装部署高可用Kubernetes集群

    转载自:https://blog.weiyigeek.top/2021/7-30-623.html 简述 Kubernetes(后续简称k8s)是 Google(2014年6月) 开源的一个容器编排引 ...

  9. k8s入门系列之集群安装篇

    关于kubernetes组件的详解介绍,请阅读上一篇文章<k8s入门系列之介绍篇> Kubernetes集群安装部署 •Kubernetes集群组件: - etcd 一个高可用的K/V键值 ...

随机推荐

  1. HashMap方法源码分析

    本文将分析put(),resize(),get()和remove()方法的源码 putval()方法 大致步骤:计算key的hash值:根据hash值计算数组下标:判断下标处是否有节点,无节点则直接插 ...

  2. v-bind的使用

    v-bind v-bind的引入 ​ 内容的绑定可以通过mustache语法,而属性的绑定往往需要通过v-bind 如动态绑定img的src属性 如动态绑定div的class属性 如动态绑定li元素的 ...

  3. IO系列测试源码

    //IO系列测试源码(需要自取) using System; using System.CodeDom; using System.Collections.Generic; using System. ...

  4. Mybatis学习之自定义持久层框架(七) 自定义持久层框架优化

    前言 接上文,这里只是出于强迫症,凭借着半年前的笔记来把之前没写完的文章写完,这里是最后一篇了. 前面自定义的持久层框架存在的问题 Dao层若使用实现类,会存在代码重复,整个操作的过程模版重复(加载配 ...

  5. JS实现单例模式的多种方案

    JS实现单例模式的多种方案 今天在复习设计模式中的-创建型模式,发现JS实现单例模式的方案有很多种,稍加总结了一下,列出了如下的6种方式与大家分享 大体上将内容分为了ES5(Function)与ES6 ...

  6. Ubuntu切换到root账户

    使用命令sudo su或sudo -i然后输入密码就可以切换到root帐号,exit可以退出.

  7. Linux单设备多路USB串口的实现方法介绍

    某设备需要提供多路USB串口的功能给主机端使用,比如一路用作业务1通信功能,一路用作业务2通信功能,一路用作debug抓log用途,诸如此类.如下图所示. 要实现上述设备功能,可以参考如下步骤. 1) ...

  8. Scanner, BufferedReader, InputStreamReader 与ACM模式输入

    Scanner, BufferedReader, InputStreamReader 与ACM模式输入html { -webkit-print-color-adjust: exact } * { bo ...

  9. [bug] Scala eclipse:找不到或无法加载主类

    原因 混合java 和scala 无法编译 解决 右键项目 > properties > scala Compiler >勾选Use Project Setting > Sca ...

  10. MySQL配置HeartBeat实现心跳监控和浮动IP

    1. 初始化环境配置 /sbin/chkconfig --add mysqld /sbin/chkconfig mysqld on ln -s /usr/local/mysql/bin/mysql / ...