k8s环境搭建

mirantis 有个培训，提供了安装脚本

git clone https://bitbucket.org/mirantis-training/kd100-scripts
网络采用的是calico

培训内容（student， sublime）

测试网址（user: pass: happy knuth）

k8s的所有项目：

https://github.com/kubernetes

有源码，有例子。

安装环境：

http://los-vmm.sc.intel.com/wiki/Start_a_devstack_in_20_minutes

wget -O- http://otcloud-gateway.bj.intel.com/runstack |bash

配置cloud init 可参考 http://www.cnblogs.com/shaohef/p/8137073.html

用户搭建

yanglin写了一个脚本安装

https://github.com/shaohef/transcoder-daemon/blob/master/k8s/installk8s.sh

kubectl 使用 snap安装

$ sudo apt update

$ sudo apt upgrade

Ubuntu使用Snap安装管理软件包

一键脚本

# https://kubernetes.io/docs/tasks/tools/install-kubectl/

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

chmod +x ./kubectl

sudo mv ./kubectl /usr/local/bin/kubectl

source <(kubectl completion bash)

# install docker

sudo docker version

if [ $? !=  ]; then

    wget -O- https://get.docker.com/ |bash

    sudo usermod -aG docker $USER

fi

# https://kubernetes.io/docs/tasks/tools/install-minikube/

# User VM to install kubernetes

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#14-installing-kubeadm-on-your-hosts

# https://kubernetes.io/doup/incs/setdependent/install-kubeadm/

sudo apt-get update && sudo apt-get install -y apt-transport-https

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

# should use tee for sudo user

cat <<EOF |sudo tee -a /etc/apt/sources.list.d/kubernetes.list

deb http://apt.kubernetes.io/ kubernetes-xenial main

EOF

sudo apt-get update

sudo apt-get install -y kubelet kubeadm   # kubectl

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#24-initializing-your-master

sudo kubeadm init --pod-network-cidr=10.244.0.0/  # flannel

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network

# https://github.com/coreos/flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel-rbac.yml

# Waiting for kube-dns ready

sleep 

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#master-isolation

kubectl taint nodes --all node-role.kubernetes.io/master-

# https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#44-joining-your-nodes

curl https://glide.sh/get | sh

通过代理访问API https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#optional-proxying-api-server-to-localhost

各种配置还需要参考： https://kubernetes.io/docs/getting-started-guides/scratch/

yanglin还有一个集群部署的

https://github.com/LinEricYang/kubernetes-vagrant-ansible

vagrant insall 被minikube install 取代 https://kubernetes.io/docs/getting-started-guides/alternatives/

开发者搭建

1. 官方参考文档

https://github.com/kubernetes/community/tree/master/contributors/devel

官方文档源码编译安装运行环境

下载community指导

git clone https://github.com/kubernetes/community.git

下载源码：

git clone https://github.com/kubernetes/kubernetes.git

cd kubernetes

https://github.com/kubernetes/community/blob/master/contributors/guide/github-workflow.md

安装go

apt 安装

sudo add-apt-repository ppa:gophers/archive

sudo apt update

sudo apt-get install golang-1.9-go

echo "export PATH=\$PATH:/usr/lib/go-1.9/bin" >> ~/.profile

source ~/.profile

源码安装：

http://jdstaerk.de/installing-go-1-9-on-ubuntu/、

https://askubuntu.com/questions/959932/installation-instructions-for-golang-1-9-into-ubuntu-16-04

Download golang 1.9 tar from official site. Then extract it into /usr/local, creating a Go tree in /usr/local/goas follows:

tar -C /usr/local -xzf go$VERSION.$OS-$ARCH.tar.gz

After extracting add the following lines to your $HOME/.profile.

# Set GOROOT

export GOROOT=/usr/local/go

export PATH=$GOROOT/bin:$PATH

安装 CFSSL

PKI 工具集 cfssl 来生成 Certificate Authority (CA) 证书和秘钥文件

Kubernetes安装之证书验证

kubernetes1.6 安装之证书（一）

go get -u github.com/cloudflare/cfssl/cmd/...

PATH=$PATH:$GOPATH/bin

安装 etcd

https://github.com/kubernetes/community/blob/master/contributors/devel/testing.md#install-etcd-dependency

hack/install-etcd.sh  # Installs in ./third_party/etcd

echo export PATH="\$PATH:$(pwd)/third_party/etcd" >> ~/.profile  # Add to PATH

Downloading https://github.com/coreos/etcd/releases/download/v3.1.10/etcd-v3.1.10-linux-amd64.tar.gz succeed
etcd v3.1.10 installed. To use:
export PATH=/home/ubuntu/kubernetes/third_party/etcd:${PATH}

测试etcd

http://cizixs.com/2016/08/02/intro-to-etcd

build （可以省略）

可以使用bazel也可以直接make

使用 bazel

需要先安装：

https://docs.bazel.build/versions/master/install.html

http://blog.csdn.net/u010510350/article/details/52247972

update 过程会报错

locale: Cannot set LC_ALL to default locale: No such file or directory

https://askubuntu.com/questions/162391/how-do-i-fix-my-locale-issue

运行k8s 集群

Getting started locally

cd kubernetes
hack/local-up-cluster.sh

修改完代码，重新运行：

cd kubernetes

make

hack/local-up-cluster.sh

output:

~/kubernetes$ ./hack/local-up-cluster.sh                                                                     [/]

WARNING : The kubelet is configured to not fail if swap is enabled; production deployments should disable swap.

WARNING : This script MAY be run as root for docker socket / iptables functionality; if failures occur, retry as root.

make: Entering directory '/home/ubuntu/kubernetes'

make[]: Entering directory '/home/ubuntu/kubernetes'

make[]: Leaving directory '/home/ubuntu/kubernetes'

+++ [ ::] Building the toolchain targets:

    k8s.io/kubernetes/hack/cmd/teststale

    k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata

+++ [ ::] Generating bindata:

    test/e2e/generated/gobindata_util.go

~/kubernetes ~/kubernetes/test/e2e/generated

~/kubernetes/test/e2e/generated

+++ [ ::] Building go targets for linux/amd64:

    cmd/kubectl

    cmd/hyperkube

+++ [ ::] +++ Warning: stdlib pkg with cgo flag not found.

+++ [ ::] +++ Warning: stdlib pkg cannot be rebuilt since /usr/lib/go-1.9/pkg is not writable by ubuntu

+++ [ ::] +++ Warning: Make /usr/lib/go-1.9/pkg writable for ubuntu for a one-time stdlib install, Or

+++ [ ::] +++ Warning: Rebuild stdlib using the command 'CGO_ENABLED=0 go install -a -installsuffix cgo std'

+++ [ ::] +++ Falling back to go build, which is slower

    **

make: Leaving directory '/home/ubuntu/kubernetes'

WARNING: No swap limit support

Kubelet cgroup driver defaulted to use: cgroupfs

API SERVER insecure port is free, proceeding...

API SERVER secure port is free, proceeding...

Detected host and ready to start services.  Doing some housekeeping first...

Using GO_OUT /home/ubuntu/kubernetes/_output/local/bin/linux/amd64                                                          [/]

Starting services now!

Starting etcd

etcd --advertise-client-urls http://127.0.0.1:2379 --data-dir /tmp/tmp.fc4lxZmyBY --listen-client-urls http://127.0.0.1:2379 --debug

 > "/dev/null" >/dev/null

Waiting for etcd to come up.

+++ [ ::] On try , etcd: : http://127.0.0.1:2379

{"action":"set","node":{"key":"/_test","value":"","modifiedIndex":,"createdIndex":}}

Generating a  bit RSA private key

.................+++

...................................................................+++

writing new private key to '/var/run/kubernetes/server-ca.key'

-----

Generating a  bit RSA private key

..................................+++

...............+++

writing new private key to '/var/run/kubernetes/client-ca.key'

-----

Generating a  bit RSA private key

....+++

.........+++

writing new private key to '/var/run/kubernetes/request-header-ca.key'

-----

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request                                                                        [/]

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR                                                                                      [/]

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

Waiting for apiserver to come up

+++ [ ::] On try , apiserver: : ok

Cluster "local-up-cluster" set.

use 'kubectl --kubeconfig=/var/run/kubernetes/admin-kube-aggregator.kubeconfig' to use the aggregated API server

service "kube-dns" created

serviceaccount "kube-dns" created

configmap "kube-dns" created

deployment "kube-dns" created

Kube-dns addon successfully deployed.

kubelet (  ) is running.

Create default storage class for

storageclass "standard" created

Local Kubernetes cluster is running. Press Ctrl-C to shut it down.

Logs:

  /tmp/kube-apiserver.log

  /tmp/kube-controller-manager.log                                                                                           [/]

  /tmp/kube-proxy.log

  /tmp/kube-scheduler.log

  /tmp/kubelet.log

To start using your cluster, you can open up another terminal/tab and run:

  export KUBECONFIG=/var/run/kubernetes/admin.kubeconfig

  cluster/kubectl.sh

Alternatively, you can write to the default kubeconfig:

  export KUBERNETES_PROVIDER=local

  cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/server-ca.

crt

  cluster/kubectl.sh config set-credentials myself --client-key=/var/run/kubernetes/client-admin.key --client-certificate=/var/run/k

ubernetes/client-admin.crt

  cluster/kubectl.sh config set-context local --cluster=local --user=myself

  cluster/kubectl.sh config use-context local

  cluster/kubectl.sh

./hack/local-up-cluster.sh: line :  Killed                  ${CONTROLPLANE_SUDO} "${GO_OUT}/hyperkube" controller-manager -

-v=${LOG_LEVEL} --vmodule="${LOG_SPEC}" --service-account-private-key-file="${SERVICE_ACCOUNT_KEY}" --root-ca-file="${ROOT_CA_FILE}"

 --cluster-signing-cert-file="${CLUSTER_SIGNING_CERT_FILE}" --cluster-signing-key-file="${CLUSTER_SIGNING_KEY_FILE}" --enable-hostpa

th-provisioner="${ENABLE_HOSTPATH_PROVISIONER}" ${node_cidr_args} --pvclaimbinder-sync-period="${CLAIM_BINDER_SYNC_PERIOD}" --featur

e-gates="${FEATURE_GATES}" ${cloud_config_arg} --kubeconfig "$CERT_DIR"/controller.kubeconfig --use-service-account-credentials --co

ntrollers="${KUBE_CONTROLLERS}" --master="https://${API_HOST}:${API_SECURE_PORT}" > "${CTLRMGR_LOG}" >&

debug

$ go get github.com/derekparker/delve/cmd/dlv
$ ps -ef |grep "hyperkube apiserver"
$ sudo sysctl -w kernel.yama.ptrace_scope=0

$ cat >> ~/.bashrc <<<'

GOROOT=`go env |grep "GOROOT" |cut -d "=" -f2`

GOROOT=${GOROOT#\"}

GOROOT=${GOROOT%\"}

GOPATH=`go env |grep GOPATH |cut -d "=" -f 2`

GOPATH=${GOPATH%\"}

GOPATH=${GOPATH#\"}

export PATH="$PATH:$GOROOT/bin:$GOPATH/bin"'

$ source ~/.bashrc

$ sudo su
# echo 0 > /proc/sys/kernel/yama/ptrace_scope
# exit
$ sudo $GOPATH/bin/dlv attach $PID

访问API or access-cluster-api

k8s-api-tutorial

kubectl-vs-http-api

kubernetes-from-the-ground-up-the-api-server

$ curl http://localhost:8080/api/v1/pods

$ CERTDIR=/var/run/kubernetes

$ curl -i https://127.0.0.1:6443/api/v1/pods --cert $CERTDIR/client-admin.crt --key $CERTDIR/client-admin.key --cacert $CERTDIR/server-ca.crt