kubernetes部署注意事项（Kubernetes1.22中apiserver使用token）
kubernetes部署注意事项（Kubernetes1.22中apiserver使用token）HTTP Token认证：通过一个Token来识别合法用户。HTTPS 证书认证：基于CA根证书签名的双向数字证书认证方式HTTP Base认证：通过用户名 密码的方式认证，这个只有1.19之前的版本适用，之后的版本不在支持┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create] └─$kubectl cluster-info Kubernetes control plane is running at https://192.168.26.81:6443 CoreDNS is running at https://192.168.26.81:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy Metrics-server is running
写在前面- 本文偏实战，内容涉及：
- token方式的API Server认证Demo
- Kubeconfig文件方式的API Server认证Demo
- Kubeconfig文件的创建Demo
只有能做到“尽人事而听天命”，一个人才能永远保持心情的平衡。 ----- 《季羡林谈人生》
Kubernetes集群中所有资源的访问和变更都是通过Kubernetes API Server的REST API来实现的 所以集群安全的关键点就在于如何鉴权和授权
一个简单的Demo 在master节点上 我们通过root用户可以直接通kubectl来请求API Service从而获取集群信息 但是我们通过其他用户登录就没有这个权限 这就涉及到k8s的一个认证问题.
root用户可以正常访问
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$kubectl get pods
NAME READY STATUS RESTARTS AGE
liruilong-grafana-5955564c75-zpbjq 3/3 Terminating 0 8h
liruilong-kube-prometheus-operator-5cb699b469-fbkw5 1/1 Terminating 0 8h
liruilong-prometheus-node-exporter-vm7s9 1/1 Terminating 2 (109m ago) 8h
prometheus-liruilong-kube-prometheus-prometheus-0 2/2 Terminating 0 8h
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$
切换tom用户来访问，没有权限，报错找不到集群ＡＰＩ的位置，那么为什么会这样呢？
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$su tom
[tom@vms81 k8s-helm-create]$ kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[tom@vms81 k8s-helm-create]$ exit
exit
为了演示认证，我们需要在集群外的机器上安装一个客户端工具kubectl，用于和集群的入口api-Service交互
┌──[root@liruilongs.github.io]-[~]
└─$ yum install -y kubectl-1.22.2-0 --disableexcludes=kubernetes
可以通过kubectl cluster-info来查看集群的相关信息
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$kubectl cluster-info
Kubernetes control plane is running at https://192.168.26.81:6443
CoreDNS is running at https://192.168.26.81:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://192.168.26.81:6443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
To further debug and diagnose cluster problems use 'kubectl cluster-info dump'.
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$
Kubernetes集群提供了3种级别的客户端身份认证方式
HTTP Token认证：通过一个Token来识别合法用户。
HTTPS 证书认证：基于CA根证书签名的双向数字证书认证方式
HTTP Base认证：通过用户名 密码的方式认证，这个只有1.19之前的版本适用，之后的版本不在支持
下面就Token和SSL和小伙伴分享下，Bash因为在高版本的K8s中不在支持，所以我们这里不聊。关于上面的普通用户范围集群的问题，我们也会改出解答
HTTP Token认证HTTP Token的认证是用一个很长的特殊编码方式的并且难以被模仿的字符串Token来表明客户身份的一种方式。
每个Token对应一个用户名 存储在APIServer能访问的一个文件中。当客户端发起API调用请求时 需要在HTTP Header里放入Token 这样一来 API Server就能识别合法用户和非法用户了。
当 API 服务器的命令行设置了--token-auth-file=SOMEFILE选项时，会从文件中 读取持有者令牌。目前，令牌会长期有效，并且在不重启 API 服务器的情况下 无法更改令牌列表。下面我们一个通过Demo来演示通过静态Token的用户认证
通过openssl生成一个令牌
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$openssl rand -hex 10
4bf636c8214b7ff0a0fb
令牌文件是一个 CSV 文件，包含至少 3 个列：令牌、用户名和用户的 UID。 其余列被视为可选的组名。这里需要注意的是，令牌文件要放到/etc/kubernetes/pki目录下才可以，可能默认读取令牌的位置即是这个位置
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$echo "4bf636c8214b7ff0a0fb admin2 3" > /etc/kubernetes/pki/liruilong.csv
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$cat /etc/kubernetes/pki/liruilong.csv
4bf636c8214b7ff0a0fb admin2 3
通过Sed添加kube-apiserver服务启动参数，- --token-auth-file=/etc/kubernetes/pki/liruilong.csv
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$sed '17a \ \ \ \ - --token-auth-file=/etc/kubernetes/pki/liruilong.csv' /etc/kubernetes/manifests/kube-apiserver.yaml | grep -A 5 command
- command:
- kube-apiserver
- --advertise-address=192.168.26.81
- --allow-privileged=true
- --token-auth-file=/etc/kubernetes/liruilong.csv
- --authorization-mode=Node RBAC
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$sed -i '17a \ \ \ \ - --token-auth-file=/etc/kubernetes/pki/liruilong.csv' /etc/kubernetes/manifests/kube-apiserver.yaml
检查修改的启动参数
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$cat -n /etc/kubernetes/manifests/kube-apiserver.yaml | grep -A 5 command
14 - command:
15 - kube-apiserver
16 - --advertise-address=192.168.26.81
17 - --allow-privileged=true
18 - --token-auth-file=/etc/kubernetes/pki/liruilong.csv
19 - --authorization-mode=Node RBAC
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$
重启kubelet服务
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$systemctl restart kubelet
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-helm-create]
└─$
确认集群能够正常访问
┌──[root@vms81.liruilongs.github.io]-[/etc/kubernetes/pki]
└─$kubectl get nodes
NAME STATUS ROLES AGE VERSION
vms81.liruilongs.github.io Ready control-plane master 34d v1.22.2
vms82.liruilongs.github.io Ready <none> 34d v1.22.2
vms83.liruilongs.github.io NotReady <none> 34d v1.22.2
┌──[root@vms81.liruilongs.github.io]-[/etc/kubernetes/pki]
└─$
在集群外的客户机访问集群信息，这里提示我们admin2用户没有访问的权限，说明已经认证成功了，只是没有权限
┌──[root@liruilongs.github.io]-[~]
└─$ kubectl -s="https://192.168.26.81:6443" --insecure-skip-tls-verify=true --token="4bf636c8214b7ff0a0fb" get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "admin2" cannot list resource "pods" in API group "" in the namespace "kube-system"
┌──[root@liruilongs.github.io]-[~]
└─$
这里我们修改一些token的字符串，Token和集群的Token文件不对应，会提示我们没有获得授权，即认证失败
┌──[root@liruilongs.github.io]-[~]
└─$ kubectl -s="https://192.168.26.81:6443" --insecure-skip-tls-verify=true --token="4bf636c8214b7ff0a0f" get pods -n kube-system
error: You must be logged in to the server (Unauthorized)
kubeconfig文件认证
在回到我们之前的那个问题，为什么使用root用户可以访问集群信息，但是通过tom用户去不能够访问集群信息，这里就涉及到一个kubeconfig 文件认证的问题
在通过kubeadm创建集群的时候，不知道小伙伴没还记不记得下面这个文件admin.conf，这个文件就是kubeadm帮我们生成的kubeconfig文件
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$ll /etc/kubernetes/admin.conf
-rw------- 1 root root 5676 12月 13 02:13 /etc/kubernetes/admin.conf
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$
我们把这个文件拷贝到tom用户的目录下，修改权限
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$cp /etc/kubernetes/admin.conf ~tom/
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$chown tom:tom ~tom/admin.conf
这个时候发现通过 --kubeconfig=admin.conf 指定这个文件，就可以访问集群信息
[tom@vms81 home]$ cd tom/
[tom@vms81 ~]$ ls
admin.conf
[tom@vms81 ~]$ kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[tom@vms81 ~]$ kubectl get pods -A --kubeconfig=admin.conf
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-controller-744d4fc6b7-t9n4l 1/1 Running 6 (8h ago) 44h
kube-system calico-kube-controllers-78d6f96c7b-85rv9 1/1 Running 193 31d
kube-system calico-node-6nfqv 1/1 Running 254 34d
kube-system calico-node-fv458 0/1 Running 50 34d
kube-system calico-node-h5lsq 1/1 Running 94 (7h10m ago) 34d
kube-system ..........................
那个，kubeconfig文件是个什么东西，官方文档中这样描述：
使用 kubeconfig 文件来组织有关集群、用户、命名空间和身份认证机制的信息。kubectl 命令行工具使用 kubeconfig 文件来查找选择集群所需的信息，并与集群的 API 服务器进行通信。
换句话讲，通过kubeconfig与集群的 API 服务器进行通信，类似上面的Token的作用，我们要说的HTTPS证书认证就是放到这里
默认情况下，kubectl 在 $HOME/.kube 目录下查找名为 config 的文件。
┌──[root@vms81.liruilongs.github.io]-[~]
└─$ls ~/.kube/config
/root/.kube/config
┌──[root@vms81.liruilongs.github.io]-[~]
└─$ll ~/.kube/config
-rw------- 1 root root 5663 1月 16 02:33 /root/.kube/config
将kubeconfig文件复制到 $HOME/.kube 目录下改名为 config 发现tom用户依旧可以访问
[tom@vms81 ~]$ ls
admin.conf
[tom@vms81 ~]$ cp admin.conf .kube/config
[tom@vms81 ~]$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-78d6f96c7b-85rv9 1/1 Running 193 31d
calico-node-6nfqv 1/1 Running 254 34d
calico-node-fv458 0/1 Running 50 34d
calico-node-h5lsq 1/1 Running 94 (7h13m ago) 34d
。。。。。。。
也可以通过设置 KUBECONFIG 环境变量或者设置 --kubeconfig参数来指定其他kubeconfig文件。
[tom@vms81 ~]$ export KUBECONFIG=admin.conf
[tom@vms81 ~]$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-78d6f96c7b-85rv9 1/1 Running 193 31d
calico-node-6nfqv 1/1 Running 254 34d
calico-node-fv458 0/1 Running 50 34d
calico-node-h5lsq 1/1 Running 94 (7h11m ago) 34d
..............
当我们什么都不设置时，tom用户获取不到kubeconfig文件，没有认证信息，无法访问
[tom@vms81 ~]$ unset KUBECONFIG
[tom@vms81 ~]$ kubectl get pods -n kube-system
The connection to the server localhost:8080 was refused - did you specify the right host or port?
查看kubeconfig文件的配置信息
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA OMITTED
server: https://192.168.26.81:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
namespace: liruilong-rbac-create
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
┌──[root@vms81.liruilongs.github.io]-[~/.kube]
└─$
所以我们要想访问集群信息，只需要把这个kubeconfig 文件拷贝到客户机上就OK了
创建 kubeconfig 文件一个kubeconfig 文件包括一下几部分：
- 集群信息：
- 集群CA证书
- 集群地址
- 上下文信息
- 所有上下文信息
- 当前上下文
- 用户信息
- 用户CA证书
- 用户私钥
要创建 kubeconfig 文件的话，我们需要一个私钥，以及集群 CA 授权颁发的证书。同理我们不能直接用私钥生成公钥，而必须是用私钥生成证书请求文件(申请书)，然后根据证书请求文件向 CA(权威机构)申请证书(身份证)，CA 审核通过之后会颁发证书。
环境准备
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$kubectl create ns liruilong-rbac-create
namespace/liruilong-rbac-create created
┌──[root@vms81.liruilongs.github.io]-[~/ansible]
└─$mkdir k8s-rbac-create;cd k8s-rbac-create
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl config set-context $(kubectl config current-context) --namespace=liruilong-rbac-create
Context "kubernetes-admin@kubernetes" modified.
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$
申请证书
生成一个 2048 位的 私钥 iruilong.key 文件
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$openssl genrsa -out liruilong.key 2048
Generating RSA private key 2048 bit long modulus
....................
...........................................................................................................
e is 65537 (0x10001)
查看私钥文件
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$cat liruilong.key
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAt9OBnwaA3VdFfjdiurJPtcaiXOGPc1AWFmrlgocq4vT5WZgq
..............................
..................................
LHd0n1yCKpwbYMGghF4iGmEGIIdsCVZP EV6lduPKjqEm9kjuLROKzRZHFoGyASO
Krb3VR4CKHvnZAPVctv7Pu 4JgMliJHl8GVYhqM5UykbLRMdNHSNIQ==
-----END RSA PRIVATE KEY-----
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$
利用刚生成的私有 liruilong.key 生成证书请求文件 liruilong.key：这里CN的值 liruilong，就是后面我们授权的用户。
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$openssl req -new -key liruilong.key -out liruilong.csr -subj "/CN=liruilong/O=cka2020"
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$ls
liruilong.csr liruilong.key
对证书请求文件进行 base64 编码
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$cat liruilong.csr | base64 |tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJ...............
编写申请证书请求文件的 yaml 文件:cat csr.yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: liruilong
spec:
signerName: kubernetes.io/kube-apiserver-client
request: LS0tLS1CRUdJTiBDRVJUSUZJ...............
usages:
- client auth
这里 request 里的是 base64 编码之后的证书请求文件。申请证书
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl apply -f csr.yaml
certificatesigningrequest.certificates.k8s.io/liruilong created
查看已经发出证书申请请求：
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl get csr
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
liruilong 15s kubernetes.io/kube-apiserver-client kubernetes-admin <none> Pending
批准证书：
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl certificate approve liruilong
certificatesigningrequest.certificates.k8s.io/liruilong approved
查看审批通过的证书：
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl get csr/liruilong -o yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"certificates.k8s.io/v1" "kind":"CertificateSigningRequest" "metadata":{"annotations":{} "name":"liruilong"} "spec":{"request":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2F6Q0NBVk1DQVFBd0pqRVNNQkFHQTFVRUF3d0piR2x5ZFdsc2IyNW5NUkF3RGdZRFZRUUtEQWRqYTJFeQpNREl3TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF0OU9CbndhQTNWZEZmamRpCnVySlB0Y2FpWE9HUGMxQVdGbXJsZ29jcTR2VDVXWmdxd1g5T0RvSnpDREJZZVFJQ3h0Wm5uUk9XY1B2dVB6K1IKb1Eybk83K3FnNUNjZzlWZmVOWFRwUDB0VXZsQ21ZVVg2dkRDdlgxUDR3VnNFdXNydlZBdkF4NmdqZTZzNW94VgphZTIwcXFBRXpTUXJhczhPeldsZ1Frd0xjNU5MZ2k3bWlpNHNzaVpQRXU1ZFZIRWs5dHdCeUZTV0dsanJETkhvCnN4UkFFNXlrWjBnODBWSzN1U1JNNmFHSEJ0QmVpbysxa2d0U0xDMlVScy9QWUwwRGNSQm9zUUx0c3JublFSMTkKSE5NWTkweUhYN3Jta3ZqcHdOdkRZWjNIWUVvbGJQZThWZjhBTFpsbDVBTnJ5SUJqbXNrY01QM2lRMzdxWGZUNwptSzhKeHdJREFRQUJvQUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUFwa09aUUNTTGxGYWJTVk9zNmtzL1ZyCmd3b3FTdjFOM1YzUC84SmNzR1pFUTc4TkpJb2R0REExS3EwN25DWjJWUktselZDN1kyMCszZUVzTXZNWnFMc1MKbUtaS0w2SFE3N2RHa1liUjhzKzRMaFo4YXR6cXVMSnlqZUZKODQ2N1ZrUXF5T1R6by9wZ3E4YWJJY01XNzlKMgoxWEkybi92RWlIMEgvWU9DaWExVHRqTnpSWGtlL2hPQTZ4Y29CcVRpdWtkUHBqZDJSaWFTRUNUS1h4ZGNOS0xLCmZVbFhkb2s5UkVkQ2V3bU9ISUdvVG9qUGRWdWlPdkYzZkFqUXZNNDJ3UjJDdklHMWs1YUQzdWVlbzcwd0pnUlQKYzhZNnUwY2padEI5ZW5xUStmRFFqdUUyZElrMDJLbm5HQVppK0wxUnRnSnA2Tm1udEg5WUc3RlBLSXYrakFZPQotLS0tLUVORCBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0K" "signerName":"kubernetes.io/kube-apiserver-client" "usages":["client auth"]}}
creationTimestamp: "2022-01-16T15:25:24Z"
name: liruilong
resourceVersion: "1185668"
uid: 51837659-7214-4dec-bcd4-b7a9129ee2bb
spec:
groups:
- system:masters
- system:authenticated
request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2F6Q0NBVk1DQVFBd0pqRVNNQkFHQTFVRUF3d0piR2x5ZFdsc2IyNW5NUkF3RGdZRFZRUUtEQWRqYTJFeQpNREl3TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF0OU9CbndhQTNWZEZmamRpCnVySlB0Y2FpWE9HUGMxQVdGbXJsZ29jcTR2VDVXWmdxd1g5T0RvSnpDREJZZVFJQ3h0Wm5uUk9XY1B2dVB6K1IKb1Eybk83K3FnNUNjZzlWZmVOWFRwUDB0VXZsQ21ZVVg2dkRDdlgxUDR3VnNFdXNydlZBdkF4NmdqZTZzNW94VgphZTIwcXFBRXpTUXJhczhPeldsZ1Frd0xjNU5MZ2k3bWlpNHNzaVpQRXU1ZFZIRWs5dHdCeUZTV0dsanJETkhvCnN4UkFFNXlrWjBnODBWSzN1U1JNNmFHSEJ0QmVpbysxa2d0U0xDMlVScy9QWUwwRGNSQm9zUUx0c3JublFSMTkKSE5NWTkweUhYN3Jta3ZqcHdOdkRZWjNIWUVvbGJQZThWZjhBTFpsbDVBTnJ5SUJqbXNrY01QM2lRMzdxWGZUNwptSzhKeHdJREFRQUJvQUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUFwa09aUUNTTGxGYWJTVk9zNmtzL1ZyCmd3b3FTdjFOM1YzUC84SmNzR1pFUTc4TkpJb2R0REExS3EwN25DWjJWUktselZDN1kyMCszZUVzTXZNWnFMc1MKbUtaS0w2SFE3N2RHa1liUjhzKzRMaFo4YXR6cXVMSnlqZUZKODQ2N1ZrUXF5T1R6by9wZ3E4YWJJY01XNzlKMgoxWEkybi92RWlIMEgvWU9DaWExVHRqTnpSWGtlL2hPQTZ4Y29CcVRpdWtkUHBqZDJSaWFTRUNUS1h4ZGNOS0xLCmZVbFhkb2s5UkVkQ2V3bU9ISUdvVG9qUGRWdWlPdkYzZkFqUXZNNDJ3UjJDdklHMWs1YUQzdWVlbzcwd0pnUlQKYzhZNnUwY2padEI5ZW5xUStmRFFqdUUyZElrMDJLbm5HQVppK0wxUnRnSnA2Tm1udEg5WUc3RlBLSXYrakFZPQotLS0tLUVORCBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0K
signerName: kubernetes.io/kube-apiserver-client
usages:
- client auth
username: kubernetes-admin
status:
certificate: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDekNDQWZPZ0F3SUJBZ0lRUC9aR05rUjdzVy9sdHhkQTNGQjBoekFOQmdrcWhraUc5dzBCQVFzRkFEQVYKTVJNd0VRWURWUVFERXdwcmRXSmxjbTVsZEdWek1CNFhEVEl5TURFeE5qRTFNakV3TWxvWERUSXpNREV4TmpFMQpNakV3TWxvd0pqRVFNQTRHQTFVRUNoTUhZMnRoTWpBeU1ERVNNQkFHQTFVRUF4TUpiR2x5ZFdsc2IyNW5NSUlCCklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF0OU9CbndhQTNWZEZmamRpdXJKUHRjYWkKWE9HUGMxQVdGbXJsZ29jcTR2VDVXWmdxd1g5T0RvSnpDREJZZVFJQ3h0Wm5uUk9XY1B2dVB6K1JvUTJuTzcrcQpnNUNjZzlWZmVOWFRwUDB0VXZsQ21ZVVg2dkRDdlgxUDR3VnNFdXNydlZBdkF4NmdqZTZzNW94VmFlMjBxcUFFCnpTUXJhczhPeldsZ1Frd0xjNU5MZ2k3bWlpNHNzaVpQRXU1ZFZIRWs5dHdCeUZTV0dsanJETkhvc3hSQUU1eWsKWjBnODBWSzN1U1JNNmFHSEJ0QmVpbysxa2d0U0xDMlVScy9QWUwwRGNSQm9zUUx0c3JublFSMTlITk1ZOTB5SApYN3Jta3ZqcHdOdkRZWjNIWUVvbGJQZThWZjhBTFpsbDVBTnJ5SUJqbXNrY01QM2lRMzdxWGZUN21LOEp4d0lECkFRQUJvMFl3UkRBVEJnTlZIU1VFRERBS0JnZ3JCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTUI4R0ExVWQKSXdRWU1CYUFGR0RjS1N1dVY1TTV5Wk5CR1AxLzZoN0xZNytlTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCagpOelREMmZ5bTc3bXQ4dzlacXRZN3NQelhmNHJQTXpWUzVqV3NzenpidlhEUzhXcFNMWklIYkQ3VU9vYlYxcFYzClYzRW02RXlpWUEvbjhMYTFRMnZra0EyUDk1d3JqWlBuemZIeUhWVFpCTUY4YU1MSHVpVHZ5WlVVV0JYMTg1UFAKQ2MxRncwanNmVThJMDBzbUNOeURBZjVMejFjRUVrNWlGYUswMDJRblUyNk5lcDF3U3BMcVZWWVptSW9UVU9DOApCNzNpU3J6Y0wyVmdBejRCaUQxdUVlUkFMM20zRTB2VVpsQjduKzF1MllrNDFCajdGYnpWR2w1dFpYT3hDMVhxCjJVc0hSbmkzY1VYZ203QlloZDU3aTFHclRRRFJpckRwVFV1RDB3ZlFYTjZLdEx1TmVDYUc0alc4ZTl4QkQrTjIKOFE4Z25UZjdPSEI3VWZkUzVnMWQKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
conditions:
- lastTransitionTime: "2022-01-16T15:26:02Z"
lastUpdateTime: "2022-01-16T15:26:01Z"
message: This CSR was approved by kubectl certificate approve.
reason: KubectlApprove
status: "True"
type: Approved
导出证书文件：
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl get csr liruilong -o jsonpath='{.status.certificate}'| base64 -d > liruilong.crt
给用户授权，这里给 liruilong 一个集群角色 cluster-role(类似于root一样的角色)，这样 liruilong 具有管理员权限
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl create clusterrolebinding test --clusterrole=cluster-admin --user=liruilong
clusterrolebinding.rbac.authorization.k8s.io/test created
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$
创建 kubeconfig 文件
拷贝 CA 证书
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$ls
csr.yaml #(申请证书请求文件yaml)
liruilong.crt #公钥(证书文件)
liruilong.csr #(证书请求文件)
liruilong.key #私钥
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$ls /etc/kubernetes/pki/
apiserver.crt apiserver.key ca.crt front-proxy-ca.crt front-proxy-client.key sa.pub
apiserver-etcd-client.crt apiserver-kubelet-client.crt ca.key front-proxy-ca.key liruilong.csv
apiserver-etcd-client.key apiserver-kubelet-client.key etcd front-proxy-client.crt sa.key
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$cp /etc/kubernetes/pki/ca.crt .
设置集群字段，这里包含集群名字，服务地址和集群证书
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl config --kubeconfig=kc1 set-cluster cluster1 --server=https://192.168.26.81:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "cluster1" set.
在上面集群中创建一个上下文context1
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl config --kubeconfig=kc1 set-context context1 --cluster=cluster1 --namespace=default --user=liruilong
Context "context1" created.
这里–embed-certs=true 的意思是把证书内容写入到此 kubeconfig 文件里。
设置用户字段，包含用户名字，用户证书，用户私钥
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl config --kubeconfig=kc1 set-credentials liruilong --client-certificate=liruilong.crt --client-key=liruilong.key --embed-certs=true
User "liruilong" set.
查看创建的kubeconfig文件信息
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$cat kc1
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1USXhNakUyTURBME1sb1hEVE14TVRJeE1ERTJNREEwTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTkdkCisrWnhFRDJRQlR2Rm5ycDRLNFBrd2lsYXUrNjdXNTVobVdwc09KSHF6ckVoWUREY3l4ZTU2Z1VJVDFCUTFwbU0KcGFrM0V4L0JZRStPeHY4ZmxtellGbzRObDZXQjl4VXovTW5HQi96dHZsTGpaVEVHZy9SVlNIZTJweCs2MUlSMQo2Mkh2OEpJbkNDUFhXN0pmR3VXNDdKTXFUNTUrZUNuR00vMCtGdnI2QUJnT2YwNjBSSFFuaVlzeGtpSVJmcjExClVmcnlPK0RFTGJmWjFWeDhnbi9tcGZEZ044cFgrVk9FNFdHSDVLejMyNDJtWGJnL3A0emd3N2NSalpSWUtnVlUKK2VNeVIyK3pwaTBhWW95L2hLYmg4RGRUZ3FZeERDMzR6NHFoQ3RGQnVia1hmb3Ftc3FGNXpQUm1ZS051RUgzVAo2c1FNSFl4emZXRkZvSGQ2Y0JNQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZHRGNLU3V1VjVNNXlaTkJHUDEvNmg3TFk3K2VNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBRVE0SUJhM0hBTFB4OUVGWnoyZQpoSXZkcmw1U0xlanppMzkraTdheC8xb01SUGZacElwTzZ2dWlVdHExVTQ2V0RscTd4TlFhbVVQSFJSY1RrZHZhCkxkUzM5Y1UrVzk5K3lDdXdqL1ZrdzdZUkpIY0p1WCtxT1NTcGVzb3lrOU16NmZxNytJUU9lcVRTbGpWWDJDS2sKUFZxd3FVUFNNbHFNOURMa0JmNzZXYVlyWUxCc01EdzNRZ3N1VTdMWmg5bE5TYVduSzFoR0JKTnRndjAxdS9MWAo0TnhKY3pFbzBOZGF1OEJSdUlMZHR1dTFDdEFhT21CQ2ZjeTBoZHkzVTdnQXh5blR6YU1zSFFTamIza0JDMkY5CkpWSnJNN1FULytoMStsOFhJQ3ZLVzlNM1FlR0diYm13Z1lLYnMvekswWmc1TE5sLzFJVThaTUpPREhTVVBlckQKU09ZPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.26.81:6443
name: cluster1
contexts:
- context:
cluster: cluster1
namespace: default
user: liruilong
name: context1
current-context: ""
kind: Config
preferences: {}
users:
- name: liruilong
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDekNDQWZPZ0F3SUJBZ0lRUC9aR05rUjdzVy9sdHhkQTNGQjBoekFOQmdrcWhraUc5dzBCQVFzRkFEQVYKTVJNd0VRWURWUVFERXdwcmRXSmxjbTVsZEdWek1CNFhEVEl5TURFeE5qRTFNakV3TWxvWERUSXpNREV4TmpFMQpNakV3TWxvd0pqRVFNQTRHQTFVRUNoTUhZMnRoTWpBeU1ERVNNQkFHQTFVRUF4TUpiR2x5ZFdsc2IyNW5NSUlCCklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF0OU9CbndhQTNWZEZmamRpdXJKUHRjYWkKWE9HUGMxQVdGbXJsZ29jcTR2VDVXWmdxd1g5T0RvSnpDREJZZVFJQ3h0Wm5uUk9XY1B2dVB6K1JvUTJuTzcrcQpnNUNjZzlWZmVOWFRwUDB0VXZsQ21ZVVg2dkRDdlgxUDR3VnNFdXNydlZBdkF4NmdqZTZzNW94VmFlMjBxcUFFCnpTUXJhczhPeldsZ1Frd0xjNU5MZ2k3bWlpNHNzaVpQRXU1ZFZIRWs5dHdCeUZTV0dsanJETkhvc3hSQUU1eWsKWjBnODBWSzN1U1JNNmFHSEJ0QmVpbysxa2d0U0xDMlVScy9QWUwwRGNSQm9zUUx0c3JublFSMTlITk1ZOTB5SApYN3Jta3ZqcHdOdkRZWjNIWUVvbGJQZThWZjhBTFpsbDVBTnJ5SUJqbXNrY01QM2lRMzdxWGZUN21LOEp4d0lECkFRQUJvMFl3UkRBVEJnTlZIU1VFRERBS0JnZ3JCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTUI4R0ExVWQKSXdRWU1CYUFGR0RjS1N1dVY1TTV5Wk5CR1AxLzZoN0xZNytlTUEwR0NTcUdTSWIzRFFFQkN3VUFBNElCQVFCagpOelREMmZ5bTc3bXQ4dzlacXRZN3NQelhmNHJQTXpWUzVqV3NzenpidlhEUzhXcFNMWklIYkQ3VU9vYlYxcFYzClYzRW02RXlpWUEvbjhMYTFRMnZra0EyUDk1d3JqWlBuemZIeUhWVFpCTUY4YU1MSHVpVHZ5WlVVV0JYMTg1UFAKQ2MxRncwanNmVThJMDBzbUNOeURBZjVMejFjRUVrNWlGYUswMDJRblUyNk5lcDF3U3BMcVZWWVptSW9UVU9DOApCNzNpU3J6Y0wyVmdBejRCaUQxdUVlUkFMM20zRTB2VVpsQjduKzF1MllrNDFCajdGYnpWR2w1dFpYT3hDMVhxCjJVc0hSbmkzY1VYZ203QlloZDU3aTFHclRRRFJpckRwVFV1RDB3ZlFYTjZLdEx1TmVDYUc0alc4ZTl4QkQrTjIKOFE4Z25UZjdPSEI3VWZkUzVnMWQKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdDlPQm53YUEzVmRGZmpkaXVySlB0Y2FpWE9HUGMxQVdGbXJsZ29jcTR2VDVXWmdxCndYOU9Eb0p6Q0RCWWVRSUN4dFpublJPV2NQdnVQeitSb1Eybk83K3FnNUNjZzlWZmVOWFRwUDB0VXZsQ21ZVVgKNnZEQ3ZYMVA0d1ZzRXVzcnZWQXZBeDZnamU2czVveFZhZTIwcXFBRXpTUXJhczhPeldsZ1Frd0xjNU5MZ2k3bQppaTRzc2laUEV1NWRWSEVrOXR3QnlGU1dHbGpyRE5Ib3N4UkFFNXlrWjBnODBWSzN1U1JNNmFHSEJ0QmVpbysxCmtndFNMQzJVUnMvUFlMMERjUkJvc1FMdHNybm5RUjE5SE5NWTkweUhYN3Jta3ZqcHdOdkRZWjNIWUVvbGJQZTgKVmY4QUxabGw1QU5yeUlCam1za2NNUDNpUTM3cVhmVDdtSzhKeHdJREFRQUJBb0lCQUJsS0I3TVErZmw1WUI0VgpFSWdPcjlpYUV3d2tHOUFKWElDSkJEb0l6bVdhdmhNTlZCUjZwd3BuOTl0UWkxdGFZM2RuVjZuTVlBMzdHck9vCjB5Z004TXpQZVczUUh6Z2p5cGFkRkJqR204Mm1iUHNoekVDT0RyeHkyT0txaEV1MS9yWjBxWU1NVzVvckU2NUQKOEJ3NmozaEp1MTlkY251bk1Lb2hyUlJ4MGNGOGJrVHpRcWk1Y0xGZ0lBUlArNklFcnQrS0FVSGFVQ0wvSUtTYgoyblJnSmhvSWszTUJnQnM3eVl0NFpVNlpkSVFLRU56RzRUd2VjNG5ESXE2TkllU1ZpSEVtbmdjRXVOdTNOaWEvCmxwTWd5ZHJKZWJKYStrc0FiZU4zU21TRGc2MXpVWUpHV2FOZUFPemdmbkZRTVp2Y2FyNEQ1b0xCQUE5Rlpic0EKc0hUZjBBRUNnWUVBNWVmdDJSV25kM2pueWJBVVExa3dRTzlzN1BmK241OXF3ekIwaEdrcC9qcjh2dGRYZkxMNgo0THBoNWFhKzlEM0w5TEpMdk8xa0cyaG5UbFE4Q25uVXpVUUUreldiYlRJblR0eE1hRzB5VjJlZ3NaQkFseERYClk1K2tZZ29EVXIyN2dWbE5QZ29SWVkzRG1ZZHplVmp0NEFHOE9JNWRPUlJ6bFE3VHN0Nk9XUUVDZ1lFQXpMQ3kKM2Q0SkRZRjBpeXFFc2FsNFNsckdEc3hmY2xxeUZaVWpmcUcvdGFHeEFTeE9zL0h3SDNwR1cwQXh3c2tPdVNkUwpWcGxOOC9uZjBMQjdPV0hQL2FjTlJLbHdJeUZrNG9EU0VMOVJ5d0FFVEF3NHdrYitoRzNZdUFHU2YvY1dEWXNtCjNJUUxlMVdFS0JSZDVBS1lkYXdyYlJtclFkSndSaFFkalNzOTJzY0NnWUVBck1WSVpwVHhUc1ViV3VQcHRscjEKK2paekl2bVM3WjI5ZTRXVWFsVWxhNW9raWI0R1R2MnBydXdoMlpVZmR5aGhkemZ0MXNLSE1sbVpHTElRbE1iTgpkcHdoS2k4MDZEQ0NmYTdyOUtYcTZPaEZTR3JoUHlVMjEvVUdjVzZZNUxzVWg3WDJhQ0xrd096cUN4eFJXT1hOCmpVT0FrUGZiY3FPOTRFeE9KdU05RWdFQ2dZRUFwNUVqN0xPL0wzcENBVWVlZDU3bjVkN244dWRtWDhSVnM0dHoKRWxDeUU2dzVybDhxVXUrR0J3N2ZtQVkyZG1LSUZoVlZ0NlVyQnNjUmJkTjhIUjZ3MmRNdTduM1RXajhWU3NQdwp0RnNiUjVkTTdVQzRHbnRxRXRtbUtBVEpmTTYzRkFGTm9Bck5KM3Q3aENBZ09PL1RCY29iaHVZVHAvL3hmNzBwCjhBNXRSYk1DZ1lCMkJTb0ZTbW1sVFN5RjdnZnhmM281dTNXM3lwTENIRFV0cFZkYlAxZm9vemJwZUs3V29IY2YKTEhkMG4xeUNLcHdiWU1HZ2hGNGlHbUVHSUlkc0NWWlArRVY2bGR1UEtqcUVtOWtqdUxST0t6UlpIRm9HeUFTTwpLcmIzVlI0Q0tIdm5aQVBWY3R2N1B1KzRKZ01saUpIbDhHVllocU01VXlrYkxSTWROSFNOSVE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$
修改kubeconfig文件当前的上下文为之前创建的上下文
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$sed 's#current-context: ""#current-context: "context1"#' kc1 | grep current-context
current-context: "context1"
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$sed -i 's#current-context: ""#current-context: "context1"#' kc1
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$cat kc1 | grep current-context
current-context: "context1"
这样 kubeconfig 文件就创建完毕了，下面开始验证 kubeconfig 文件。
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl auth can-i list pods --as liruilong #检查是否具有 list 当前命名空间里的 pod 的权限
yes
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$kubectl auth can-i list pods -n kube-system --as liruilong #检查 是否具有 list 命名空间 kube-system 里 pod 的权限
yes
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$
拷贝证书到客户机
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create]
└─$scp kc1 root@192.168.26.55:~
客户机指定证书访问测试
┌──[root@liruilongs.github.io]-[~]
└─$ kubectl --kubeconfig=kc1 get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-78d6f96c7b-85rv9 1/1 Running 194 (14h ago) 33d
calico-node-6nfqv 0/1 Running 255 (14h ago) 35d
calico-node-fv458 0/1 Running 50 35d
calico-node-h5lsq 1/1 Running 94 (38h ago) 35d
。。。。。。。。。。。。
┌──[root@liruilongs.github.io]-[~]
└─$
这样一个kubeconfig文件就创建完成
