首先从左到右看,上面是Dev 也可以是Ops,通过kubectl已经配置文件和API-Server交互,
下面是pod,在集群内部通过service account和API-Server 交互。
然后依次经过了身份验证(authentication)、授权(authorization)和准入控制(admission control)

Authentication:即身份验证,这个环节它面对的输入是整个http request,它负责对来自client的请求进行身份校验,支持的方法包括:client证书验证(https双向验证)、basic auth、普通token以及jwt token(用于serviceaccount)。APIServer启动时,可以指定一种Authentication方法,也可以指定多种方法。如果指定了多种方法,那么APIServer将会逐个使用这些方法对客户端请求进行验证,只要请求数据通过其中一种方法的验证,APIServer就会认为Authentication成功;在较新版本kubeadm引导启动的k8s集群的apiserver初始配置中,默认支持client证书验证和serviceaccount两种身份验证方式。在这个环节,apiserver会通过client证书或http header中的字段(比如serviceaccount的jwt token)来识别出请求的“用户身份”,包括”user”、”group”等,这些信息将在后面的authorization环节用到。

Authorization:授权。这个环节面对的输入是http request context中的各种属性,包括:user、group、request path(比如:/api/v1、/healthz、/version等)、request verb(比如:get、list、create等)。APIServer会将这些属性值与事先配置好的访问策略(access policy)相比较。APIServer支持多种authorization mode,包括Node、RBAC、Webhook等。APIServer启动时,可以指定一种authorization mode,也可以指定多种authorization mode,如果是后者,只要Request通过了其中一种mode的授权,那么该环节的最终结果就是授权成功。在较新版本kubeadm引导启动的k8s集群的apiserver初始配置中,authorization-mode的默认配置是”Node,RBAC”。Node授权器主要用于各个node上的kubelet访问apiserver时使用的,其他一般均由RBAC授权器来授权。

API Server 可以通过-enable-admission-plugins参数指定加载的授权插件
AlwaysAdmit:允许所有请求通过
AlwaysPullImages:在启动容器之前总是去下载镜像,相当于每当容器启动前做一次用于是否有权使用该容器镜像的检查
AlwaysDeny:禁止所有请求通过,用于测试
DenyEscalatingExec:拒绝exec和attach命令到有升级特权的Pod的终端用户访问。如果集中包含升级特权的容器,而要限制终端用户在这些容器中执行命令的能力,推荐使用此插件
ImagePolicyWebhook
ServiceAccount:这个插件实现了serviceAccounts等等自动化,如果使用ServiceAccount对象,强烈推荐使用这个插件
SecurityContextDeny:将Pod定义中定义了的SecurityContext选项全部失效。SecurityContext包含在容器中定义了操作系统级别的安全选型如fsGroup,selinux等选项
ResourceQuota:用于namespace上的配额管理,它会观察进入的请求,确保在namespace上的配额不超标。推荐将这个插件放到准入控制器列表的最后一个。ResourceQuota准入控制器既可以限制某个namespace中创建资源的数量,又可以限制某个namespace中被Pod请求的资源总量。ResourceQuota准入控制器和ResourceQuota资源对象一起可以实现资源配额管理。
LimitRanger:用于Pod和容器上的配额管理,它会观察进入的请求,确保Pod和容器上的配额不会超标。准入控制器LimitRanger和资源对象LimitRange一起实现资源限制管理
NamespaceLifecycle:当一个请求是在一个不存在的namespace下创建资源对象时,该请求会被拒绝。当删除一个namespace时,将会删除该namespace下的所有资源对象
DefaultStorageClass
DefaultTolerationSeconds
PodSecurityPolicy

Client Key

首先我们看一下kubectl的默认配置文件~/.kube/config

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://172.16.66.101:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED

这有三个重要的信息 user client-certificate-data, client-key-data,首先把data提取出来

1
2
3
4
5
grep 'client-certificate-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.crt
grep 'client-key-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.key
#openssl x509 -noout -text --in kubecfg.crt

Subject: O=system:masters, CN=kubernetes-admin

1
2
3
4
5
6
7
8
9
10
11
12
kubectl -n kube-system describe clusterrolebindings cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate=true
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:masters

Service account

当创建一个Service account会生成一个token,通过
kubectl -n kube-system describe secrets ${Service account}可以查看
得到TOKEN之后即可通过安全端口访问 8080端口没有认证,可以用http访问
curl –insecure -H “Authorization: Bearer $TOKEN” https://$MASTER:6443/api

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
kubectl -n kube-system get roles,clusterroles,rolebindings,clusterrolebindings
NAME AGE
role.rbac.authorization.k8s.io/extension-apiserver-authentication-reader 24m
role.rbac.authorization.k8s.io/kubeadm:kubelet-config-1.11 24m
role.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager 24m
role.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler 24m
role.rbac.authorization.k8s.io/system:controller:bootstrap-signer 24m
role.rbac.authorization.k8s.io/system:controller:cloud-provider 24m
role.rbac.authorization.k8s.io/system:controller:token-cleaner 24m
NAME AGE
clusterrole.rbac.authorization.k8s.io/admin 24m
clusterrole.rbac.authorization.k8s.io/cluster-admin 24m
clusterrole.rbac.authorization.k8s.io/edit 24m
clusterrole.rbac.authorization.k8s.io/flannel 23m
clusterrole.rbac.authorization.k8s.io/system:aggregate-to-admin 24m
clusterrole.rbac.authorization.k8s.io/system:aggregate-to-edit 24m
clusterrole.rbac.authorization.k8s.io/system:aggregate-to-view 24m
clusterrole.rbac.authorization.k8s.io/system:auth-delegator 24m
clusterrole.rbac.authorization.k8s.io/system:aws-cloud-provider 24m
clusterrole.rbac.authorization.k8s.io/system:basic-user 24m
clusterrole.rbac.authorization.k8s.io/system:certificates.k8s.io:certificatesigningrequests:nodeclient 24m
clusterrole.rbac.authorization.k8s.io/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 24m
clusterrole.rbac.authorization.k8s.io/system:controller:attachdetach-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:certificate-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:cronjob-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:daemon-set-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:deployment-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:disruption-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:endpoint-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:expand-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:generic-garbage-collector 24m
clusterrole.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler 24m
clusterrole.rbac.authorization.k8s.io/system:controller:job-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:namespace-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:node-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:persistent-volume-binder 24m
clusterrole.rbac.authorization.k8s.io/system:controller:pod-garbage-collector 24m
clusterrole.rbac.authorization.k8s.io/system:controller:pv-protection-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:pvc-protection-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:replicaset-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:replication-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:resourcequota-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:route-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:service-account-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:service-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:statefulset-controller 24m
clusterrole.rbac.authorization.k8s.io/system:controller:ttl-controller 24m
clusterrole.rbac.authorization.k8s.io/system:coredns 24m
clusterrole.rbac.authorization.k8s.io/system:csi-external-attacher 24m
clusterrole.rbac.authorization.k8s.io/system:csi-external-provisioner 24m
clusterrole.rbac.authorization.k8s.io/system:discovery 24m
clusterrole.rbac.authorization.k8s.io/system:heapster 24m
clusterrole.rbac.authorization.k8s.io/system:kube-aggregator 24m
clusterrole.rbac.authorization.k8s.io/system:kube-controller-manager 24m
clusterrole.rbac.authorization.k8s.io/system:kube-dns 24m
clusterrole.rbac.authorization.k8s.io/system:kube-scheduler 24m
clusterrole.rbac.authorization.k8s.io/system:kubelet-api-admin 24m
clusterrole.rbac.authorization.k8s.io/system:node 24m
clusterrole.rbac.authorization.k8s.io/system:node-bootstrapper 24m
clusterrole.rbac.authorization.k8s.io/system:node-problem-detector 24m
clusterrole.rbac.authorization.k8s.io/system:node-proxier 24m
clusterrole.rbac.authorization.k8s.io/system:persistent-volume-provisioner 24m
clusterrole.rbac.authorization.k8s.io/system:volume-scheduler 24m
clusterrole.rbac.authorization.k8s.io/view 24m
NAME AGE
rolebinding.rbac.authorization.k8s.io/kubeadm:kubelet-config-1.11 24m
rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager 24m
rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler 24m
rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer 24m
rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider 24m
rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner 24m
NAME AGE
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin 24m
clusterrolebinding.rbac.authorization.k8s.io/flannel 23m
clusterrolebinding.rbac.authorization.k8s.io/kubeadm:kubelet-bootstrap 24m
clusterrolebinding.rbac.authorization.k8s.io/kubeadm:node-autoapprove-bootstrap 24m
clusterrolebinding.rbac.authorization.k8s.io/kubeadm:node-autoapprove-certificate-rotation 24m
clusterrolebinding.rbac.authorization.k8s.io/kubeadm:node-proxier 24m
clusterrolebinding.rbac.authorization.k8s.io/system:aws-cloud-provider 24m
clusterrolebinding.rbac.authorization.k8s.io/system:basic-user 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:attachdetach-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:certificate-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:cronjob-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:daemon-set-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:deployment-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:disruption-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:endpoint-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:expand-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:generic-garbage-collector 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:job-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:namespace-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:node-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:persistent-volume-binder 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:pod-garbage-collector 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:pv-protection-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:pvc-protection-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:replicaset-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:replication-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:resourcequota-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:route-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-account-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:statefulset-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:controller:ttl-controller 24m
clusterrolebinding.rbac.authorization.k8s.io/system:coredns 24m
clusterrolebinding.rbac.authorization.k8s.io/system:discovery 24m
clusterrolebinding.rbac.authorization.k8s.io/system:kube-controller-manager 24m
clusterrolebinding.rbac.authorization.k8s.io/system:kube-dns 24m
clusterrolebinding.rbac.authorization.k8s.io/system:kube-scheduler 24m
clusterrolebinding.rbac.authorization.k8s.io/system:node 24m
clusterrolebinding.rbac.authorization.k8s.io/system:node-proxier 24m
clusterrolebinding.rbac.authorization.k8s.io/system:volume-scheduler 24m

下面我们来看一下例子

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
---
# ------------------- ClusterRole ------------------- #
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
---
# ------------------- ClusterRoleBinding ------------------- #
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
---
# ------------------- ServiceAccount ------------------- #
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system

role 和 clusterrole 规定了权限的内容,也就是某个人或者角色能干什么
Service account 也就是具体的角色或者账户
rolebinding 和 clusterrolebinding 就是将两者结合在一起,规定什么人有什么样的权限

当然带有cluster都是集群范围的,不没有cluster的都是命名空间内(比如现实中的部门)的

https://tonybai.com/2016/11/25/the-security-settings-for-kubernetes-cluster/
https://github.com/kubernetes/kubernetes/blob/2cd6cd6e860bcf24d75d53b28b9a30c70e828d49/pkg/master/ports/ports.go
https://www.cnblogs.com/RainingNight/p/deploying-k8s-dashboard-ui.html
https://blog.qikqiak.com/post/use-rbac-in-k8s/
https://mritd.me/2018/03/20/use-rbac-to-control-kubectl-permissions/
https://blog.frognew.com/2017/04/kubernetes-1.6-rbac.html