当有多个可用的 Pod Security Policy 时 k8s 的 PSP 选择策略

前言

简单记录一下，当有多个可用的 psp 时，k8s 会为当前 pod 选择哪个 psp，即 psp 的选择策略是啥。

官方文档的介绍

官方文档 对这种情况的介绍如下：

Policy Order

In addition to restricting pod creation and update, pod security policies can also be used to provide default values for many of the fields that it controls. When multiple policies are available, the pod security policy controller selects policies according to the following criteria:

  1. PodSecurityPolicies which allow the pod as-is, without changing defaults or mutating the pod, are preferred. The order of these non-mutating PodSecurityPolicies doesn’t matter.
  2. If the pod must be defaulted or mutated, the first PodSecurityPolicy (ordered by name) to allow the pod is selected.

Note: During update operations (during which mutations to pod specs are disallowed) only non-mutating PodSecurityPolicies are used to validate the pod.

简单来说就是：

  • 优先考虑不用修改默认值（修改 pod/container 的 securityContext 值）或改变 pod spec（增加安全相关注解等） 的 psp（详见 源码 源码 ），如果没有满足不变条件的 psp 则选择按名称从小到大(a-z)排序后的第一个 psp。
  • 当然，还有一个没有明说的策略是，可供选择的 psp 必须满足 pod 声明所需的安全相关需求（securityContext 中定义的），对于不满足需求的 psp 自然是直接就过滤掉了不会参与上面的选择。

下面我们来做一些实验来验证这几个选择策略。

按名称排序选择第一个

首先，我们定义以下两个 PSP 并且配置 default namespace 的 psp serviceaccount 可以使用这两个 psp （假设文件名叫 psp.yaml）:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp-a
  annotations:
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
spec:
  privileged: false
  runAsUser:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  volumes:
  - configMap
  - emptyDir
  - secret
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp-b
  annotations:
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
spec:
  privileged: false
  runAsUser:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  volumes:
  - configMap
  - emptyDir
  - secret
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: psp:test
  namespace: default
rules:
- apiGroups:
  - policy
  resourceNames:
  - psp-b
  - psp-a
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:test:binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: psp:test
subjects:
  - kind: ServiceAccount
    name: psp
    namespace: default

应用一下上面的 yaml 文件:

$ kubectl create sa psp
serviceaccount/psp created
$ kubectl apply -f psp.yaml
podsecuritypolicy.policy/psp-a created
podsecuritypolicy.policy/psp-b created
role.rbac.authorization.k8s.io/psp:test created
rolebinding.rbac.authorization.k8s.io/psp:test:binding created
$ kubectl get psp
NAME    PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
psp-a   false          RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            configMap,emptyDir,secret
psp-b   false          RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            configMap,emptyDir,secret

然后创建一个 deployment，看看出来的 pod 实际会用哪个 psp （假设文件名称叫 deployment.yaml）:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: test
  name: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      serviceAccountName: psp
      containers:
      - image: busybox
        name: busybox
        command:
          - sleep
          - "233666"

创建 deployment 并查看生成的 pod 使用的 psp:

$ kubectl apply -f deployment.yaml
deployment.apps/test created
$ kubectl get pod -l app=test
NAME                    READY   STATUS              RESTARTS   AGE
test-7ff7bc8569-s9487   0/1     ContainerCreating   0          12s
$ kubectl get pod test-7ff7bc8569-s9487 -o jsonpath='{.metadata.annotations}'
map[kubernetes.io/psp:psp-a seccomp.security.alpha.kubernetes.io/pod:docker/default]

通过 pod 的注解我们可以知道这个 pod 使用的 psp 是 psp-a ， psp-a 和 psp-b 两个定义的 psp 策略完全一样，当时却使用了 psp-a 符合第二个按名称排序选择第一个的选择策略。

下面我们来验证一下第一个策略。

优先考虑不用修改默认值或改变 pod 的 psp

前面定义的 psp-a 和 psp-b 中的注解 seccomp.security.alpha.kubernetes.io/defaultProfileName 和 seccomp.security.alpha.kubernetes.io/allowedProfileNames 就是一个会改变 pod 的规则，下面删除一下 psp-b 中的这两个注解，按照选择策略，这次应该会使用 psp-b 这个 psp。

修改 psp-b 删除 seccomp 注解:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp-a
  annotations:
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
spec:
  privileged: false
  runAsUser:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  volumes:
  - configMap
  - emptyDir
  - secret
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp-b
spec:
  privileged: false
  runAsUser:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  volumes:
  - configMap
  - emptyDir
  - secret
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: psp:test
  namespace: default
rules:
- apiGroups:
  - policy
  resourceNames:
  - psp-b
  - psp-a
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:test:binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: psp:test
subjects:
  - kind: ServiceAccount
    name: psp
    namespace: default

更新一下已有的 psp:

$ kubectl apply -f psp.yaml
podsecuritypolicy.policy/psp-a configured
podsecuritypolicy.policy/psp-b configured
role.rbac.authorization.k8s.io/psp:test unchanged
rolebinding.rbac.authorization.k8s.io/psp:test:binding unchanged

还是使用前面的 deployment.yaml 来测试:

$ kubectl delete -f deployment.yaml
deployment.apps "test" deleted
$ kubectl apply -f deployment.yaml
deployment.apps/test created
$ kubectl get pod -l app=test
NAME                    READY   STATUS              RESTARTS   AGE
test-7ff7bc8569-bc62g   0/1     ContainerCreating   0          8s
$ kubectl get pod test-7ff7bc8569-bc62g -o jsonpath='{.metadata.annotations}'
map[kubernetes.io/psp:psp-b]

可以看到，这次确实是使用了 psp-b 这个 psp，验证了第一个选择策略。

Comments