How Kyverno helps with policy management
The proliferation of the cloud and Kubernetes made it "easier" to provision new environments dynamically, democratizing the access to resources that can be used, for example, for testing. That comes with its own set of challenges; dynamically provisioning Roles and RoleBindings being some of them. If we think of a new "environment" as a namespace in Kubernetes, each of them will have its own Roles and RoleBindings and we would like for them to be managed dynamically.
We could attack this problem with several approaches but we'll leverage Kyverno and one of its advanced features to manage that for us. Kyverno "is a policy engine designed for Kubernetes. (...) Kyverno policies can validate, mutate, and generate Kubernetes resources". Kyverno can:
- validate - check resource configurations for policy compliance;
- mutate - modify resources during admission control;
- generate - create additional resources based on resource creation or updates.
We will leverage Kyverno's generate capabilities to create Roles and RoleBindings every time a new namespace is created.
Get the ball rolling
We will be spinning up a local Kubernetes cluster with k3d and we'll need the following tools to make it happen:
We start by creating a local Kubernetes cluster with k3d:
k3d cluster create -a 2 kyvernok3d kubeconfig merge kyverno --switch-context
We then deploy Kyverno into the cluster:
kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/definitions/release/install.yaml
Because we're generating resources we need to give Kyverno permissions to do so:
# kyverno.yamlapiVersion: rbac.authorization.k8s.io/v1beta1kind: ClusterRolemetadata:name: kyverno:generatecontrollerrules:- apiGroups:- "*"resources:- namespaces- networkpolicies- secrets- configmaps- resourcequotas- limitranges- pods- roles- rolebindingsverbs:- create- get- update- delete- watch---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata:name: kyverno-admin-generateroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: kyverno:generatecontrollersubjects:- kind: ServiceAccountname: kyverno-service-accountnamespace: kyverno
There's a bit more than what we need but we can go ahead and apply these permissions:
kubectl apply -f kyverno.yaml
Now that Kyverno is equipped with the necessary permissions to manage our Roles and RoleBindings we need to create the necessary policies that handle that:
# role-policy.yamlapiVersion: kyverno.io/v1kind: ClusterPolicymetadata:name: rolespec:rules:- name: rolematch:resources:kinds:- Namespaceexclude:resources:namespaces:- "kube-system"- "default"- "kube-public"- "kyverno"generate:kind: Rolename: pod-readernamespace: "{{request.object.metadata.name}}"data:rules:- apiGroups: [""] # "" indicates the core API groupresources: ["pods"]verbs: ["get", "watch", "list"]---# rolebinding-policy.yamlapiVersion: kyverno.io/v1kind: ClusterPolicymetadata:name: rolebindingspec:rules:- name: rolebindingmatch:resources:kinds:- Namespaceexclude:resources:namespaces:- "kube-system"- "default"- "kube-public"- "kyverno"generate:kind: RoleBindingname: read-podsnamespace: "{{request.object.metadata.name}}"data:subjects:- kind: Username: "{{request.object.metadata.labels.user}}"apiGroup: rbac.authorization.k8s.ioroleRef:kind: Rolename: pod-readerapiGroup: rbac.authorization.k8s.io
To achieve our goal we create two policies, one to create a Role and another for a RoleBinding where Kyverno will be "listening" for changes to Namespace resources (excluding the ones it doesn't care about). The {{request.object}} contains the fields of the Namespace resource and Kyverno makes use of them to identify the namespace and decide to whom the permission is given, which in our case is a specific user:
kubectl apply -f role-policy.yamlkubectl apply -f rolebinding-policy.yaml
If we now create a namespace, with the proper fields, Kyverno will do it's magic:
apiVersion: v1kind: Namespacemetadata:labels:user: janename: test-namespace
➜ kubectl -n test-namespace get roleNAME CREATED ATpod-reader 2020-12-27T20:09:49Z➜ kubectl -n test-namespace get rolebindingNAME ROLE AGEread-pods Role/pod-reader 11s
How did Kyverno do this?
Kubernetes has the concept of Admission Controller which is software that can intercept requests to the API server, prior to its creation, and do something with it. It also possesses the concept of Dynamic Admission Control which are HTTP callbacks that provide similar functionality, allowing us to code the behavior we desired. While we could have developed our own Dynamic Admission Control, Kyverno already has the capabilities we seek and makes leveraging it a no-brainer.