Kubernetes policy enforcement with OPA Gatekeeper and Kyverno

Policy and Governance in Kubernetes:

In brief, policy and governance provide a set of rules which define a guideline that either can be enforced or audited. In the kubernetes ecosystem the governance model plays a crucial role in managing and organising the infrastructure, security and consistency of the resources and deployments. Kubernetes provides Role-based Access Control (RBAC) which is an authorization mechanism for creating and managing permission around Kubernetes resources. But RBAC doesn’t allow admins to control the specification of the kubernetes resources. By controlling these specifications could help the administrators to be able to define the policy boundaries and limits. These limitations could be overcome by using Kubernetes native Policy management and Governance tools. Advantages of using these policy and governance tools:

• Restrict running pods as the root user
• Required labels to group resources
• Provide resource requests and limits
• Whitelisting of trusted container registries and images
• Permitting the publicly exposed Load-Balancer services
• Restrict Updates and Deletes
• Restrict deprecated resources and a lot more..

OPA Gatekeeper:

• OPA Gatekeeper is a customizable kubernetes admission webhook that helps to enforce custom policies on any kubernetes object at its creation time. It is also a CNCF graduated open source project.

Architecture: OPA gatekeeper is undoubtedly the most widely adopted tool for implementing Kubernetes policies. OPA uses a “policy-as-code” approach by decoupling the policy from the code and combining the policy enforcement. OPA uses a Domain Specific Language (DSL) called Rego to describe its policies. Gatekeeper policies run with the help of kubernetes admission controller webhooks which are triggered whenever a resource is created, updated or deleted inside a kubernetes cluster. In general Gatekeeper is a validating (mutating TBA as per official doc) webhook that enforces CRD-based policies executed by Open Policy Agent.

Kyverno:

Kyverno (meaning “govern” in Greek) is a policy engine that is specifically designed only for Kubernetes. Kyverno works by using a dynamic admission controller that verifies each and every request you send via Kubectl to the Kube API server. Then it will match the request with the policy and take appropriate action (mutate, validate, etc..) accordingly. Also, Kyverno is an open-source and a part of CNCF Sandbox Project. Architecture: In Kyverno, policies are managed as Kubernetes resources and no new language (yaml) is required to write policies. This allows using familiar tools such as kubectl, git, and kustomize to manage policies. Kyverno policies can validate, mutate, and generate Kubernetes resources. Also, Kyverno can be used to scan the existing workloads in search of best practices or any regulation/compliance standards. The Kyverno CLI can be used to test policies and validate resources as part of a CI/CD pipeline. Side by Side Analysis:

OPA Gatekeeper	Kyverno
Advantages
Capable of handling very complex policy CNCF graduated and more established in the community Supports multiple replicas for availability and scalability	Easiness in writing policies Excellent mutation abilities Supports multiple replicas for availability and scalability Generation feature and syncing ability is unique and allows new use cases Quick validation and high degree of flexibility
Disadvantages
Requires to learn a new programming language Mutation ability is in nascent stage Limited to validation use cases Policy is complex and often lengthy	Highly complex policy may not be possible since no programming language is integrated Young, still largely unproven due to lower establishments

Analysing Real Time Examples:

Here we use a scenario to compare both OPA Gatekeeper and Kyverno. We are using a policy that will not allow you to create a namespace that doesn’t have a specific label.

OPA Gatekeeper:

OPA Gatekeeper has been already deployed on the AKS Cluster. The OPA gatekeeper will have an audit and controller manager deployed, audit functionality will enable periodic evaluations of replicated resources against the Constraints enforced in the cluster to detect any existing misconfigurations.

• Deploying an OPA gatekeeper policy, the OPA gatekeeper uses OPA Constraint framework to describe and enforce the policy. OPA policies have a constraint and a constraint template, you must define the constraint template first. The constraint template describes both the Rego that enforces the constraint and the schema of the constraint.
• Constraint Template that restricts the creation of namespaces that don’t have a label “gatekeeper‘.

$ kubectl apply -f k8srequiredlabels_template.yaml constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels created k8srequiredlabels_template.yaml apiVersion: templates.gatekeeper.sh/v1 kind: ConstraintTemplate metadata: name: k8srequiredlabels spec: crd: spec: names: kind: K8sRequiredLabels validation: # Schema for the `parameters` field openAPIV3Schema: type: object properties: labels: type: array items: type: string targets: – target: admission.k8s.gatekeeper.sh rego: | package k8srequiredlabels violation[{“msg”: msg, “details”: {“missing_labels”: missing}}] { provided := {label | input.review.object.metadata.labels[label]} required := {label | label := input.parameters.labels[/vc_column_text][/vc_column][/vc_row] } missing := required – provided count(missing) > 0 msg := sprintf(“you must provide labels: %v”, [missing]) } Constraint CRD: $ kubectl apply -f all_ns_must_have_gatekeeper.yaml k8srequiredlabels.constraints.gatekeeper.sh/ns-must-have-gk created apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sUniqueLabel metadata: name: ns-gk-label-unique spec: match: kinds: – apiGroups: [“”] kinds: [“Namespace”] parameters: label: gatekeeper

• Listing the constraints that are currently active.
• Testing the OPA Gatekeeper by creating a namespace without the label.
• Deploying a namespace that have a label “gatekeeper”

  Kyverno:

  Kyverno has already been installed on the cluster and it will run as a Dynamic Admission Controller. Kyverno verifies the api requests and applies matching policies that are intended to enforce admission policies or reject requests with the help of validating and mutating webhook controllers.
  • Listing all the Kubernetes resources that are being deployed with the Kyverno.
  • Getting the validating and mutating webhook configurations.
    • Kyverno policies are a collection of rules which consist of a match and exclude(optional) declaration, and one of a validate, mutate, generate, or verify Imagesdeclaration. Here, applying a Kyverno policy that is intended to reject the namespaces that don’t have a label “Kyverno”.
    $ kubectl apply -f all_ns_must_have_kyverno.yaml clusterpolicy.kyverno.io/require-labels created apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: require-labels annotations: policies.kyverno.io/title: Require Labels policies.kyverno.io/category: Best Practices policies.kyverno.io/severity: medium policies.kyverno.io/subject: Namespace, Label policies.kyverno.io/description: >- Define and use labels that identify semantic attributes of your application or Deployment. A common set of labels allows tools to work collaboratively, describing objects in a common manner that all tools can understand. The recommended labels describe applications in a way that can be queried. This policy validates that the label `kyverno` is specified with some value. spec: validationFailureAction: enforce background: true rules: – name: check-for-labels match: resources: kinds: – Namespace validate:s message: “The label `kyverno` is required. example: `(kyverno: value)`” pattern: metadata: labels: kyverno: “?*”
    • Listing all the resource types of Kyverno
    • Listing the cluster policies that are active, here you can find the already deployed policy.
    • Testing the policy by creating a namespace without any labels.
    • Now checking with a matching scenario, deploying a namespace with the label “kyverno”.

      Conclusion:

      Both OPA Gatekeeper and Kyverno are great policy-as-code tools that are potent in implementing best security and governance at an organisation. Each of these tools have its own advantages, OPA Gatekeeper can handle very complex policies because of the usage of its own DSL (Rego). As it has predefined libraries, it is easy to write the policies from the scratch. Likewise, Kyverno is flexible and there is no need for learning extra DSLs. So the one who knows Kubernetes YAML can write and understand policies easily. Kyverno has good mutation ability and HA support. Looking for the help to implement a good Devops strategy? You can send a message to us and Stackgenie’s team will get back to you.