Yes, you have set up a platform on top of Kubernetes cluster and given appropriate access to your developers to deploy their apps. Cool! but now you face issues from a resources, security and maintenance point of view because your developers deploy them as they wish without following any set of rules, even if you have the rules, you are at their mercy to follow them.

To avoid this issue, Kyverno to the rescue. It is a policy engine that allows you to write policies to validate, generate and mutate Kubernetes workloads. It acts as an intermediary to check if the admitting resource is compliant with the policies or not.

Let us take a simple problem of pod definitions not having resource requests and limits set. This can cause problems in terms of pod eating up all the resources if it has bad code in it. So I will create a Kyverno policy to reject all such pods which does not have resources specified in pod definition file.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-requests-limits
  annotations:
    policies.kyverno.io/title: Require Limits and Requests
    policies.kyverno.io/category: Best Practices, EKS Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/minversion: 1.6.0
    policies.kyverno.io/description: >-
      As application workloads share cluster resources, it is important to limit resources
      requested and consumed by each Pod. It is recommended to require resource requests and
      limits per Pod, especially for memory and CPU. If a Namespace level request or limit is specified,
      defaults will automatically be applied to each Pod based on the LimitRange configuration.
      This policy validates that all containers have something specified for memory and CPU
      requests and memory limits.      
spec:
  validationFailureAction: audit
  background: true
  rules:
  - name: validate-resources
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "CPU and memory resource requests and limits are required."
      pattern:
        spec:
          containers:
          - resources:
              requests:
              limits:
                memory: "?*"

This is how you create a Kyverno policy to check if the pod definition file have resource limits. But there are some scenarios where your policy spans 100's of lines due to complex conditionals.

CEL to the rescue

CEL is a light weight language created by Google for expression evaluation. This is a perfect fit for the Kubernetes validation because of its speed. By using CEL with Kyverno gives you a simpler policy definition file which is readable and universally similar. To rewrite the same policy as above I can write like

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-requests-limits  
spec:
  validationFailureAction: audit
  background: true
  rules:
  - name: validate-resources
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      cel:
        expressions:
          - message: "CPU and memory limits must have a value"
            expression: "object.spec.containers.all(container, has(container.resources.limits.cpu) && has(container.resources.limits.memory))"

Here are some of more policies from Kyverno blog

Conclusion

CEL can be an excellent way to write policies, there maybe an use-case where you can keep seperate source-code for CEL expressions and can create policies dynamically as per cluster-type(like test and prod) using automation.

• CEL Playground

• Kyverno Playground