# Using K8s Self Hosted Runners for GitHub Actions

As the popularity for GitHub Actions for CI increasing these days, so is the the bill for GitHub hosted Runners. As this [article](https://vishnudeva.medium.com/cost-effective-github-actions-9409fa7b2147) explains, the typical costs for GHR(GitHub hosted Runners) is 23x of AWS spot instances for the same compute. As it is comparatively easy😜 to setup a managed (or self-managed) K8s cluster with Karpenter and spot-instances nodepool setup for GHA than to use GHR. Moreover if you have to have complete control over the CI process for compliance purposes. It is a no brainer to use SHR.

In order to setup K8s cluster as SH environment, you need to have Action Runner Controller installed. It is a K8s operator to manage SHR lifecycle. It comes with its own CRDs.

## ARC Installation

In order to install ARC you need to figure out authentication method first. I have used GitHubApp approach. So you will create an app and give required permissions and add the App to required repositories.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1721001467774/a0ec76ca-0ee3-4669-b1c1-76410c91d9f9.png align="center")

ARC needs to be authenticated with GitHub. In order to do that you need to create secret with App details so ARC can pick these up during installation.

```json
➜  k8s-gha k get secrets -n actions controller-manager -o json |jq '.data | keys'
[
  "github_app_id",
  "github_app_installation_id",
  "github_app_private_key"
]
```

Now you can install ARC in the cluster using Helm

```plaintext
➜  ~ helm install actions-runner-controller actions-runner-controller/actions-runner-controller --namespace=actions --version=0.22.0 -f runner-controller.yaml
NAME: actions-runner-controller
LAST DEPLOYED: Sun Jul 14 16:36:39 2024
NAMESPACE: actions
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace actions -l "app.kubernetes.io/name=actions-runner-controller,app.kubernetes.io/instance=actions-runner-controller" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace actions $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace actions port-forward $POD_NAME 8080:$CONTAINER_PORT
```

Here is how the runner-controller file looks like

```yaml
replicaCount: 1
webhookPort: 9443
syncPeriod: 1m
defaultScaleDownDelay: 5m
enableLeaderElection: true

authSecret:
  enabled: true
  create: false
  name: "controller-manager"

image:
  repository: "summerwind/actions-runner-controller"
  actionsRunnerRepositoryAndTag: "summerwind/actions-runner:ubuntu-20.04"
  dindSidecarRepositoryAndTag: "docker:dind"
  pullPolicy: IfNotPresent

serviceAccount:
  create: true

service:
  type: ClusterIP
  port: 443

certManagerEnabled: true

logFormat: text

githubWebhookServer:
  enabled: false
```

Now you are all set-up 🎉.

Below are the CRDs that comes with ARC

```plaintext
➜  k8s-gha k get crds |grep summer
horizontalrunnerautoscalers.actions.summerwind.dev    2024-07-14T21:36:26Z
runnerdeployments.actions.summerwind.dev              2024-07-14T21:36:27Z
runnerreplicasets.actions.summerwind.dev              2024-07-14T21:36:28Z
runners.actions.summerwind.dev                        2024-07-14T21:36:29Z
runnersets.actions.summerwind.dev                     2024-07-14T21:36:32Z
```

This follows the same pattern of K8s workloads where runners are equivalent to Pods and runnerdeployments are Deployments. You need to create a runnerdeployment with the image name, labels so GHA can pick up runners from this deployment based on the label

```yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  annotations:
    karpenter.sh/do-not-evict: "true"
  name: self-hosted-runner-deployment
  namespace: actions
spec:
  template:
    spec:
      repository: HighonAces/actions-1
      image: summerwind/actions-runner:ubuntu-20.04
      resources:
        requests:
          cpu: 1500m
          memory: 2000Mi
```

You also need to create HorizontalRunnerAutoscaler to scale this deployment.

```yaml
apiVersion: v1
items:
- apiVersion: actions.summerwind.dev/v1alpha1
  kind: HorizontalRunnerAutoscaler
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"actions.summerwind.dev/v1alpha1","kind":"HorizontalRunnerAutoscaler","metadata":{"annotations":{},"name":"self-hosted-runner-deployment-autoscaler","namespace":"actions"},"spec":{"maxReplicas":30,"minReplicas":0,"scaleTargetRef":{"kind":"RunnerDeployment","name":"self-hosted-runner-deployment"},"scaleUpTriggers":[{"duration":"30m","githubEvent":{"workflowJob":{}}}]}}
    creationTimestamp: "2024-07-14T21:52:23Z"
    generation: 2
    name: self-hosted-runner-deployment-autoscaler
    namespace: actions
    resourceVersion: "9689"
    uid: 41291b4a-cbd8-4af2-bfe3-02c2b29d8262
  spec:
    maxReplicas: 30
    minReplicas: 2
    scaleTargetRef:
      kind: RunnerDeployment
      name: self-hosted-runner-deployment
    scaleUpTriggers:
    - duration: 30m
      githubEvent:
        workflowJob: {}
  status:
    desiredReplicas: 2
    lastSuccessfulScaleOutTime: "2024-07-14T22:28:46Z"
kind: List
metadata:
  resourceVersion: ""
```

This continuously keeps 2 replicas running as I have not given any metrics definition. By making use of PercentageRunnersBusy, you can scaleup or scaledown. Here is the metrics definition from ARC documentation

```yaml
  - type: PercentageRunnersBusy
    scaleUpThreshold: '0.75'
    scaleDownThreshold: '0.25'
    scaleUpFactor: '2'
    scaleDownFactor: '0.5'
```

Once you have min number of runners running you can see the corresponding pods in cluster and same in GH runners page.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1721002803495/aaa5b594-42c9-4a85-b193-996aefa75653.png align="center")

These runners will pickup the jobs whenever you have actions configured with this parameter `runs-on: self-hosted-linux`.

## Conclusion

This setup is rather easy and customizable. It is a very good alternative for the GHR and I cannot wait to see what future holds for K8s based Self hosted runners.  
  
Source: [https://medium.com/simform-engineering/how-to-setup-self-hosted-github-action-runner-on-kubernetes-c8825ccbb63c](https://medium.com/simform-engineering/how-to-setup-self-hosted-github-action-runner-on-kubernetes-c8825ccbb63c)

[https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller)

[https://github.com/actions/actions-runner-controller](https://github.com/actions/actions-runner-controller)