Guide to Using AL2023 AMI with Amazon EKS

AL2023 family of EKS optimized AMIs will become de-facto AMIs starting from EKS version 1.33. This new AMI brings in new changes to the way node is added to the EKS cluster and default usage of IDMSv2 also brings in additional security features with it.

Node Initialization

AL2023 no longer uses bootstrap.sh script to join the cluster but rather it uses nodeadm. This nodeadm relies on the YAML file that you provide it during start-up. This YAML file of type NodeConfig must have fields like cluster name, endpoint and CA. This information can be obtained from DescribeCluster API call. Here is an example NodeConfig file

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster:
    name: my-cluster
    apiServerEndpoint: https://example.com
    certificateAuthority: Y2VydGlmaWNhdGVBdXRob3JpdHk=
    cidr: 10.100.0.0/16
  kubelet:
    config:
      shutdownGracePeriod: 30s
      featureGates:
        DisableKubeletCloudCredentialProviders: true

Once this file is populated, node can join the cluster by running nodeadm init nodeconfig.yaml command. Using NodeConfig file, you are also modifying the Kubelet config. This enables to keep all your node specifc changes in one file rather than editing multiple files. Now all you have to worry about is the ways to automate and generate this file reliably whenever new node joins your cluster. Here is a golang based application that I developed to automatically generate NodeConfig file reliabliy during startup

IDMSv2

Now as IDMSv2 is set as default, pods can no longer use node credentials by-default. This provides additional security. In order to allow pods to use node role, you need to set HttpPutResponseHopLimit to 2. The better way to provide access to AWS resources for your applications is to use EKS Pod Identity or IRSA.

Conclusion

There are many other changes that comes with AL2023 which can affect your Kubernetes workloads like default versions of VPC CNI and its support to DNF packages etc. Overall the AL2023 marks a significant step towards Kubernetes optimized linux OS like Talos.