Terraform EBS + KMS on EKS

July 17, 2024

Ideally you want every volume on your AWS account to be encrypted with a Customer managed KMS key (CMKs) by default.

You can set this up with the following Terraform code after you setup a proper IAM policy for the key with your specific needs.

data "aws_iam_policy_document" "this" {}

resource "aws_kms_key" "ebs" {
  description             = "EBS CMK"
  deletion_window_in_days = 7
  enable_key_rotation     = true
  policy                  = data.aws_iam_policy_document.this.json
}

resource "aws_ebs_default_kms_key" "this" {
  key_arn = aws_kms_key.ebs.arn
}

resource "aws_ebs_encryption_by_default" "this" {
  enabled = true
}

In EKS you will probably want to use the AWS EBS CSI Driver to automatically create EBS volumes to your Kubernetes volumes to allow making snapshots and resizing when needed.

To enable encryption on the cluster and configure it do use the gp3 volumes you should create a new Storage Class and annotate as default.

resource "kubernetes_storage_class" "gp3_encrypted" {
  metadata {
    name = "gp3-kms"
    annotations = {
      "storageclass.kubernetes.io/is-default-class" = "true"
    }
  }

  storage_provisioner = "ebs.csi.aws.com"
  reclaim_policy      = "Delete"

  parameters = {
    type      = "gp3",
    encrypted = true
    kmsKeyId  = aws_kms_key.ebs.arn
  }

  volume_binding_mode = "WaitForFirstConsumer"
}

We will be using the IAM Role for Service Accounts in EKS project that provides policies for various controllers and deploy the Helm chart.

We will assuming you are using a eks module that exports the cluster_name, cluster_endpoint and oidc_provider_arn.

data "aws_region" "current" {}

module "aws_ebs_csi_irsa_role" {
  source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"

  role_name = "${module.eks.cluster_name}-ebs-csi-controller"

  attach_ebs_csi_policy = true

  ebs_csi_kms_cmk_ids = [aws_kms_key.ebs.arn]

  oidc_providers = {
    main = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:ebs-csi-controller-sa"]
    }
  }
}

resource "helm_release" "aws_ebs_csi_driver" {
  chart      = "aws-ebs-csi-driver"
  namespace  = "kube-system"
  name       = "aws-ebs-csi-driver"
  repository = "https://kubernetes-sigs.github.io/aws-ebs-csi-driver"

  set {
    name  = "controller.serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"
    value = module.aws_ebs_csi_irsa_role.iam_role_arn
  }

  set {
    name  = "node.serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"
    value = module.aws_ebs_csi_irsa_role.iam_role_arn
  }

  set {
    name  = "controller.region"
    value = data.aws_region.current.name
  }
}

To authenticate Helm you can use:

data "aws_eks_cluster_auth" "cluster" {
  name = module.eks.cluster_name
}

provider "helm" {
  kubernetes {
    host                   = module.eks.cluster_endpoint
    token                  = data.aws_eks_cluster_auth.cluster.token
    cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
  }
}