Encrypt Kubernetes secrets, stored in etcd, at the application-layer using a customer-managed key in Cloud KMS. Rationale: By default, GKE encrypts customer content stored at rest, including Secrets. GKE handles and manages this default encryption for you without any additional action on your part. Application-layer Secrets Encryption provides an additional layer of security for sensitive data, such as user defined Secrets and Secrets required for the operation of the cluster, such as service account keys, which are all stored in etcd. Using this functionality, you can use a key, that you manage in Cloud KMS, to encrypt data at the application layer. This protects against attackers in the event that they manage to gain access to etcd.
Solution
To enable Application-layer Secrets Encryption, several configuration items are required. These include: A key ring A key A GKE service account with Cloud KMS CryptoKey Encrypter/Decrypter role Once these are created, Application-layer Secrets Encryption can be enabled on an existing or new cluster. Using Google Cloud Console To create a key: Go to Cloud KMS by visiting https://console.cloud.google.com/security/kms Select CREATE KEY RING Enter a Key ring name and the region where the keys will be stored Click CREATE Enter a Key name and appropriate rotation period within the Create key pane Click CREATE To enable on a new cluster: Go to Kubernetes Engine by visiting https://console.cloud.google.com/kubernetes/list Click CREATE CLUSTER Expand the template by clicking 'Availability, networking, security, and additional features' and check the 'Enable Application-layer Secrets Encryption' checkbox. Select the desired Key as the customer-managed key and if prompted grant permissions to the GKE Service account Click CREATE. To enable on an existing cluster: Go to Kubernetes Engine by visiting https://console.cloud.google.com/kubernetes/list Click to edit cluster you want to modify. Enable Application-layer Secrets Encryption and choose the desired Key Click SAVE. Using Command Line To create a key: Create a key ring: gcloud kms keyrings create [RING_NAME] \ --location [LOCATION] \ --project [KEY_PROJECT_ID] Create a key: gcloud kms keys create [KEY_NAME] \ --location [LOCATION] \ --keyring [RING_NAME] \ --purpose encryption \ --project [KEY_PROJECT_ID] Grant the Kubernetes Engine Service Agent service account the Cloud KMS CryptoKey Encrypter/Decrypter role: gcloud kms keys add-iam-policy-binding [KEY_NAME] \ --location [LOCATION] \ --keyring [RING_NAME] \ --member serviceAccount:[SERVICE_ACCOUNT_NAME] \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter \ --project [KEY_PROJECT_ID] To create a new cluster with Application-layer Secrets Encryption: gcloud container clusters create [CLUSTER_NAME] \ --cluster-version=latest \ --zone [ZONE] \ --database-encryption-key projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME] \ --project [CLUSTER_PROJECT_ID] To enable on an existing cluster: gcloud container clusters update [CLUSTER_NAME] \ --zone [ZONE] \ --database-encryption-key projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME] \ --project [CLUSTER_PROJECT_ID] Impact: To use the Cloud KMS CryptoKey to protect etcd in the cluster, the 'Kubernetes Engine Service Agent' Service account must hold the 'Cloud KMS CryptoKey Encrypter/Decrypter' role. Default Value: By default, Application-layer Secrets Encryption is disabled.