How I Set Up Vault with GCP KMS Auto-Unseal in Kubernetes

Jan 20, 2024

I’ll never forget the panic of a 3 AM page for a Vault outage, fumbling for my unseal keys. Traditional Vault setups require a quorum of operators to manually unseal Vault after every restart. It’s a tedious and insecure process. That’s why I moved to an auto-unseal workflow using Google Cloud KMS. It completely automates the unsealing process, making my Vault deployment much more resilient. Here’s my guide to setting it up in Kubernetes.

My Architecture: Raft + KMS

My architecture for Vault on Kubernetes is designed for simplicity and high availability. I use Vault’s integrated Raft storage backend, which means I don’t need to manage a separate Consul cluster. For high availability, I run multiple Vault replicas. The magic is the GCP KMS Auto-Unseal: instead of splitting the master key among operators, Vault encrypts it with a key from GCP KMS and stores it in its own storage. On startup, Vault authenticates to GCP, decrypts its master key, and unseals itself automatically.

graph TB
    subgraph GCP
        KMS[GCP KMS<br/>Encryption Key]
    end

    subgraph Kubernetes Cluster
        Vault0[Vault-0<br/>Leader]
        Vault1[Vault-1<br/>Follower]
        Vault0 -.->|Raft Replication| Vault1
    end

    Vault0 -->|Encrypt/Decrypt<br/>Master Key| KMS
    Vault1 -->|Encrypt/Decrypt<br/>Master Key| KMS

    Users[Users] --> LB[LoadBalancer]
    LB --> Vault0
    LB --> Vault1

    style KMS fill:#4285f4,color:#fff
    style Vault0 fill:#7c4dff,color:#fff

My Setup Process

Step 1: Create the GCP KMS Key and Service Account

First, I set up the GCP side. I create a KMS key ring and a crypto key that Vault will use to encrypt its master key.

gcloud kms keyrings create vault-unseal-kr --location=global
gcloud kms keys create vault-unseal-key --location=global --keyring=vault-unseal-kr --purpose=encryption

Next, I create a dedicated GCP Service Account. This is crucial. I grant this service account the roles/cloudkms.cryptoKeyEncrypterDecrypter role on the key I just created. This is the only permission it needs.

gcloud iam service-accounts create vault-kms-sa
gcloud kms keys add-iam-policy-binding vault-unseal-key ... --member="serviceAccount:vault-kms-sa@..."

Step 2: Create the Kubernetes Secret

I then download the JSON key for this service account and create a Kubernetes secret from it in my vault namespace. This is how the Vault pods will authenticate to GCP.

gcloud iam service-accounts keys create vault-kms-key.json --iam-account=vault-kms-sa@...
kubectl create namespace vault
kubectl create secret generic kms-vault-creds -n vault --from-file=kms-creds.json=./vault-kms-key.json

Step 3: Install Vault with Helm

Now for the main event: installing Vault with Helm. I use the official HashiCorp chart and a custom values-vault.yaml file. In this file, I configure the ha.raft backend, mount the GCP credentials secret as a volume, and most importantly, I add the seal "gcpckms" block, pointing it to my GCP project and KMS key.

# values-vault.yaml
server:
  ha:
    enabled: true
    replicas: 2
    raft:
      enabled: true
      config: |
        ui = true
        listener "tcp" {
          tls_disable = 1
          address = "[::]:8200"
          cluster_address = "[::]:8201"
        }
        storage "raft" {
          path = "/vault/data"
        }
        seal "gcpckms" {
          project     = "your-gcp-project-id"
          region      = "global"
          key_ring    = "vault-unseal-kr-production"
          crypto_key  = "vault-unseal-key-production"
        }

  # Mount the GCP credentials secret
  extraEnvironmentVars:
    GOOGLE_APPLICATION_CREDENTIALS: "/vault/userconfig/kms-vault-creds/kms-creds.json"
  extraVolumes:
    - type: secret
      name: kms-vault-creds

I install the chart with helm upgrade -i vault hashicorp/vault -n vault -f values-vault.yaml.

Step 4: Initialize and Join the Cluster

After installing, I exec into the first pod (vault-0) and run vault operator init. This is the only time I have to deal with keys. I immediately save the initial root token and the recovery keys (which are different from unseal keys) in a secure location.

Then, I join the other Vault pods to the Raft cluster. I exec into vault-1 and run vault operator raft join http://vault-0.vault-internal:8200.

Step 5: Verify Auto-Unseal

The moment of truth is testing the auto-unseal. I just delete one of the Vault pods. As it comes back up, I can check its status and see Sealed: false without me having to do anything. It’s a beautiful thing. I also test HA by deleting the leader pod and watching the other pod take over leadership within seconds.

My Final Thoughts

My key takeaway is that combining Raft storage with GCP KMS auto-unseal is the simplest, most robust way to run HA Vault on Kubernetes. It eliminates the operational burden of both managing a separate storage backend and manually unsealing Vault. My production setup always includes at least 3 replicas, resource limits, audit logging, and a CronJob that regularly takes Raft snapshots and uploads them to GCS for disaster recovery. This setup has made our secrets management infrastructure both highly available and much easier to manage.

✦ El Muhammad's Portfolio

© 2025 Aria

Instagram YouTube TikTok 𝕏 GitHub