Skip to main content

​
Overview

The kit uses External Secrets Operator to sync secrets from AWS Secrets Manager into Kubernetes. This keeps sensitive data out of Git while providing a GitOps-friendly secret management workflow.

​
Architecture

AWS Secrets Manager          Kubernetes Cluster
┌───────────────────┐        ┌────────────────────────────┐
│                   │        │                            │
│ ksk-use2-staging- │        │  ClusterSecretStore        │
│   myapp-db        │<───────│  (aws-secrets-manager)     │
│                   │        │            │               │
└───────────────────┘        │            ▼               │
                             │  ExternalSecret            │
                             │  (myapp-secrets)           │
                             │            │               │
                             │            ▼               │
                             │  Secret                    │
                             │  (myapp-secrets)           │
                             │                            │
                             └────────────────────────────┘
  1. ClusterSecretStore - Configures access to AWS Secrets Manager (one per cluster)
  2. ExternalSecret - Defines which secrets to fetch and how to map them
  3. Secret - The Kubernetes Secret created and kept in sync

​
Prerequisites

The External Secrets Operator uses Pod Identity to authenticate to AWS. This is configured automatically by the EKS Terraform module. Verify the ClusterSecretStore is working: 
kubectl get clustersecretstores
kubectl describe clustersecretstore aws-secrets-manager

​
Create a Secret in AWS

1

Create the secret in Secrets Manager

Using AWS CLI:
export REGION="us-east-2"  # Your AWS region

aws secretsmanager create-secret \
  --name "ksk-use2-staging-myapp-db" \
  --secret-string "postgres://user:pass@host:5432/db" \
  --region ${REGION}
Or using the AWS Console:
  1. Navigate to Secrets Manager
  2. Click “Store a new secret”
  3. Choose “Other type of secret”
  4. Enter key/value pairs or plaintext
  5. Name it following the naming convention
2

Create the ExternalSecret manifest

Create ExternalSecret.myapp-secrets.yaml in your service’s templates:
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: myapp-secrets
  namespace: myapp
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: myapp-secrets
    creationPolicy: Owner
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: ksk-use2-staging-myapp-db
3

Deploy and verify

# Apply the ExternalSecret
kubectl apply -f ExternalSecret.myapp-secrets.yaml

# Check the ExternalSecret status
kubectl get externalsecret myapp-secrets -n myapp

# Verify the Secret was created
kubectl get secret myapp-secrets -n myapp
kubectl get secret myapp-secrets -n myapp -o jsonpath='{.data.DATABASE_URL}' | base64 -d

​
Secret Naming Conventions

The kit uses CloudPosse null-label for consistent resource naming. Secrets follow the same pattern: 
{namespace}-{environment}-{stage}-{name}
ComponentDescriptionExamples
namespaceOrganization abbreviationksk (kube-starter-kit), myco, etc.
environmentAWS region abbreviationuse2 (us-east-2), use1 (us-east-1), gbl (global)
stageDeployment stagestaging, prod, shared
nameDescriptive secret namemyapp-db, argocd-github-dex, signoz-ingestion-key
The examples below use ksk as the namespace (the kit’s default). Replace this with the namespace you configured during Bootstrap Accounts.
Examples:
  • ksk-use2-staging-myapp-db - Database credentials for myapp
  • ksk-use2-staging-argocd-github-dex - ArgoCD GitHub OAuth credentials
  • ksk-use2-staging-signoz-ingestion-key - SigNoz observability ingestion key
This convention:
  • Aligns with all other Terraform-managed resources
  • Makes secrets easy to identify by cluster/environment
  • Enables fine-grained IAM policies using prefixes
  • Avoids naming conflicts across environments

​
Fetch Multiple Values from One Secret

AWS Secrets Manager secrets can contain JSON with multiple key/value pairs:
1

Create a JSON secret

export REGION="us-east-2"  # Your AWS region

aws secretsmanager create-secret \
  --name "ksk-use2-staging-myapp-config" \
  --secret-string '{"DB_HOST":"localhost","DB_USER":"admin","DB_PASS":"secret"}' \
  --region ${REGION}
2

Extract specific keys

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: myapp-config
  namespace: myapp
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: myapp-config
  data:
    - secretKey: DB_HOST
      remoteRef:
        key: ksk-use2-staging-myapp-config
        property: DB_HOST
    - secretKey: DB_USER
      remoteRef:
        key: ksk-use2-staging-myapp-config
        property: DB_USER
    - secretKey: DB_PASS
      remoteRef:
        key: ksk-use2-staging-myapp-config
        property: DB_PASS

​
Use Secrets in Pods

Reference the synced Secret in your Deployment: 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
        - name: myapp
          env:
            # Single environment variable
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: myapp-secrets
                  key: DATABASE_URL

            # Or load all keys as env vars
          envFrom:
            - secretRef:
                name: myapp-config

​
Refresh and Sync Behavior

​
Automatic Refresh

ExternalSecrets periodically refresh from AWS based on refreshInterval: 
spec:
  refreshInterval: 1h  # Check for updates every hour

​
Force Refresh

To immediately sync a secret: 
# Annotate to trigger refresh
kubectl annotate externalsecret myapp-secrets -n myapp force-sync=$(date +%s) --overwrite

​
Reloader Integration

The kit includes Reloader, which automatically restarts pods when their Secrets change: 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  annotations:
    reloader.stakater.com/auto: "true"  # Restart on any Secret/ConfigMap change
Or for specific secrets: 
metadata:
  annotations:
    secret.reloader.stakater.com/reload: "myapp-secrets"

​
Environment-Specific Secrets

Use different secret paths for each environment: 
# values.yaml (staging)
secrets:
  databaseUrlKey: ksk-use2-staging-myapp-db

# values.production.yaml
secrets:
  databaseUrlKey: ksk-use2-prod-myapp-db
Then template the ExternalSecret: 
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: myapp-secrets
spec:
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: {{ .Values.secrets.databaseUrlKey }}

​
IAM Permissions

The External Secrets service account needs permission to read secrets. This is configured via Pod Identity in the EKS Terraform module. By default, the External Secrets Pod Identity role has access to all secrets (arn:aws:secretsmanager:*:*:secret:*). To restrict access to specific prefixes, update the policy in terraform/modules/eks/base-infra-resources.tf: 
module "external_secrets_pod_identity" {
  # ...
  # Replace region and namespace prefix with your values
  external_secrets_secrets_manager_arns = [
    "arn:aws:secretsmanager:<REGION>:*:secret:<namespace>-<env>-staging-*",
    "arn:aws:secretsmanager:<REGION>:*:secret:<namespace>-<env>-prod-*"
  ]
}

​
Next Steps