Advanced Patterns and Techniques

Environment-Specific Configurations

Use Kustomize overlays to manage environment-specific configurations:

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: frontend
        image: frontend:v1
        resources:
          requests:
            memory: 256Mi
            cpu: 100m
          limits:
            memory: 512Mi
            cpu: 200m

# overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patchesStrategicMerge:
- replicas-patch.yaml
- resources-patch.yaml

# overlays/production/replicas-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 5

# overlays/production/resources-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  template:
    spec:
      containers:
      - name: frontend
        resources:
          requests:
            memory: 512Mi
            cpu: 200m
          limits:
            memory: 1Gi
            cpu: 500m

Security Considerations for GitOps

Security is a critical aspect of any GitOps implementation:

Securing Git Repositories

Branch Protection Rules
- Require pull request reviews
- Enforce status checks
- Restrict who can push to specific branches

Signed Commits

Require commit signing
Verify commit signatures

# Configure Git for commit signing
git config --global user.signingkey <YOUR-GPG-KEY-ID>
git config --global commit.gpgsign true

Access Control
- Use fine-grained permissions
- Implement the principle of least privilege
- Regularly audit access

Securing Secrets in GitOps

Never store plain-text secrets in Git. Instead, use these approaches:

Sealed Secrets

# Install kubeseal CLI
brew install kubeseal

# Create a sealed secret
kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=t0p-s3cr3t \
  --dry-run=client -o yaml | \
  kubeseal --format yaml > sealed-db-credentials.yaml

# sealed-db-credentials.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: default
spec:
  encryptedData:
    password: AgBy8hCL8...
    username: AgCtrOD9R...

External Secret Operators

# AWS Secrets Manager example
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secretsmanager
    kind: ClusterSecretStore
  target:
    name: db-credentials
  data:
  - secretKey: username
    remoteRef:
      key: db-credentials
      property: username
  - secretKey: password
    remoteRef:
      key: db-credentials
      property: password

HashiCorp Vault Integration

# Vault secret with Vault Operator
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: db-credentials
spec:
  vaultAuthRef: vault-auth
  mount: kv
  path: db-credentials
  destination:
    name: db-credentials
    create: true
  refreshAfter: 30s
  type: kv-v2

RBAC and Least Privilege

Implement proper RBAC for your GitOps controllers:

# Restricted service account for Flux
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flux
  namespace: flux-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: flux-restricted
rules:
- apiGroups: [""]
  resources: ["namespaces", "services"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "patch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: flux-restricted
subjects:
- kind: ServiceAccount
  name: flux
  namespace: flux-system
roleRef:
  kind: ClusterRole
  name: flux-restricted
  apiGroup: rbac.authorization.k8s.io

Monitoring and Observability for GitOps

Effective monitoring is essential for GitOps success:

Monitoring GitOps Controllers

Monitor the health and performance of your GitOps controllers:

# Prometheus ServiceMonitor for Flux
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: flux-system
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: flux
  namespaceSelector:
    matchNames:
      - flux-system
  endpoints:
  - port: http
    interval: 30s

Key Metrics to Monitor:

Reconciliation success/failure rates
Reconciliation duration
Git operations success/failure
Resource drift detection

Drift Detection and Alerting

Set up alerts for configuration drift:

# Prometheus alert rule for Flux
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: flux-alerts
  namespace: monitoring
spec:
  groups:
  - name: flux.rules
    rules:
    - alert: FluxReconciliationFailure
      expr: sum(increase(gotk_reconcile_condition{status="False",type="Ready"}[5m])) by (namespace, name) > 0
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: "Flux reconciliation failing for {{ $labels.name }} in {{ $labels.namespace }}"
        description: "Flux has been unable to reconcile {{ $labels.name }} in {{ $labels.namespace }} for more than 10 minutes."
    - alert: GitOpsConfigurationDrift
      expr: gotk_reconcile_condition{status="False",type="Ready"} == 1
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: "Configuration drift detected for {{ $labels.name }}"
        description: "GitOps has detected configuration drift for {{ $labels.name }} in {{ $labels.namespace }}."