chore: init gsd

2026-10-31 01:49:25 +03:00
parent 85376e1f1a
commit 83f5f59015
363 changed files with 100924 additions and 0 deletions
@@ -0,0 +1,106 @@
+---
+name: deployment-pipeline-design
+description: Design multi-stage CI/CD pipelines with approval gates, security checks, and deployment orchestration. Use this skill when designing zero-downtime deployment pipelines, implementing canary rollout strategies, setting up multi-environment promotion workflows, or debugging failed deployment gates in CI/CD.
+---
+
+# Deployment Pipeline Design
+
+Architecture patterns for multi-stage CI/CD pipelines with approval gates, deployment strategies, and environment promotion workflows.
+
+## Purpose
+
+Design robust, secure deployment pipelines that balance speed with safety through proper stage organization, automated quality gates, and progressive delivery strategies. This skill covers both the structural design of pipeline architecture and the operational patterns for reliable production deployments.
+
+## Input / Output
+
+### What You Provide
+
+- **Application type**: Language/runtime, containerized or bare-metal, monolith or microservices
+- **Deployment target**: Kubernetes, ECS, VMs, serverless, or platform-as-a-service
+- **Environment topology**: Number of environments (dev/staging/prod), region layout, air-gap requirements
+- **Rollout requirements**: Acceptable downtime, rollback SLA, traffic splitting needs, canary vs blue-green preference
+- **Gate constraints**: Approval teams, required test coverage thresholds, compliance scans (SAST, DAST, SCA)
+- **Monitoring stack**: Prometheus, Datadog, CloudWatch, or other metrics sources used for automated promotion decisions
+
+### What This Skill Produces
+
+- **Pipeline configuration**: Stage definitions, job dependencies, parallelism, and caching strategy
+- **Deployment strategy**: Chosen rollout pattern with annotated configuration (canary weights, blue-green switchover, rolling parameters)
+- **Health check setup**: Shallow vs deep readiness probes, post-deployment smoke test scripts
+- **Gate definitions**: Automated metric thresholds and manual approval workflows
+- **Rollback plan**: Automated rollback triggers and manual runbook steps
+
+## When to Use
+
+- Design CI/CD architecture for a new service or platform migration
+- Implement deployment gates between environments
+- Configure multi-environment pipelines with mandatory security scanning
+- Establish progressive delivery with canary or blue-green strategies
+- Debug pipelines where stages succeed but production behavior is wrong
+- Reduce mean time to recovery by automating rollback on metric degradation
+
+## Detailed patterns and worked examples
+
+Detailed pattern documentation lives in `references/details.md`. Read that file when the navigation tier above is insufficient.
+
+## Troubleshooting
+
+### Health check passes in pipeline but service is unhealthy in production
+
+The pipeline health check is hitting a shallow `/ping` endpoint that returns 200 even when the database is unreachable. Use a deep readiness check that verifies actual dependencies (see Health Checks section above).
+
+### Canary deployment never promotes to 100%
+
+Argo Rollouts requires a valid `AnalysisTemplate` to auto-promote. If the Prometheus query returns no data (e.g., metric name changed), the analysis stays inconclusive and promotion stalls. Add `inconclusiveLimit` so the rollout fails fast rather than hanging:
+
+```yaml
+spec:
+  metrics:
+  - name: error-rate
+    failureCondition: "result[0] > 0.05"
+    inconclusiveLimit: 2   # fail after 2 inconclusive results, not hang indefinitely
+    provider:
+      prometheus:
+        query: |
+          sum(rate(http_requests_total{status=~"5.."}[2m]))
+          / sum(rate(http_requests_total[2m]))
+```
+
+### Staging deploy succeeds but production job never starts
+
+Check that production environment protection rules are configured — a missing reviewer assignment means the approval gate waits indefinitely with no notification. In GitHub Actions, ensure `Required reviewers` is set to an existing user or team in **Settings → Environments → production**.
+
+### Docker layer cache busted on every run causing slow builds
+
+If `COPY . .` appears before dependency installation, any source file change invalidates the dependency layer. Reorder to copy dependency manifests first:
+
+```dockerfile
+# Good: dependencies cached separately from source code
+COPY package*.json ./
+RUN npm ci
+COPY . .
+RUN npm run build
+```
+
+### Rollback leaves database migrations applied to old code
+
+A service rollback without a migration rollback causes schema/code mismatch errors. Always make migrations backward-compatible (additive only) for at least one release cycle, and keep undo scripts versioned alongside the migration:
+
+```bash
+# migrations/V20240315__add_nullable_column.sql       (forward)
+# migrations/V20240315__add_nullable_column.undo.sql  (backward)
+```
+
+Never run destructive migrations (DROP COLUMN, ALTER NOT NULL) until the old code version is fully retired from all environments.
+
+## Advanced Topics
+
+For platform-specific pipeline configurations, multi-region promotion workflows, and advanced Argo Rollouts patterns, see:
+
+- [`references/advanced-strategies.md`](references/advanced-strategies.md) — Extended YAML examples, platform-specific configs (GitHub Actions, GitLab CI, Azure Pipelines), multi-region canary patterns, and database migration rollback strategies
+
+## Related Skills
+
+- `github-actions-templates` - For GitHub Actions implementation patterns and reusable workflows
+- `gitlab-ci-patterns` - For GitLab CI/CD pipeline implementation
+- `secrets-management` - For secrets handling in CI/CD pipelines
@@ -0,0 +1,629 @@
+# Advanced Deployment Strategies Reference
+
+Extended configurations, platform-specific patterns, and advanced rollback strategies.
+Core patterns and decision tables live in [`../SKILL.md`](../SKILL.md).
+
+---
+
+## Platform-Specific Pipeline Configurations
+
+### GitHub Actions — Full Production Pipeline with Reusable Workflows
+
+```yaml
+# .github/workflows/production.yml
+name: Production Pipeline
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+    inputs:
+      skip_tests:
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+  id-token: write   # for OIDC auth to cloud providers
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    outputs:
+      image_tag: ${{ steps.meta.outputs.version }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Docker metadata
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=sha,prefix=,format=short
+
+      - name: Build and push (with layer cache)
+        uses: docker/build-push-action@v5
+        with:
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+
+  security-scan:
+    needs: build
+    runs-on: ubuntu-latest
+    steps:
+      - name: Run Trivy vulnerability scan
+        uses: aquasecurity/trivy-action@0.28.0
+        with:
+          image-ref: ghcr.io/${{ github.repository }}:${{ needs.build.outputs.image_tag }}
+          exit-code: 1
+          severity: CRITICAL,HIGH
+
+      - name: SAST with Semgrep
+        uses: semgrep/semgrep-action@v1
+        with:
+          config: auto
+
+  test:
+    needs: build
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:16
+        env:
+          POSTGRES_PASSWORD: test
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+    steps:
+      - uses: actions/checkout@v4
+      - name: Run test suite
+        run: make test-ci
+        env:
+          DATABASE_URL: postgres://postgres:test@localhost/test
+
+  deploy-staging:
+    needs: [test, security-scan]
+    environment:
+      name: staging
+      url: https://staging.example.com
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Configure AWS credentials (OIDC)
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: arn:aws:iam::123456789:role/deploy-staging
+          aws-region: us-east-1
+
+      - name: Deploy to EKS staging
+        run: |
+          aws eks update-kubeconfig --name my-cluster-staging
+          kubectl set image deployment/my-app \
+            app=ghcr.io/${{ github.repository }}:${{ needs.build.outputs.image_tag }}
+          kubectl rollout status deployment/my-app --timeout=5m
+
+  e2e-tests:
+    needs: deploy-staging
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Run Playwright E2E suite
+        run: npx playwright test --reporter=github
+        env:
+          BASE_URL: https://staging.example.com
+
+  deploy-production:
+    needs: e2e-tests
+    environment:
+      name: production
+      url: https://app.example.com
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Configure AWS credentials (OIDC)
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: arn:aws:iam::123456789:role/deploy-production
+          aws-region: us-east-1
+
+      - name: Deploy canary to production
+        run: |
+          aws eks update-kubeconfig --name my-cluster-prod
+          kubectl argo rollouts set image my-app \
+            app=ghcr.io/${{ github.repository }}:${{ needs.build.outputs.image_tag }}
+
+      - name: Monitor canary promotion
+        run: |
+          kubectl argo rollouts status my-app --timeout=30m
+
+      - name: Rollback on failure
+        if: failure()
+        run: kubectl argo rollouts abort my-app
+```
+
+---
+
+### GitLab CI — Multi-Environment Pipeline with Dynamic Environments
+
+```yaml
+# .gitlab-ci.yml
+stages:
+  - build
+  - test
+  - staging
+  - production
+
+variables:
+  DOCKER_DRIVER: overlay2
+  IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
+
+build:
+  stage: build
+  image: docker:24
+  services:
+    - docker:24-dind
+  script:
+    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
+    - docker build --cache-from $CI_REGISTRY_IMAGE:latest -t $IMAGE .
+    - docker push $IMAGE
+    - docker tag $IMAGE $CI_REGISTRY_IMAGE:latest
+    - docker push $CI_REGISTRY_IMAGE:latest
+
+test:unit:
+  stage: test
+  image: $IMAGE
+  script:
+    - make test
+  coverage: '/coverage: \d+\.\d+%/'
+  artifacts:
+    reports:
+      coverage_report:
+        coverage_format: cobertura
+        path: coverage.xml
+
+test:security:
+  stage: test
+  image:
+    name: aquasec/trivy:latest
+    entrypoint: [""]
+  script:
+    - trivy image --exit-code 1 --severity CRITICAL,HIGH $IMAGE
+
+deploy:staging:
+  stage: staging
+  environment:
+    name: staging
+    url: https://staging.example.com
+    on_stop: stop:staging
+  script:
+    - kubectl apply -f k8s/staging/
+    - kubectl set image deployment/my-app app=$IMAGE -n staging
+    - kubectl rollout status deployment/my-app -n staging
+  only:
+    - main
+
+stop:staging:
+  stage: staging
+  environment:
+    name: staging
+    action: stop
+  script:
+    - kubectl delete namespace staging --ignore-not-found
+  when: manual
+  only:
+    - main
+
+deploy:production:
+  stage: production
+  environment:
+    name: production
+    url: https://app.example.com
+  script:
+    - kubectl set image deployment/my-app app=$IMAGE -n production
+    - kubectl rollout status deployment/my-app -n production --timeout=10m
+  when: manual
+  only:
+    - main
+  rules:
+    - if: '$CI_COMMIT_BRANCH == "main"'
+      when: manual
+      allow_failure: false
+```
+
+---
+
+### Azure Pipelines — Multi-Stage with Approvals and Environments
+
+```yaml
+# azure-pipelines.yml
+trigger:
+  branches:
+    include:
+      - main
+
+variables:
+  imageRepository: 'myapp'
+  containerRegistry: 'myregistry.azurecr.io'
+  tag: '$(Build.BuildId)'
+
+stages:
+  - stage: Build
+    displayName: 'Build & Test'
+    jobs:
+      - job: BuildAndTest
+        pool:
+          vmImage: ubuntu-latest
+        steps:
+          - task: Docker@2
+            displayName: Build image
+            inputs:
+              command: build
+              repository: $(imageRepository)
+              dockerfile: Dockerfile
+              tags: $(tag)
+
+          - task: Docker@2
+            displayName: Push image
+            inputs:
+              command: push
+              repository: $(imageRepository)
+              tags: $(tag)
+
+          - script: make test
+            displayName: Run tests
+
+  - stage: Staging
+    displayName: 'Deploy to Staging'
+    dependsOn: Build
+    jobs:
+      - deployment: DeployStaging
+        environment: staging
+        pool:
+          vmImage: ubuntu-latest
+        strategy:
+          runOnce:
+            deploy:
+              steps:
+                - task: KubernetesManifest@0
+                  inputs:
+                    action: deploy
+                    manifests: k8s/staging/*.yaml
+                    containers: $(containerRegistry)/$(imageRepository):$(tag)
+
+  - stage: Production
+    displayName: 'Deploy to Production'
+    dependsOn: Staging
+    jobs:
+      - deployment: DeployProduction
+        environment:
+          name: production
+          resourceType: Kubernetes
+        pool:
+          vmImage: ubuntu-latest
+        strategy:
+          canary:
+            increments: [10, 25, 50]
+            preDeploy:
+              steps:
+                - task: ManualValidation@0
+                  inputs:
+                    notifyUsers: 'release-managers@example.com'
+                    instructions: 'Verify staging metrics. Approve to start canary.'
+                    onTimeout: reject
+            deploy:
+              steps:
+                - task: KubernetesManifest@0
+                  inputs:
+                    action: deploy
+                    manifests: k8s/production/*.yaml
+                    containers: $(containerRegistry)/$(imageRepository):$(tag)
+            postRouteTraffic:
+              steps:
+                - script: ./scripts/verify-deployment.sh https://app.example.com
+            on:
+              failure:
+                steps:
+                  - task: KubernetesManifest@0
+                    inputs:
+                      action: reject
+```
+
+---
+
+## Multi-Region Canary Promotion
+
+Deploy to a pilot region first, validate, then promote to remaining regions in parallel:
+
+```yaml
+# deploy-multiregion.yml (GitHub Actions)
+jobs:
+  deploy-pilot:
+    environment: production-us-east-1
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy canary to pilot region
+        run: |
+          aws eks update-kubeconfig --name cluster-us-east-1 --region us-east-1
+          kubectl argo rollouts set image my-app app=$IMAGE
+          kubectl argo rollouts status my-app --timeout=20m
+
+  deploy-secondary:
+    needs: deploy-pilot
+    strategy:
+      matrix:
+        region: [us-west-2, eu-west-1, ap-southeast-1]
+    environment: production-${{ matrix.region }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy to ${{ matrix.region }}
+        run: |
+          aws eks update-kubeconfig --name cluster-${{ matrix.region }} \
+            --region ${{ matrix.region }}
+          kubectl argo rollouts set image my-app app=$IMAGE
+          kubectl argo rollouts status my-app --timeout=20m
+```
+
+---
+
+## Advanced Argo Rollouts Patterns
+
+### Experiment-Based Canary (A/B Analysis)
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Rollout
+metadata:
+  name: my-app
+spec:
+  replicas: 20
+  strategy:
+    canary:
+      canaryService: my-app-canary
+      stableService: my-app-stable
+      trafficRouting:
+        istio:
+          virtualService:
+            name: my-app-vsvc
+      analysis:
+        templates:
+          - templateName: success-rate
+          - templateName: latency-p99
+        startingStep: 1
+        args:
+          - name: service-name
+            value: my-app-canary
+      steps:
+        - setWeight: 5
+        - experiment:
+            templates:
+              - name: baseline
+                specRef: stable
+              - name: canary
+                specRef: canary
+            analyses:
+              - templateName: ab-test
+                requiredForCompletion: true
+        - setWeight: 20
+        - pause: { duration: 10m }
+        - setWeight: 50
+        - pause: { duration: 15m }
+        - setWeight: 100
+```
+
+### Header-Based Canary (Internal Testing Before Traffic Split)
+
+```yaml
+# Route users with X-Canary: true header to canary pods
+apiVersion: networking.istio.io/v1alpha3
+kind: VirtualService
+metadata:
+  name: my-app-vsvc
+spec:
+  http:
+    - match:
+        - headers:
+            x-canary:
+              exact: "true"
+      route:
+        - destination:
+            host: my-app-canary
+    - route:
+        - destination:
+            host: my-app-stable
+          weight: 90
+        - destination:
+            host: my-app-canary
+          weight: 10
+```
+
+---
+
+## Database Migration Rollback Strategies
+
+### Expand/Contract Pattern (Safe Schema Changes)
+
+Never break backward compatibility within a single deployment cycle. Use the expand/contract pattern across three releases:
+
+```
+Release N:   Add nullable column (expand)
+Release N+1: Backfill data, deploy new code that reads new column
+Release N+2: Drop old column (contract) — safe because no code references it
+```
+
+```bash
+# Release N migration — backward compatible
+cat > migrations/V20240315__expand_add_email_v2.sql <<'EOF'
+ALTER TABLE users ADD COLUMN email_v2 VARCHAR(255);
+CREATE INDEX CONCURRENTLY idx_users_email_v2 ON users(email_v2);
+EOF
+
+# Release N+2 migration — contract (only after old code retired)
+cat > migrations/V20240415__contract_drop_email.sql <<'EOF'
+ALTER TABLE users DROP COLUMN email;
+ALTER TABLE users RENAME COLUMN email_v2 TO email;
+EOF
+```
+
+### Flyway with Undo Scripts
+
+```bash
+# Forward migration
+flyway migrate
+
+# Undo the most recent migration (requires Flyway Teams)
+flyway undo
+
+# Undo a specific version
+flyway undo -target=20240315
+```
+
+Undo script naming convention: `U20240315__expand_add_email_v2.sql` (prefix `U` instead of `V`).
+
+### Zero-Downtime Index Creation (PostgreSQL)
+
+```sql
+-- Blocking (avoid in production):
+CREATE INDEX idx_users_email ON users(email);
+
+-- Non-blocking (safe for live systems):
+CREATE INDEX CONCURRENTLY idx_users_email ON users(email);
+```
+
+Use `CREATE INDEX CONCURRENTLY` in all production migrations. Add it to a pre-deploy migration step that runs before the application update.
+
+---
+
+## Blue-Green with Database (Full Example)
+
+```yaml
+# blue-green-deploy.yml
+steps:
+  - name: Deploy green environment
+    run: |
+      kubectl apply -f k8s/green/
+
+  - name: Run database migrations (forward, backward-compatible)
+    run: |
+      kubectl exec -n production deploy/migration-job -- flyway migrate
+
+  - name: Smoke test green
+    run: |
+      kubectl port-forward -n production svc/my-app-green 8080:80 &
+      sleep 3
+      curl -sf http://localhost:8080/health/ready
+
+  - name: Switch traffic to green
+    run: |
+      kubectl patch service my-app \
+        -p '{"spec":{"selector":{"slot":"green"}}}'
+
+  - name: Verify green is live
+    run: ./scripts/verify-deployment.sh https://app.example.com
+
+  - name: Scale down blue
+    run: |
+      kubectl scale deployment my-app-blue --replicas=0
+
+  - name: Rollback to blue on failure
+    if: failure()
+    run: |
+      kubectl patch service my-app \
+        -p '{"spec":{"selector":{"slot":"blue"}}}'
+      kubectl scale deployment my-app-blue --replicas=5
+```
+
+---
+
+## Deployment Freeze Automation
+
+Automatically block production deployments during freeze windows:
+
+```python
+#!/usr/bin/env python3
+# scripts/check-freeze-window.py
+import sys
+from datetime import datetime, timezone
+
+FREEZE_WINDOWS = [
+    # (month, day_start, day_end, description)
+    (11, 25, 30, "US Thanksgiving"),
+    (12, 20, 31, "Year-end freeze"),
+    (1, 1, 2, "New Year"),
+]
+
+now = datetime.now(timezone.utc)
+month, day = now.month, now.day
+
+for fm, d_start, d_end, label in FREEZE_WINDOWS:
+    if fm == month and d_start <= day <= d_end:
+        print(f"BLOCKED: Deployment freeze active — {label}")
+        print("Override with FORCE_DEPLOY=true environment variable if critical.")
+        if not os.environ.get("FORCE_DEPLOY"):
+            sys.exit(1)
+
+print("No active freeze window — deployment allowed.")
+```
+
+Use in pipeline:
+
+```yaml
+- name: Check deployment freeze window
+  run: python scripts/check-freeze-window.py
+  env:
+    FORCE_DEPLOY: ${{ vars.FORCE_DEPLOY }}
+```
+
+---
+
+## Notification Templates
+
+### Slack Deployment Notification
+
+```bash
+#!/usr/bin/env bash
+# scripts/notify-slack.sh
+STATUS="${1:?pass 'success' or 'failure'}"
+WEBHOOK="${SLACK_WEBHOOK:?SLACK_WEBHOOK not set}"
+REPO="${GITHUB_REPOSITORY:-unknown}"
+SHA="${GITHUB_SHA:-unknown}"
+ACTOR="${GITHUB_ACTOR:-unknown}"
+
+if [ "$STATUS" = "success" ]; then
+  COLOR="good"
+  EMOJI=":white_check_mark:"
+  TEXT="Production deploy succeeded"
+else
+  COLOR="danger"
+  EMOJI=":red_circle:"
+  TEXT="Production deploy FAILED — rollback triggered"
+fi
+
+curl -sf -X POST "$WEBHOOK" \
+  -H "Content-Type: application/json" \
+  -d "{
+    \"attachments\": [{
+      \"color\": \"$COLOR\",
+      \"text\": \"$EMOJI $TEXT\",
+      \"fields\": [
+        {\"title\": \"Repo\", \"value\": \"$REPO\", \"short\": true},
+        {\"title\": \"SHA\", \"value\": \"${SHA:0:7}\", \"short\": true},
+        {\"title\": \"Deployed by\", \"value\": \"$ACTOR\", \"short\": true}
+      ]
+    }]
+  }"
+```
@@ -0,0 +1,399 @@
+# deployment-pipeline-design — detailed patterns and worked examples
+
+## Pipeline Stages
+
+### Standard Pipeline Flow
+
+```
+┌─────────┐   ┌──────┐   ┌─────────┐   ┌────────┐   ┌──────────┐
+│  Build  │ → │ Test │ → │ Staging │ → │ Approve│ → │Production│
+└─────────┘   └──────┘   └─────────┘   └────────┘   └──────────┘
+```
+
+### Detailed Stage Breakdown
+
+1. **Source** - Code checkout, dependency graph resolution
+2. **Build** - Compile, package, containerize, sign artifacts
+3. **Test** - Unit, integration, SAST/SCA security scans
+4. **Staging Deploy** - Deploy to staging environment with smoke tests
+5. **Integration Tests** - E2E, contract tests, performance baselines
+6. **Approval Gate** - Manual or automated metric-based gate
+7. **Production Deploy** - Canary, blue-green, or rolling strategy
+8. **Verification** - Deep health checks, synthetic monitoring
+9. **Rollback** - Automated rollback on failure signals
+
+## Approval Gate Patterns
+
+### Pattern 1: Manual Approval (GitHub Actions)
+
+```yaml
+production-deploy:
+  needs: staging-deploy
+  environment:
+    name: production
+    url: https://app.example.com
+  runs-on: ubuntu-latest
+  steps:
+    - name: Deploy to production
+      run: kubectl apply -f k8s/production/
+```
+
+Environment protection rules in GitHub enforce required reviewers before this job starts. Configure reviewers at **Settings → Environments → production → Required reviewers**.
+
+### Pattern 2: Time-Based Approval (GitLab CI)
+
+```yaml
+deploy:production:
+  stage: deploy
+  script:
+    - deploy.sh production
+  environment:
+    name: production
+  when: delayed
+  start_in: 30 minutes
+  only:
+    - main
+```
+
+### Pattern 3: Multi-Approver (Azure Pipelines)
+
+```yaml
+stages:
+  - stage: Production
+    dependsOn: Staging
+    jobs:
+      - deployment: Deploy
+        environment:
+          name: production
+          resourceType: Kubernetes
+        strategy:
+          runOnce:
+            preDeploy:
+              steps:
+                - task: ManualValidation@0
+                  inputs:
+                    notifyUsers: "team-leads@example.com"
+                    instructions: "Review staging metrics before approving"
+```
+
+### Pattern 4: Automated Metric Gate
+
+Use an AnalysisTemplate (Argo Rollouts) or a custom gate script to block promotion when error rates exceed a threshold:
+
+```yaml
+# Argo Rollouts AnalysisTemplate — blocks canary promotion automatically
+apiVersion: argoproj.io/v1alpha1
+kind: AnalysisTemplate
+metadata:
+  name: success-rate
+spec:
+  metrics:
+  - name: success-rate
+    interval: 60s
+    successCondition: "result[0] >= 0.95"
+    failureCondition: "result[0] < 0.90"
+    inconclusiveLimit: 3
+    provider:
+      prometheus:
+        address: http://prometheus:9090
+        query: |
+          sum(rate(http_requests_total{status!~"5..",job="my-app"}[2m]))
+          / sum(rate(http_requests_total{job="my-app"}[2m]))
+```
+
+## Deployment Strategies
+
+### Decision Table
+
+| Strategy     | Downtime | Rollback Speed | Cost Impact     | Best For                        |
+|-------------|----------|----------------|-----------------|----------------------------------|
+| Rolling      | None     | ~minutes       | None            | Most stateless services          |
+| Blue-Green   | None     | Instant        | 2x infra (temp) | High-risk or database migrations |
+| Canary       | None     | Instant        | Minimal         | High-traffic, metric-driven      |
+| Recreate     | Yes      | Fast           | None            | Dev/test, batch jobs             |
+| Feature Flag | None     | Instant        | None            | Gradual feature exposure         |
+
+### 1. Rolling Deployment
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: my-app
+spec:
+  replicas: 10
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxSurge: 2         # at most 12 pods during rollout
+      maxUnavailable: 1   # at least 9 pods always serving
+```
+
+Characteristics: gradual rollout, zero downtime, easy rollback, best for most applications.
+
+### 2. Blue-Green Deployment
+
+```bash
+# Switch traffic from blue to green
+kubectl apply -f k8s/green-deployment.yaml
+kubectl rollout status deployment/my-app-green
+
+# Flip the service selector
+kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'
+
+# Rollback instantly if needed
+kubectl patch service my-app -p '{"spec":{"selector":{"version":"blue"}}}'
+```
+
+Characteristics: instant switchover, easy rollback, doubles infrastructure cost temporarily, good for high-risk deployments with long warm-up times.
+
+### 3. Canary Deployment (Argo Rollouts)
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Rollout
+metadata:
+  name: my-app
+spec:
+  replicas: 10
+  strategy:
+    canary:
+      analysis:
+        templates:
+          - templateName: success-rate
+        startingStep: 2
+      steps:
+        - setWeight: 10
+        - pause: { duration: 5m }
+        - setWeight: 25
+        - pause: { duration: 5m }
+        - setWeight: 50
+        - pause: { duration: 10m }
+        - setWeight: 100
+```
+
+Characteristics: gradual traffic shift, real-user metric validation, automated promotion or rollback, requires Argo Rollouts or a service mesh.
+
+### 4. Feature Flags
+
+```python
+from flagsmith import Flagsmith
+
+flagsmith = Flagsmith(environment_key="API_KEY")
+
+if flagsmith.has_feature("new_checkout_flow"):
+    process_checkout_v2()
+else:
+    process_checkout_v1()
+```
+
+Characteristics: deploy without releasing, A/B testing, instant rollback per user segment, granular control independent of deployment.
+
+## Pipeline Orchestration
+
+### Multi-Stage Pipeline Example (GitHub Actions)
+
+```yaml
+name: Production Pipeline
+
+on:
+  push:
+    branches: [main]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    outputs:
+      image: ${{ steps.build.outputs.image }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Build and push Docker image
+        id: build
+        run: |
+          IMAGE=myapp:${{ github.sha }}
+          docker build -t $IMAGE .
+          docker push $IMAGE
+          echo "image=$IMAGE" >> $GITHUB_OUTPUT
+
+  test:
+    needs: build
+    runs-on: ubuntu-latest
+    steps:
+      - name: Unit tests
+        run: make test
+      - name: Security scan
+        run: trivy image ${{ needs.build.outputs.image }}
+
+  deploy-staging:
+    needs: test
+    environment:
+      name: staging
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy to staging
+        run: kubectl apply -f k8s/staging/
+
+  integration-test:
+    needs: deploy-staging
+    runs-on: ubuntu-latest
+    steps:
+      - name: Run E2E tests
+        run: npm run test:e2e
+
+  deploy-production:
+    needs: integration-test
+    environment:
+      name: production        # blocks here until required reviewers approve
+    runs-on: ubuntu-latest
+    steps:
+      - name: Canary deployment
+        run: |
+          kubectl apply -f k8s/production/
+          kubectl argo rollouts promote my-app
+
+  verify:
+    needs: deploy-production
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deep health check
+        run: |
+          for i in {1..12}; do
+            STATUS=$(curl -sf https://app.example.com/health/ready | jq -r '.status')
+            [ "$STATUS" = "ok" ] && exit 0
+            sleep 10
+          done
+          exit 1
+      - name: Notify on success
+        run: |
+          curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
+            -d '{"text":"Production deployment successful: ${{ github.sha }}"}'
+```
+
+## Health Checks
+
+### Shallow vs Deep Health Endpoints
+
+A shallow `/ping` returns 200 even when downstream dependencies are broken. Use a deep readiness endpoint that verifies actual dependencies before promoting traffic.
+
+```python
+# /health/ready — checks real dependencies, used by pipeline gate
+@app.get("/health/ready")
+async def readiness():
+    checks = {
+        "database": await check_db_connection(),
+        "cache":    await check_redis_connection(),
+        "queue":    await check_queue_connection(),
+    }
+    status = "ok" if all(checks.values()) else "degraded"
+    code = 200 if status == "ok" else 503
+    return JSONResponse({"status": status, "checks": checks}, status_code=code)
+```
+
+### Post-Deployment Verification Script
+
+```bash
+#!/usr/bin/env bash
+# verify-deployment.sh — run after every production deploy
+set -euo pipefail
+
+ENDPOINT="${1:?usage: verify-deployment.sh <base-url>}"
+MAX_ATTEMPTS=12
+SLEEP_SECONDS=10
+
+for i in $(seq 1 $MAX_ATTEMPTS); do
+  STATUS=$(curl -sf "$ENDPOINT/health/ready" | jq -r '.status' 2>/dev/null || echo "unreachable")
+  if [ "$STATUS" = "ok" ]; then
+    echo "Health check passed after $((i * SLEEP_SECONDS))s"
+    exit 0
+  fi
+  echo "Attempt $i/$MAX_ATTEMPTS: status=$STATUS — retrying in ${SLEEP_SECONDS}s"
+  sleep "$SLEEP_SECONDS"
+done
+
+echo "Health check failed after $((MAX_ATTEMPTS * SLEEP_SECONDS))s"
+exit 1
+```
+
+## Rollback Strategies
+
+### Automated Rollback in Pipeline
+
+```yaml
+deploy-and-verify:
+  steps:
+    - name: Deploy new version
+      run: kubectl apply -f k8s/
+
+    - name: Wait for rollout
+      run: kubectl rollout status deployment/my-app --timeout=5m
+
+    - name: Post-deployment health check
+      id: health
+      run: ./scripts/verify-deployment.sh https://app.example.com
+
+    - name: Rollback on failure
+      if: failure()
+      run: |
+        kubectl rollout undo deployment/my-app
+        echo "Rolled back to previous revision"
+```
+
+### Manual Rollback Commands
+
+```bash
+# List revision history with change-cause annotations
+kubectl rollout history deployment/my-app
+
+# Rollback to previous version
+kubectl rollout undo deployment/my-app
+
+# Rollback to a specific revision
+kubectl rollout undo deployment/my-app --to-revision=3
+
+# Verify rollback completed
+kubectl rollout status deployment/my-app
+```
+
+For advanced rollback strategies including database migration rollbacks and Argo Rollouts abort flows, see [`references/advanced-strategies.md`](references/advanced-strategies.md).
+
+## Monitoring and Metrics
+
+### Key DORA Metrics to Track
+
+| Metric                    | Target (Elite) | How to Measure                           |
+|--------------------------|----------------|------------------------------------------|
+| Deployment Frequency      | Multiple/day   | Pipeline run count per day               |
+| Lead Time for Changes     | < 1 hour       | Commit timestamp → production deploy     |
+| Change Failure Rate       | < 5%           | Failed deploys / total deploys           |
+| Mean Time to Recovery     | < 1 hour       | Incident open → service restored         |
+
+### Post-Deployment Metric Verification
+
+```yaml
+- name: Verify error rate post-deployment
+  run: |
+    sleep 60  # allow metrics to accumulate
+
+    ERROR_RATE=$(curl -sf "$PROMETHEUS_URL/api/v1/query" \
+      --data-urlencode 'query=sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))' \
+      | jq '.data.result[0].value[1]')
+
+    echo "Current error rate: $ERROR_RATE"
+    if (( $(echo "$ERROR_RATE > 0.01" | bc -l) )); then
+      echo "Error rate $ERROR_RATE exceeds 1% threshold — triggering rollback"
+      exit 1
+    fi
+```
+
+## Pipeline Best Practices
+
+1. **Fail fast** — Run quick checks (lint, unit tests) before slow ones (E2E, security scans)
+2. **Parallel execution** — Run independent jobs concurrently to minimize total pipeline time
+3. **Caching** — Cache dependency layers and build artifacts between runs
+4. **Artifact promotion** — Build once, promote the same artifact through all environments
+5. **Environment parity** — Keep staging infrastructure as close to production as possible
+6. **Secrets management** — Use secret stores (Vault, AWS Secrets Manager, GitHub encrypted secrets) — never hardcode
+7. **Deployment windows** — Prefer low-traffic windows; enforce change freeze periods via gate policies
+8. **Idempotent deploys** — Ensure re-running a deploy produces the same result
+9. **Rollback automation** — Trigger rollback automatically on health check or metric threshold failure
+10. **Annotate deployments** — Send deployment markers to monitoring tools (Datadog, Grafana) for correlation