CI/CD and Build Automation

What this module covers

CI, continuous delivery, and continuous deployment Pipeline stages and job dependencies

Trigger rules and pull-request validation Build matrices and environment variables Caching dependencies and build outputs

Test, lint, static analysis, and coverage gates Artifact upload, retention, and promotion Secrets management in CI Reusable pipeline templates Self-hosted versus managed runners

Failure diagnosis, retries, and flaky tests Release automation and rollback strategy

14.1 CI/CD concepts

Continuous Integration (CI): integrate changes frequently and validate them automatically.

Continuous Delivery: keep software in a releasable state; production release may require approval.

Continuous Deployment: qualifying changes are automatically deployed to production.

A pipeline often performs:

Checkout → Validate → Resolve Dependencies → Build → Test
       → Static Analysis/Security Scan → Package → Publish Artifact
       → Approval/Promotion → Deploy/Release

14.2 Pipeline design principles

Fail early on syntax, formatting, or configuration problems.

Make build and test commands reproducible locally and in CI.

Separate untrusted validation jobs from privileged release jobs.

Cache only data whose reuse is safe and correctly keyed.

Keep secrets out of logs and build artifacts.

Upload test reports even when a later stage fails, when feasible.

Retain artifacts according to policy.

Promote the same verified artifact rather than rebuilding differently for each environment.

14.3 Example generic CI pipeline pseudocode

This is platform-neutral pseudocode, not a directly executable provider-specific workflow:

Real providers use their own YAML schema, expression language, permissions model, and runner configuration. Convert this plan to the platform's official syntax before use.

Text equivalent: the validation job checks out, validates, restores locked dependencies, builds, tests, scans and packages; the artifact is published once and then promoted through environments after approval, rather than rebuilt for each one.

Try it: pipeline simulator

pipeline:
  on:
    - pull_request
    - push_to_main

  jobs:
    validate:
      steps:
        - checkout
        - install_pinned_toolchain
        - restore_dependencies_from_lockfile
        - run_format_check
        - run_static_analysis
        - build
        - run_unit_tests
        - upload_test_reports

    release:
      condition: trusted_release_tag
      needs: [validate]
      steps:
        - checkout_exact_tag_commit
        - build_release_artifact
        - run_release_tests
        - generate_sbom
        - sign_artifact
        - publish_artifact

flowchart TD
  CO[Checkout] --> V[Validate]
  V --> D[Resolve deps from lockfile]
  D --> B[Build]
  B --> T[Test]
  T --> S[Scan]
  S --> P[Package]
  P --> PUB[Publish artifact]
  PUB --> PR[Approval and promotion]

14.4 Caching in CI

Good cache candidates:

Package-manager download caches.

Compiler caches with correct toolchain/input keys.

Avoid treating a cache as the source of truth. A cache miss must still produce a correct build. Separate caches by OS, architecture, compiler, configuration, and trust boundary when needed.

14.5 Artifacts and promotion

A build artifact is a produced binary/package/test report. A release process should record:

Source commit and tag.

Toolchain and dependency versions.

Test and scan results.

Artifact digest/checksum.

Signing identity and verification instructions.

Promote the tested artifact across stages where possible; avoid silently rebuilding a different binary after approval.

14.6 Flaky tests and retries

Retries can reduce transient noise but can also hide defects. Track flaky-test rate, capture logs, isolate shared resources, and fix root causes. A pipeline should distinguish infrastructure failures from genuine product failures.

14.7 Secrets and permissions

Use the CI provider's secret store.

Grant minimum permissions to each job.

Avoid exposing release secrets to untrusted pull requests.

Prefer short-lived identity federation over long-lived tokens where available.

Rotate credentials and audit access.

Masking logs is helpful but not a reason to print secrets.

Quiz

? Why promote the same artifact instead of rebuilding for production?
+ The binary that was tested is the binary that ships. :: A rebuild can differ.
- Rebuilding is not possible in CI. :: It is possible, just risky.
- Promotion skips tests. :: The tests already ran on that artifact.
- Artifacts cannot be signed after rebuild. :: They can; it is still a different binary.

? A cache restore fails in CI. What should happen?
+ The build still succeeds from scratch, just slower. :: A cache is an optimisation, never the source of truth.
- The pipeline fails. :: A cache miss must not break correctness.
- The previous artifact is reused. :: That would ship stale output.
- Tests are skipped. :: No.

? Which secret-handling rules apply to CI? (select all that apply)
+ Do not expose release secrets to untrusted pull requests :: Forked PRs can run arbitrary code.
+ Grant each job the minimum permissions :: Limits blast radius.
+ Prefer short-lived identity federation over long-lived tokens :: Less to steal and rotate.
- Print secrets once to confirm they are set :: Masking is not a reason to print secrets.