CI/CD for Python
Continuous Integration and Continuous Delivery automate the checks (tests, linting, type checking) and steps (build, package, deploy) that turn a code change into a trustworthy, shippable release.
CI/CD is a workflow pattern, not a Python-specific technology: Continuous Integration means every code change automatically runs a suite of checks (tests, linting, type checking) before it can merge, and Continuous Delivery/Deployment means a passing change can be automatically packaged and shipped toward production with minimal manual steps. Applied to Python, CI/CD pipelines typically run on a third-party or platform CI service (GitHub Actions, GitLab CI, CircleCI, Jenkins, and others — none of these are part of Python itself) and orchestrate the standard-library and third-party tools covered elsewhere in this course: unittest/pytest for tests, Ruff/Black for linting and formatting, mypy/pyright for type checking, and build/pip/uv for packaging.
What Is CI/CD
Continuous Integration is the practice of merging small changes frequently, with automated checks gating every merge — the goal is to catch problems (a failing test, a type error, a style violation) within minutes of them being introduced, rather than discovering them days later when many changes have piled up and the cause is hard to isolate. Continuous Delivery extends this: once a change passes CI, it's automatically built into a deployable artifact and, ideally, could be released at any time with a single action. Continuous Deployment goes one step further and actually deploys every passing change automatically, with no manual approval gate.
None of this is Python-specific machinery — it's a general software-engineering discipline. What's Python-specific is *which* tools a Python pipeline typically wires together at each stage.
Automated Testing in CI
The first and most fundamental CI stage is running the test suite (see the Testing topics for unittest and the third-party pytest) on every push/pull request, across whatever combination of Python versions and operating systems the project supports. A red (failing) test suite should block a merge — that's the entire mechanism by which CI prevents regressions from landing.
A build matrix runs the same job across multiple Python versions in parallel, catching version-specific regressions before release.
# .github/workflows/ci.yml (excerpt, third-party GitHub Actions)
name: CI
on: [push, pull_request]
jobs:
test:
strategy:
matrix:
python-version: ["3.11", "3.12", "3.13"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: pip install -e ".[test]"
- run: pytest --cov=myappLinting in CI
Formatting/linting checks (see the Formatting & Linting topic — Ruff, Black, Flake8, all third-party) run as their own CI step, usually in "check, don't fix" mode: the job fails if the code isn't already formatted/lint-clean, rather than silently rewriting a contributor's PR. This keeps the responsibility for running the formatter locally (or via a pre-commit hook) with the author, while CI acts as the final backstop.
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install ruff
- run: ruff check .
- run: ruff format --check . # fails if files aren't already formattedType Checking in CI
Type checking (see the Type Checking & Static Analysis topic — mypy/pyright, third-party) runs as another gated step, catching type errors before merge the same way tests catch behavioral regressions. Because a full type check can be slower than tests on very large codebases, some pipelines cache the checker's own incremental-analysis cache between runs to keep CI fast.
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install mypy
- run: mypy src/Packaging and Build Artifacts
Once checks pass, a delivery pipeline builds a distributable artifact — for a Python library, that's a wheel (.whl, a pre-built binary-compatible distribution format) and/or an sdist (source distribution), built via the standard build frontend against the project's pyproject.toml metadata (see the related Packaging & Distribution topic for depth). For an application rather than a library, the artifact is more often a container image (built via Docker, third-party) bundling the interpreter, dependencies, and application code together.
needs: [...] makes the build job wait until test, lint, and typecheck all pass -- a broken pipeline never produces a release artifact.
build:
needs: [test, lint, typecheck]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install build
- run: python -m build # produces dist/*.whl and dist/*.tar.gz
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/Deployment Concept
Deployment is the step that takes a built artifact and makes it live somewhere: publishing a package to PyPI (or a private index) for a library, or rolling a new container image out to a server/cluster for an application. CI/CD pipelines commonly gate deployment behind additional conditions beyond "tests passed" — a merge to a specific branch (e.g. main), a manually approved release tag, or a staged rollout through environments (dev → staging → production) so a bad change can be caught in a lower environment before reaching real users.
This is a conceptual overview, not a deployment tutorial — the exact deployment mechanism (PyPI publish, container registry push, server rollout) depends heavily on what's being shipped and where; the constant across all of them is: only ship an artifact that passed every gate before it.
publish:
needs: build
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: production # can require manual approval before running
steps:
- uses: actions/download-artifact@v4
with: { name: dist, path: dist/ }
- run: pip install twine
- run: twine upload dist/* # publish to PyPI (third-party tool)Caching and Pipeline Speed
A slow CI pipeline erodes the whole benefit of fast feedback — if checks take twenty minutes, developers stop waiting for them before moving on, which defeats the purpose. Caching dependency installs (pip's or uv's download/build cache) between runs, running independent jobs (test/lint/typecheck) in parallel rather than sequentially, and only running the full matrix on the branches that need it (e.g. skip the multi-OS matrix on every draft commit, run it only before merge) are the standard levers.
Best Practices for CI/CD Pipelines
A few practices consistently separate pipelines that are trusted from ones that get ignored: keep pipeline configuration in version control alongside the code it checks (so pipeline changes go through the same review as everything else); make failures actionable (a clear error message, not a cryptic log dump); keep the pipeline itself fast via caching and parallelism; and never let a flaky (intermittently failing for unrelated reasons) check stay in the required set — a flaky gate trains people to ignore *all* failures, including real ones.
Complete Example
# .github/workflows/ci.yml -- full pipeline shape
name: CI
on: [push, pull_request]
jobs:
test:
strategy: { matrix: { python-version: ["3.12", "3.13"] } }
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: ${{ matrix.python-version }} }
- run: pip install -e ".[test]"
- run: pytest
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install ruff
- run: ruff check .
- run: ruff format --check .
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install mypy
- run: mypy src/
build:
needs: [test, lint, typecheck]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install build
- run: python -m buildQuick Reference
Stage Tools (all third-party / platform-provided)
------------ ---------------------------------------------------
Test pytest (or stdlib unittest)
Lint Ruff / Black / Flake8
Type check mypy / pyright
Build python -m build (produces .whl and sdist)
Publish twine (PyPI) / docker (container images)
Platforms GitHub Actions, GitLab CI, CircleCI, Jenkins, etc.
CI = automated checks gate every merge
CD = a passing change is automatically built (Delivery) or
also automatically released (Deployment)Key Takeaways
CI/CD is a general engineering discipline, not Python-specific — Python pipelines wire together pytest, Ruff/Black, mypy/pyright, and a build tool on top of a CI platform.
CI gates every merge on automated checks (tests, lint, type check); CD automates building (Delivery) or also releasing (Deployment) what passes.
A CI/CD pipeline's checks should run in "verify, don't silently fix" mode — failing the build rather than rewriting a contributor's code.
Deployment should be gated by more than "tests passed" — branch protections, staged environments, or manual approval keep bad changes from reaching production.
A slow or flaky pipeline erodes trust; cache dependencies, parallelize independent jobs, and never leave a flaky check in the required gate.
Having CI silently rewrite and push code back removes the author's opportunity to review the diff and can mask the fact that local tooling isn't configured -- CI should verify, and local/pre-commit hooks should fix.
A required check that's known to fail intermittently for unrelated reasons trains developers to click 'rerun' or ignore red builds altogether, undermining trust in every other check.
Deploying from any branch on any push means unreviewed, in-progress work can reach production; gating deployment to protected branches/tags is a basic safety boundary.
What is the difference between Continuous Delivery and Continuous Deployment?
Continuous Delivery means every change that passes CI is automatically built into a releasable artifact, but a human still decides when/whether to actually release it. Continuous Deployment goes further and releases every passing change automatically, with no manual gate.
Name three kinds of automated checks a typical Python CI pipeline gates a merge on.
Automated tests (pytest/unittest), linting/formatting checks (Ruff/Black/Flake8), and type checking (mypy/pyright) -- often alongside a build step to confirm the package actually builds.
Why should a lint check in CI run in "fail if not already clean" mode rather than auto-fixing and committing?
Auto-fixing and pushing in CI removes the author's chance to review what changed and hides misconfigured local tooling; CI should verify, leaving fixing to the author's local run or a pre-commit hook.
What is a build matrix, and why would a Python project use one in CI?
A build matrix runs the same CI job across multiple combinations of parameters (e.g. Python 3.11/3.12/3.13, multiple operating systems) in parallel, catching version- or platform-specific regressions before release.
What are the two standard build artifacts produced for a Python library, and what tool commonly produces them?
A wheel (.whl, a pre-built distribution) and an sdist (source distribution, .tar.gz), typically produced by running python -m build against the project's pyproject.toml metadata.
Why is a flaky required check worse than having no check for that behavior at all?
A flaky check fails intermittently for reasons unrelated to the actual change, training developers to ignore or rerun red builds reflexively -- which erodes trust in every check, including ones catching real regressions.
Write the minimal GitHub Actions job steps to install a project and run its pytest suite on Python 3.12 (assume a pyproject.toml with a [test] extra).
jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.12" - run: pip install -e ".[test]" - run: pytest checkout pulls the repo, setup-python provisions the interpreter, pip install -e ".[test]" installs the project plus its test dependencies in editable mode, and pytest runs the suite -- the minimal shape of a test gate.
Given three CI jobs test, lint, and build where build should only run after both test and lint succeed, write the GitHub Actions dependency declaration for build.
jobs: build: needs: [test, lint] runs-on: ubuntu-latest steps: - run: echo "building only after test and lint pass"
The needs: [test, lint] field makes build wait for both listed jobs to succeed before it starts, and GitHub Actions skips build automatically if either dependency fails -- this is how a pipeline enforces 'only ship what passed every gate'.
Explain (in a short comment) why a CD pipeline for a public library typically gates the twine upload (publish to PyPI) step behind if: github.ref == 'refs/heads/main' rather than running it on every pull request.
Restricting publish to the main branch (or a release tag) ensures the artifact being shipped has actually passed review and merged, rather than shipping speculative work from an open PR -- and avoids burning immutable version numbers on unreviewed commits.