Module 7: CI Integration with GitHub Actions

Coverage in CI and Branch Protection

Coverage in CI and Branch Protection

Capsule overview

You have tests running in CI and coverage reports generated locally. But if coverage isn't part of the pipeline, nobody reviews it before the merge. A PR can drop the coverage from 90% to 60% and get merged without anyone noticing. And without branch protection, someone can merge even when the tests fail.

This capsule closes the loop: adding coverage to the CI pipeline, publishing it as an artifact, showing it as a comment on PRs, and configuring branch protection so that no broken code or code with insufficient coverage reaches main. By the end you'll have a complete professional pipeline that combines tests, coverage, PR reports and branch protections. Also a prompt so Claude Code generates the whole configuration.


Adding coverage to the CI pipeline

The pytest-cov dependency

Make sure pytest-cov is in your dependencies:

# requirements.txt
pytest>=7.0.0
pytest-cov>=4.0.0

Or in pyproject.toml:

[project.optional-dependencies]
dev = ["pytest", "pytest-cov"]

The coverage step in the workflow

- name: Run tests with coverage
  run: |
    pytest tests/ -v --cov=src --cov-report=term-missing --cov-report=html --cov-report=xml
  • --cov=src: Measures the src package.
  • --cov-report=term-missing: Shows the uncovered lines in the terminal.
  • --cov-report=html: Generates htmlcov/ for the HTML report.
  • --cov-report=xml: Generates coverage.xml, needed for many actions that comment on PRs.

A minimal workflow with coverage

# .github/workflows/tests.yml
name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests with coverage
        run: |
          pytest tests/ -v --cov=src --cov-report=term-missing --cov-report=html --cov-report=xml

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: coverage-report
          path: htmlcov/

The coverage-report artifact contains the htmlcov/ directory. After the run, you download it, unzip it and open index.html to see the visual report.


Publishing coverage as an artifact

Why htmlcov?

The HTML report is visual, easy to explore and doesn't require external services. Uploading it as an artifact lets any reviewer download the report and check which lines were left uncovered after a PR.

Configuring the upload

- name: Upload coverage report
  uses: actions/upload-artifact@v4
  if: always()  # Uploads even if the tests fail
  with:
    name: coverage-report-${{ github.run_number }}
    path: htmlcov/

run_number makes each run have an artifact with a unique name. If you'd rather it be overwritten, use just name: coverage-report.

Artifact retention

By default, artifacts are kept for 90 days. In Settings → Actions → General you can change the retention. For projects with many PRs, 7-30 days is usually enough.


Coverage as a comment on PRs

The coverage-comment action

There are several actions that publish the coverage as a comment on the PR. A popular one is py-cov-action or coverage-comment. The idea: after the tests, the action reads coverage.xml (or the coverage report) and writes a comment on the PR with the percentage and a coverage diff per file.

An example with py-cov-action

- name: Run tests with coverage
  run: |
    pytest tests/ -v --cov=src --cov-report=xml

- name: Comment coverage on PR
  if: github.event_name == 'pull_request'
  uses: py-cov-action/python-coverage-report-action@v1
  with:
    badge: true
    fail_below: 80

This action comments on the PR with a coverage badge and a table. fail_below: 80 makes the job fail if the coverage is below 80%, blocking the merge.

An alternative: coverage-comment

- name: Run tests with coverage
  run: |
    pytest tests/ -v --cov=src --cov-report=xml

- name: Post coverage comment
  if: github.event_name == 'pull_request'
  uses: py-cov-action/python-coverage-report-action@v1
  with:
    coverage-files: coverage.xml

Check the documentation of the action you choose for the exact parameters; they can change between versions.

What to expect in the comment

The comment typically includes:

  • The total coverage percentage.
  • A table per file: name, coverage, covered/total lines.
  • An indication of whether the PR raises or lowers the coverage compared to the base (main).

Branch Protection Rules

What are they?

Branch protection rules are rules GitHub applies before allowing a merge. You can require certain checks to pass, that there be approved reviews, that there be no direct push to main, etc.

The basic configuration

  1. Repo → Settings → Branches.
  2. Add branch protection rule.
  3. Branch name pattern: main (or * for all).
  4. Enable:
    • Require a pull request before merging
    • Require status checks to pass before merging
    • Require branches to be up to date before merging (optional)
    • Do not allow bypassing the above settings

Required status checks

In "Require status checks to pass", select the name of the job or workflow that must pass. For example, if your job is called test, something like test (or the workflow's name) will appear. You must select at least one.

If no check appears, it's because the branch hasn't had a workflow run yet. Do a push or open a PR so the workflow runs once; afterwards the check will appear in the list.

Require reviews

You can require 1 or more approvals before the merge. For small teams, 1 is usually enough.

Not allowing a direct push to main

Enable "Do not allow bypassing" so that not even admins can push directly without going through a PR. Optional but recommended in teams.


Coverage thresholds (fail if it's below)

Option 1: pytest-cov --cov-fail-under

pytest-cov can fail if the coverage is below a threshold:

- name: Run tests with coverage
  run: |
    pytest tests/ -v --cov=src --cov-report=term-missing --cov-report=xml --cov-fail-under=80

If the coverage is under 80%, pytest returns exit code 2 (or similar) and the job fails. The PR won't be mergeable until the coverage goes up or the threshold is lowered.

Option 2: coverage.py in setup.cfg or pyproject.toml

[tool.coverage.run]
source = ["src"]
omit = ["tests/*", "*/__pycache__/*"]

[tool.coverage.report]
fail_under = 80

With that, pytest --cov=src will use this configuration and will fail if the coverage is below 80%.

Option 3: An external action

Some coverage actions (like py-cov-action) have a fail_below parameter. If you configure it, the action fails and the job fails.

Choosing the threshold

  • 70-80% for growing projects.
  • 90%+ for critical code or libraries.
  • Avoid an inflexible 100%: it usually generates trivial tests to cover impossible or unimportant lines.

Thresholds per type of code

In large projects you may want different thresholds per module. coverage.py and pytest-cov allow granular configuration:

# pyproject.toml
[tool.coverage.report]
fail_under = 70

[tool.coverage.report.fail_under]
# Critical files demand more
"src/auth/*" = 90
"src/payments/*" = 95

Not all versions of coverage support this; check the documentation. An alternative is to use a separate job that checks coverage only in src/auth/ with a higher threshold.

Excluding code that isn't tested

[tool.coverage.run]
source = ["src"]
omit = [
  "tests/*",
  "*/__pycache__/*",
  "*/migrations/*",
  "src/cli_main.py"
]

The files in omit don't count in the report. Useful for CLI scripts, DB migrations, or generated code.


Alternatives to py-cov-action

Besides py-cov-action/python-coverage-report-action, there are other actions for commenting coverage on PRs:

  • codecov/codecov-action: Uploads the report to Codecov.io and comments with a bot. It requires a Codecov account. It offers coverage history, charts and comparison between branches.
  • coverallsapp/github-action: Similar to Codecov, it uses Coveralls. Both integrate with GitHub and offer external dashboards.
  • dorny/paths-filter: It doesn't comment coverage but it lets the coverage job run only when relevant files change (e.g. code in src/), saving time on PRs that only touch docs.
  • EnricoMi/publish-unit-test-result-action: If you generate JUnit XML with coverage, some variants can include metrics. Check the documentation.

For projects that don't want external services, py-cov-action with a local coverage.xml is enough. For teams that want history and trends, Codecov or Coveralls are solid options.


A strategy for when a PR legitimately lowers the coverage

Sometimes a PR adds new code (features, refactors) that temporarily lowers the coverage: more lines without tests yet. Options:

  1. Lower the threshold temporarily: Not recommended; the threshold tends to stay low.
  2. Add tests in the same PR: The ideal. The PR doesn't merge until the coverage goes up.
  3. Exclude new files: With omit in coverage, you can exclude a file you don't test yet. Useful for code in development that will be tested in a later PR. Use sparingly.
  4. pragma: no cover on specific lines: For defensive or impossible-to-test lines, use # pragma: no cover. Don't abuse it.

The practical rule: if the PR adds 100 lines and none is covered, the PR isn't ready. Ask for tests or exclude them explicitly with justification in the code review.


The complete pipeline: tests + coverage + protection

Here's a workflow that integrates everything above.

# .github/workflows/tests.yml
name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests with coverage
        run: |
          pytest tests/ -v --cov=src --cov-report=term-missing --cov-report=html --cov-report=xml --cov-fail-under=80

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: coverage-report
          path: htmlcov/

      - name: Comment coverage on PR
        if: github.event_name == 'pull_request'
        uses: py-cov-action/python-coverage-report-action@v1
        with:
          badge: true
          fail_below: 80

Branch protection for this workflow

  1. Settings → Branches → Add rule.
  2. Branch name: main.
  3. Enable "Require status checks to pass".
  4. Select the test check (or the name GitHub assigns to the job).
  5. Enable "Require pull request reviews" (1 approval if you want).
  6. Save.

From then on, no PR will be mergeable into main if:

  • The tests fail.
  • The coverage is below 80%.
  • There's no approval (if you configured it).

Advanced branch protection configuration

Besides "Require status checks" and "Require reviews", you can enable:

  • Require branches to be up to date before merging: It forces the branch to be up to date with main before the merge. It prevents an outdated PR from merging code that could conflict with recent changes.
  • Require conversation resolution before merging: All the conversations in the PR must be resolved (no unresolved comments left).
  • Require linear history: It forces commits to be merged with squash or rebase, avoiding merges with multiple parents.
  • Require signed commits: It only accepts signed commits. Stricter, useful for teams with security processes.
  • Restrict who can push to matching branches: Only certain users or teams can push to main. Everyone else only via PR.
  • Allow force pushes: Disabled by default on protected branches. Enable it only on development branches, never on main.
  • Allow deletions: Normally disabled for main.

For Module 7's project, "Require status checks" and "Require a pull request" are enough. The advanced options are useful when the team grows or the compliance requirements increase.


Alternatives to py-cov-action

Besides py-cov-action/python-coverage-report-action, there are other actions for commenting coverage on PRs:

  • orhun/github-action-coverage-badge: Generates a coverage badge that can be committed to the repo or used in the README.
  • codecov/codecov-action: Uploads the report to Codecov (an external service) that integrates with GitHub and comments on PRs. It requires an account.
  • coveralls/github-action: Similar to Codecov, it uses Coveralls.io.
  • davelosert/video-downloader-action (doesn't apply) — An example of a generic name. Search GitHub Marketplace for "coverage comment" for more options.

For projects that prefer not to depend on external services, py-cov-action or similar actions that read coverage.xml and comment directly are the simplest option. Codecov and Coveralls offer coverage history between commits and dashboards, but they add an external dependency.


A strategy for when a PR legitimately lowers the coverage

Sometimes a PR adds new code (features, refactors) that isn't tested yet, and the global coverage drops. Options:

  1. Add tests in the same PR: The ideal. The developer writes tests for the new code before the merge.
  2. Temporarily lower the threshold: Not recommended as a habit. It creates debt.
  3. Use [pragma: no cover] on defensive code: For lines that should never run (impossible asserts, error branches that don't apply), you can mark # pragma: no cover to exclude them from the report.
  4. Omit files or directories: If you add an experimental or WIP module, you can temporarily omit it in [tool.coverage.run] omit until it's ready for production.
  5. Merge with an exception approval: Some teams allow a maintainer to "bypass" in exceptional cases, documenting the reason. It requires configuring "Allow specified actors to bypass" in branch protection.

The golden rule: the threshold must be reachable with reasonable effort. If 80% constantly blocks legitimate PRs, maybe the threshold is too high for the project's current phase. Better a sustainable 70% than an 80% nobody meets.


Claude Code: generating the complete CI configuration

A prompt for the complete pipeline

Generate the complete CI configuration for my Python project:

The structure:
- src/ with the code
- tests/ with pytest
- requirements.txt with pytest, pytest-cov

I need:
1. A .github/workflows/tests.yml workflow that runs on push and pull_request to main
2. Python 3.12
3. Tests with coverage (--cov=src, term-missing, html, xml reports)
4. A coverage threshold of 80% (--cov-fail-under=80)
5. An upload of the htmlcov report as an artifact
6. A coverage comment on PRs using py-cov-action
7. A matrix for Python 3.10, 3.11, 3.12
8. pip caching

The file must be runnable and complete from the very first line.

Claude Code will generate a workflow that integrates tests, coverage, artifacts, PR comments, matrix and caching. Review it, adjust the action's version if needed, and push.

Adjusting after generating

  • Action versions: Use actions/checkout@v4, actions/setup-python@v5, actions/upload-artifact@v4. Claude Code may use different versions; verify against each action's documentation.
  • The job's name: If you use a matrix, the job can have a composite name. In branch protection, select the correct check.
  • fail_below vs --cov-fail-under: They can be in the action or in pytest. Don't duplicate them with contradictory values.

Guided practice: from zero to a pipeline with coverage and protection

1. Create the minimal structure

mkdir -p cov-demo/src cov-demo/tests cov-demo/.github/workflows
cd cov-demo

2. Code and tests

# src/calc.py (and an empty src/__init__.py)
def add(a: float, b: float) -> float:
    return a + b

def unmapped(x: int) -> int:
    return x * 2  # No test, not covered
# tests/test_calc.py
from src.calc import add

def test_add():
    assert add(2, 3) == 5

3. requirements.txt

pytest>=7.0.0
pytest-cov>=4.0.0

4. A workflow with coverage

Create .github/workflows/tests.yml with the complete workflow (tests + coverage + artifact + fail-under 80). With a single test that covers add, the coverage will be low (~50%); the job will fail if you use --cov-fail-under=80. That demonstrates that the threshold works.

5. Push and open a PR

git init
git add .
git commit -m "Add project with coverage CI"
git remote add origin https://github.com/your-username/cov-demo.git
git push -u origin main

Create a feature/foo branch, add a test for unmapped that raises the coverage, push and open a PR. You'll see the workflow run. If the coverage goes above 80%, the check will pass.

6. Configure branch protection

In Settings → Branches → Add rule for main:

  • Require status checks: select test.
  • Require pull request before merging.
  • Save.

Try to merge a PR that lowers the coverage (for example, comment out a test). The merge will be blocked.


Alternatives for commenting coverage on PRs

Besides py-cov-action, there are other actions:

ActionCharacteristics
coverage-commentComments on PRs, supports multiple formats, can update the same comment on each push
codecov/codecov-actionUploads to Codecov.io; the comment includes links to the detailed report on their site
romeovs/lcov-report-actionGenerates a report from lcov; more common in JS/TS ecosystems
dorny/test-reporterA general reporter that can also show coverage if you generate it in a compatible format

For pure Python projects, py-cov-action or coverage-comment are usually enough. Codecov is useful if you want coverage history, trends and centralized dashboards (it requires an account on codecov.io).


A strategy for when a PR legitimately lowers the coverage

Sometimes a PR adds new code (features, refactors) that legitimately lowers the total percentage because it adds more lines than tests. Options:

  1. Write tests for the new code: The ideal. If the PR adds 100 lines, add tests that cover them.
  2. Temporarily lower the threshold: Not advisable as a habit; it erodes the discipline.
  3. Exclude specific modules: If the code is provisional or generated, add it to omit in coverage. Use carefully.
  4. Coverage per modified file: Some actions let you fail only if the coverage of the files the PR touches drops. More complex to configure but fairer.
  5. Approve as an exception: If you have "Allow specified actors to bypass required pull requests", a maintainer can merge in exceptional cases. Use it only when there's documented justification.

The healthiest practice: treat the threshold as a team commitment. If it's lowered, it must be an explicit decision in a meeting or RFC, not a silent bypass.


A visual summary: the pipeline flow with coverage

Push or PR to main
       │
       ▼
┌──────────────────┐
│ Check out code   │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Setup Python     │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Install deps     │
└────────┬─────────┘
         │
         ▼
┌──────────────────────────────┐
│ pytest --cov=src             │
│ --cov-report=html,xml        │
│ --cov-fail-under=80          │
└────────┬─────────────────────┘
         │
         ├── If tests fail or coverage < 80% → Job FAILED, merge blocked
         │
         ▼ (if it passes)
┌──────────────────┐
│ Upload htmlcov   │
│ (artifact)       │
└────────┬─────────┘
         │
         ▼ (if it's a PR)
┌──────────────────┐
│ Comment coverage │
│ on the PR        │
└──────────────────┘

Branch protection prevents a merge while the "test" check isn't green. That includes tests passing and coverage above the threshold.


A checklist for a professional CI pipeline

Before considering Module 7's project finished, verify:

  • The tests run on push and pull_request to main
  • At least two Python versions in a matrix (or one if the project declares it)
  • pip caching configured and working (see "Cache hit" in the logs)
  • Coverage generated with --cov-report=xml for PR comments
  • A coverage threshold configured (--cov-fail-under)
  • An htmlcov artifact to download the report
  • Branch protection with "Require status checks" enabled
  • The check appears in the branch protection list (once the workflow has run)
  • A documented Claude Code prompt to regenerate the workflow if the project's structure changes

A visual summary of the flow

Push / PR to main
       │
       ▼
┌──────────────┐
│   Checkout   │
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ Setup Python │
└──────┬───────┘
       │
       ▼
┌──────────────────────────┐
│ Install dependencies    │
└──────┬──────────────────┘
       │
       ▼
┌─────────────────────────────────┐
│ pytest + coverage (xml, html)   │
│ --cov-fail-under=80             │
└──────┬──────────────────────────┘
       │
       ├──▶ If it fails: job failed, merge blocked
       │
       ▼
┌──────────────────┐
│ Upload htmlcov   │
└──────┬───────────┘
       │
       ▼
┌─────────────────────────────┐
│ Comment coverage on the PR  │
│ (if event = pull_request)   │
└─────────────────────────────┘

Branch protection verifies that the "test" job (or your job's name) passed before allowing the merge. Without a green check, the Merge button is disabled.


Exercises

Exercise 1: Add coverage to the workflow (Basic)

You have a workflow that only runs pytest tests/ -v. Add pytest-cov, the --cov=src flag and the term-missing and html reports. Verify that the job generates htmlcov/.

See solution
- name: Install dependencies
  run: pip install -r requirements.txt
# Make sure requirements.txt has pytest-cov

- name: Run tests with coverage
  run: pytest tests/ -v --cov=src --cov-report=term-missing --cov-report=html

- name: Upload coverage report
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: coverage-report
    path: htmlcov/

Exercise 2: A coverage threshold (Basic)

Configure the workflow so it fails if the coverage is below 75%. Run it and verify that with low coverage the job fails.

See solution
- name: Run tests with coverage
  run: pytest tests/ -v --cov=src --cov-report=xml --cov-fail-under=75

Or in pyproject.toml:

[tool.coverage.report]
fail_under = 75

Exercise 3: Branch protection (Intermediate)

Configure a branch protection rule for main that requires the test check to pass before the merge. Open a PR with a failing test and verify that you can't merge.

See solution
  1. Settings → Branches → Add rule.
  2. Branch name pattern: main.
  3. Enable "Require status checks to pass before merging".
  4. Search for and select the test check (or "Tests / test").
  5. Enable "Require a pull request before merging".
  6. Save.

Create a branch, break a test, push, open a PR. The check will be red and the Merge button will be disabled.

Exercise 4: A coverage comment on a PR (Intermediate)

Integrate an action that comments the coverage on every PR. Use py-cov-action/python-coverage-report-action or similar. Verify that the comment appears when you open or update a PR.

See solution
- name: Run tests with coverage
  run: pytest tests/ -v --cov=src --cov-report=xml

- name: Comment coverage on PR
  if: github.event_name == 'pull_request'
  uses: py-cov-action/python-coverage-report-action@v1
  with:
    badge: true

Generate coverage.xml with --cov-report=xml. The action reads it and comments. Check the action's current documentation for parameter changes.

Exercise 5: A pipeline with matrix and coverage (Advanced)

Combine a matrix (Python 3.10, 3.11, 3.12) with coverage. Should the PR coverage comment run once or once per Python version? Explain and configure the most sensible option.

See solution

The most sensible: a single coverage report that represents one version (for example, 3.12). If you generate coverage in every matrix job, you'd have 3 different reports (they can vary slightly). For the PR comment, using just one avoids noise.

Options:

  1. A separate coverage job: A job that depends on test and only runs with Python 3.12, generating the coverage and commenting.
  2. A job without a matrix for coverage: A coverage job that runs with 3.12, generates the report and comments; the test job uses a matrix to validate that the tests pass on every version.

An example with a separate job:

jobs:
  test:
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v

  coverage:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: pytest tests/ --cov=src --cov-report=xml
      - uses: py-cov-action/python-coverage-report-action@v1
        if: github.event_name == 'pull_request'
        with:
          badge: true

Exercise 6: A prompt for Claude Code (Advanced)

Write a complete prompt for Claude Code that generates a workflow with: tests in a 3.10/3.11/3.12 matrix, pip caching, coverage with an 85% threshold, an htmlcov artifact, a PR comment, and branch protection (just the explanation of what to configure manually in Settings).

See solution
Generate the .github/workflows/tests.yml file for a Python project with:

- The code in src/, the tests in tests/
- requirements.txt with pytest, pytest-cov
- A matrix for Python 3.10, 3.11, 3.12
- Caching of pip dependencies using actions/cache with a key based on a hash of requirements.txt
- Tests with coverage --cov=src, xml and html reports
- A threshold of --cov-fail-under=85
- An upload of htmlcov as an artifact
- A coverage comment on PRs with py-cov-action (only on pull_request events)
- Triggers: push and pull_request to main

Include at the end a comment with instructions to configure branch protection in GitHub Settings → Branches: require the status check "test" (or the job's name), require a PR before merging.

Claude Code will generate the YAML. The branch protection instructions are for doing manually in the UI, since it can't be configured via YAML.


Troubleshooting

The coverage comment doesn't appear on the PR

Cause: The action may require coverage.xml at a specific path, or the event isn't pull_request, or the action fails silently.

Solution: Verify that --cov-report=xml generates coverage.xml at the root. Check that the step has if: github.event_name == 'pull_request'. Look at the action step's logs to see errors.

Branch protection doesn't show the check

Cause: The workflow hasn't run on that branch yet, or the check's name doesn't match.

Solution: Push or open a PR so the workflow runs. After one run, the check will appear in Settings → Branches when configuring the rule. The name is usually the job's (e.g. test) or "WorkflowName / JobName".

Coverage at 0% or "No data to report"

Cause: The --cov=src path doesn't match what you import in the tests, or the directory doesn't exist.

Solution: If you import from src.calc import add, the source must be src. Run it from the root. Verify that src/ exists and has an __init__.py if it's a package. If you use app/ instead of src/, use --cov=app.

fail_under makes it fail even though the coverage went up

Cause: There may be contradictory configuration (e.g. an omit that excludes too much, or branch coverage that counts differently).

Solution: Review [tool.coverage.run] and [tool.coverage.report] in pyproject.toml. Make sure source and omit are correct. Run pytest --cov=src --cov-fail-under=80 locally to reproduce it.

The py-cov-action action fails

Cause: Changes in the action's API, or coverage.xml doesn't exist or is at another path.

Solution: Check the action's current documentation. Some versions use coverage-files instead of auto-detection. Specify the path explicitly if necessary.

A strategy for when a PR legitimately lowers the coverage

Sometimes you add new code (e.g. an endpoint) that lowers the total percentage because it doesn't have tests yet. Options:

  1. Add tests in the same PR: The ideal option. The PR doesn't merge until the coverage goes up.
  2. Temporarily lower the threshold: Not recommended as a norm; it generates technical debt.
  3. Exclude new files with a pragma: In the new code, # pragma: no cover on the lines you temporarily don't test. coverage ignores them. Remove the pragma when you add the tests in a later PR.
  4. Branch protection with exceptions: Admins can have permission to bypass in emergencies. Use it with judgment; don't make it a habit.

Project Connection

This pipeline with coverage and branch protection is the deliverable of Module 7's project. Together with capsules 02-04 you have: a basic workflow, pytest in CI, matrix and caching, and now coverage + protection. Module 8's final project requires this CI as part of the delivery.


Summary

  • Add pytest-cov and --cov=src --cov-report=term-missing --cov-report=html --cov-report=xml to the pytest step
  • Publish htmlcov/ as an artifact to review the report
  • Use actions like py-cov-action to comment coverage on PRs
  • Configure branch protection to require the checks to pass before the merge
  • Use --cov-fail-under or fail_under in coverage to block PRs with low coverage
  • Claude Code can generate the complete workflow if you give it detailed context

Next capsule: Project — A complete working CI pipeline.


Additional Resources

  1. pytest-cov documentation - pytest-cov options
  2. coverage.py configuration - Coverage configuration
  3. GitHub Branch protection - Branch protection documentation
  4. py-cov-action - The action for commenting coverage on PRs
  5. actions/upload-artifact - Uploading artifacts
  6. GitHub Status checks - Status checks and branch protection

Module 7, Capsule 05 — Testing with Claude Code Guide