Module 4: Deployment Automation

Changelog Generation Based on Diffs

Overview

The changelog is the document that communicates to the team and stakeholders what changed between versions. Done by hand, it's tedious and prone to inconsistencies between releases. Done wrong (based only on commit messages), it's vague and omits important context. This capsule teaches you to generate professional and precise changelogs automatically, analyzing the real diffs of the code — not just the commit messages.

By the end, you'll have a workflow that triggers on each release tag, analyzes the real changes, categorizes by type (features, fixes, breaking changes), and produces a structured changelog ready to publish — with automatic detection of breaking changes that are normally omitted.

Why Commit Messages Aren't Enough

Imagine a commit with the message "fix typo in payment.py". Sounds trivial. But the "typo" was changing if amount < threshold to if amount > threshold — it inverts the logic. It's a critical functional change, but a changelog based on commit messages would say:

## Bug fixes
- fix typo in payment.py

Useless. The team doesn't know what happened. Stakeholders don't understand the risk.

Diff-based generation reads the real change:

## Bug fixes
- **payment.py**: Fixed amount validation logic. The condition
  `if amount < threshold` was changed to `if amount > threshold`. This change
  inverts the previous behavior — review regression tests before
  the deploy.

The difference is actionable information vs noise.

Anatomy of a Professional Changelog

# Changelog

## [2.5.0] - 2026-05-03

### ⚠️ Breaking Changes
- **API**: The `/api/users/<id>` endpoint now requires authentication
  (it used to be public). Clients must update.
- **Database**: The `users.legacy_email` column was removed. Make sure
  you've migrated to `users.email` before the deploy.

### 🚀 Features
- **Payments**: Support for Stripe Connect, allowing payments to multiple accounts
- **Notifications**: New SMS channel in addition to email

### 🐛 Bug Fixes
- **payment.py**: Fixed amount validation logic that rejected
  valid payments at high amounts
- **OrderService**: Correct handling of a race condition under concurrency

### 🔧 Internal / Refactoring
- Migration to SQLAlchemy 2.0 complete
- Integration tests separated from unit tests

### 📚 Documentation
- README updated with Docker instructions
- New architecture doc

### 🔒 Security
- Update of dependencies with CVEs (requests, urllib3)

The 6 standard categories (based on Keep a Changelog + extensions):

Category	When
⚠️ Breaking Changes	Changes that break compatibility
🚀 Features	New functionality
🐛 Bug Fixes	Bug fixes
🔧 Internal	Refactoring, cleanup, technical improvements with no functional change
📚 Documentation	Docs only
🔒 Security	Security patches

The Workflow: Trigger by Tag

# .github/workflows/release-changelog.yml
name: Generate Release Changelog

on:
  push:
    tags:
      - 'v*'  # triggers on tags like v2.5.0, v3.0.0-beta

permissions:
  contents: write
  pull-requests: write

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # full history to compare tags
      
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'
      
      - run: pip install anthropic requests
      
      - name: Generate changelog
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITHUB_REPOSITORY: ${{ github.repository }}
          NEW_TAG: ${{ github.ref_name }}
        run: python scripts/generate_changelog.py
      
      - name: Upload changelog as artifact
        uses: actions/upload-artifact@v4
        with:
          name: changelog
          path: CHANGELOG_${{ github.ref_name }}.md

The Script: Generation with Real Diffs

"""scripts/generate_changelog.py

Generates a changelog by analyzing real diffs between the previous and new tag.
"""
import json
import os
import re
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from anthropic import Anthropic
import requests


@dataclass
class CommitInfo:
    sha: str
    message: str
    author: str
    pr_number: int | None
    files_changed: list[str]
    diff_summary: str  # truncated


def get_previous_tag(new_tag: str) -> str:
    """Find the tag prior to the new one."""
    result = subprocess.run(
        ["git", "tag", "--sort=-creatordate"],
        capture_output=True, text=True, check=True,
    )
    tags = result.stdout.strip().split("\n")
    
    # The new tag should be first or near the top
    if new_tag in tags:
        idx = tags.index(new_tag)
        if idx + 1 < len(tags):
            return tags[idx + 1]
    
    # Fallback: first commit
    first_commit = subprocess.run(
        ["git", "rev-list", "--max-parents=0", "HEAD"],
        capture_output=True, text=True, check=True,
    )
    return first_commit.stdout.strip().split("\n")[0]


def get_commits_between(old_ref: str, new_ref: str) -> list[CommitInfo]:
    """List commits between two refs with useful info."""
    result = subprocess.run(
        ["git", "log", f"{old_ref}..{new_ref}",
         "--format=%H||%s||%an"],
        capture_output=True, text=True, check=True,
    )
    
    commits = []
    for line in result.stdout.strip().split("\n"):
        if not line:
            continue
        parts = line.split("||")
        if len(parts) < 3:
            continue
        sha, message, author = parts[0], parts[1], parts[2]
        
        # Extract the PR number from the message (typical format: "feat: X (#123)")
        pr_match = re.search(r"\(#(\d+)\)$", message)
        pr_number = int(pr_match.group(1)) if pr_match else None
        
        # Modified files
        files_result = subprocess.run(
            ["git", "show", "--name-only", "--format=", sha],
            capture_output=True, text=True, check=True,
        )
        files = [f for f in files_result.stdout.strip().split("\n") if f]
        
        # Diff summary (truncated so it doesn't overflow)
        diff_result = subprocess.run(
            ["git", "show", "--stat", "--format=", sha],
            capture_output=True, text=True, check=True,
        )
        
        commits.append(CommitInfo(
            sha=sha[:8],
            message=message,
            author=author,
            pr_number=pr_number,
            files_changed=files[:10],  # max 10 files in the summary
            diff_summary=diff_result.stdout[:500],  # max 500 chars
        ))
    
    return commits


def fetch_pr_descriptions(pr_numbers: list[int], repo: str, token: str) -> dict[int, str]:
    """Fetch PR descriptions for additional context."""
    descriptions = {}
    headers = {
        "Authorization": f"Bearer {token}",
        "Accept": "application/vnd.github+json",
    }
    
    for pr in pr_numbers:
        try:
            url = f"https://api.github.com/repos/{repo}/pulls/{pr}"
            r = requests.get(url, headers=headers, timeout=10)
            if r.status_code == 200:
                data = r.json()
                descriptions[pr] = data.get("body", "") or ""
        except Exception:
            pass  # tolerate individual failures
    
    return descriptions


def categorize_with_claude(
    commits: list[CommitInfo],
    pr_descriptions: dict[int, str],
    new_tag: str,
) -> str:
    """Call Claude to generate the structured changelog."""
    
    # Build the context to pass to the model
    commits_summary = []
    for c in commits:
        commits_summary.append(f"""
## Commit {c.sha} (PR #{c.pr_number or 'N/A'})
**Author:** {c.author}
**Message:** {c.message}
**Files:** {', '.join(c.files_changed[:5])}
**Stats:**
{c.diff_summary}
{f'**PR description:** {pr_descriptions.get(c.pr_number, '')[:300]}' if c.pr_number else ''}
""")
    
    commits_text = "\n---\n".join(commits_summary)
    
    prompt = f"""Generate a professional changelog for version {new_tag}.

You have this information from commits and PRs:

{commits_text}

INSTRUCTIONS:
1. Categorize each commit into: Breaking Changes, Features, Bug Fixes, Internal, Documentation, Security
2. **Breaking changes are critical** — if you detect changes that break compatibility (removing APIs, changing signatures, modifying schemas), mark it even if the commit message doesn't say so
3. Write **descriptive** entries (not just the commit message). Explain what changed and why it matters
4. If a commit is trivial (a typo in a doc, formatting), include it in Internal
5. Group related commits into one entry when applicable
6. Output: ONLY the changelog markdown, with this structure:

```markdown
## [{new_tag}] - YYYY-MM-DD

### ⚠️ Breaking Changes
[items or "None" if there are none]

### 🚀 Features
[items or omit the section if there are none]

### 🐛 Bug Fixes
[items]

### 🔧 Internal
[items]

### 📚 Documentation
[items or omit]

### 🔒 Security
[items or omit]

Return ONLY the changelog markdown, with no preamble. """

client = Anthropic()
response = client.messages.create(
    model="claude-sonnet-5",  # Sonnet for more quality in categorization
    max_tokens=4000,
    messages=[{"role": "user", "content": prompt}],
)

text = response.content[0].text.strip()
# Clean code fences if there are any
if text.startswith("```"):
    text = "\n".join(text.split("\n")[1:-1])

return text

def main() -> int: new_tag = os.environ["NEW_TAG"] repo = os.environ["GITHUB_REPOSITORY"] token = os.environ["GITHUB_TOKEN"]

print(f"Generating changelog for {new_tag}...")

# 1. Find the previous tag
prev_tag = get_previous_tag(new_tag)
print(f"Previous tag: {prev_tag}")

# 2. List commits between tags
commits = get_commits_between(prev_tag, new_tag)
print(f"Commits to include: {len(commits)}")

if not commits:
    print("No new commits. Skipping.")
    return 0

# 3. Fetch PR descriptions for context
pr_numbers = [c.pr_number for c in commits if c.pr_number]
pr_descriptions = fetch_pr_descriptions(pr_numbers, repo, token)

# 4. Generate the changelog with Claude
changelog = categorize_with_claude(commits, pr_descriptions, new_tag)

# 5. Save
output_file = Path(f"CHANGELOG_{new_tag}.md")
output_file.write_text(changelog)
print(f"Changelog generated: {output_file}")
print(f"\n--- Preview ---\n{changelog[:1000]}\n...")

return 0

if name == "main": sys.exit(main())


---

## Breaking Change Detection

The most valuable part of the automatic changelog is **detecting breaking changes** that the commit message doesn't mention explicitly. Patterns that Claude Code can identify:

### Breaking change patterns

| Pattern | Example |
|--------|---------|
| **Removal of public functions/endpoints** | `def get_user(id)` is removed |
| **Signature change** | `def create_user(name, email)` → `def create_user(name, email, role)` (a new required parameter) |
| **Change in the structure of responses** | An endpoint that returned `{user: {...}}` now returns `{...}` directly |
| **Change in column/schema names** | `users.legacy_email` is renamed or removed |
| **Change in configuration values** | A required env variable changes its name |
| **Change in output format** | A CLI that printed JSON now prints text |
| **Change in default behavior** | An optional parameter changes its default value |

### Prompt emphasizing detection

A better version of the prompt for detection:

```python
prompt += """
BREAKING CHANGE DETECTION (critical):
Even if the commit message doesn't say so, mark it as a breaking change if:
- A public identifier (function, class, method, endpoint) was removed or renamed
- The signature of a public function changed (new required parameter, changed types)
- An API response structure changed (fields removed, renamed, type changed)
- DB schemas changed (columns removed, renamed)
- Required environment variables changed name
- Default behaviors changed

If in DOUBT, INCLUDE IT as a breaking change. Better to over-flag than under-flag in this case.
"""

Integrating the Changelog into the Repo

After generating, you want it to live in the repo (not just as an artifact):

- name: Update CHANGELOG.md in repo
  run: |
    # Add the new changelog to the top of CHANGELOG.md
    if [ -f CHANGELOG.md ]; then
      mv CHANGELOG.md CHANGELOG.md.bak
      cat CHANGELOG_${{ github.ref_name }}.md > CHANGELOG.md
      echo "" >> CHANGELOG.md
      cat CHANGELOG.md.bak >> CHANGELOG.md
      rm CHANGELOG.md.bak
    else
      echo "# Changelog" > CHANGELOG.md
      echo "" >> CHANGELOG.md
      cat CHANGELOG_${{ github.ref_name }}.md >> CHANGELOG.md
    fi
    
    git config user.name "github-actions[bot]"
    git config user.email "github-actions[bot]@users.noreply.github.com"
    git add CHANGELOG.md
    git commit -m "chore: update changelog for ${{ github.ref_name }}"
    git push origin HEAD:main

Or the alternative: create a PR with the changelog for human review before merge.

Publish as a GitHub Release

- name: Create GitHub Release
  uses: actions/create-release@v1
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    tag_name: ${{ github.ref_name }}
    release_name: Release ${{ github.ref_name }}
    body_path: CHANGELOG_${{ github.ref_name }}.md
    draft: false
    prerelease: ${{ contains(github.ref_name, 'beta') || contains(github.ref_name, 'alpha') }}

Now each tag automatically generates:

A changelog in the repo's CHANGELOG.md file
A GitHub Release with that same changelog as the description

Common Pitfalls

Error 1: Generating a changelog but not detecting breaking changes

Symptom: The changelog has Features and Bug Fixes but is missing the Breaking Changes section — and there are some.

Why it happens: A generic prompt that doesn't emphasize detection.

How to fix it: Explicit emphasis in the prompt on breaking change patterns. "Better to over-flag than under-flag" is the guide.

Error 2: Passing all the complete diffs to the model

Symptom: High cost, can overflow the context window on large releases.

Why it happens: Not truncating the diff per commit.

How to fix it: Use git show --stat (which gives a summary, not the complete diff). Pass the model the change summary + commit message + PR description, not the complete diff.

Error 3: Trivial commits are included as Features

Symptom: The changelog includes "Updated dependencies" or "Fixed typo" in Features.

Why it happens: The model doesn't distinguish between significant and trivial changes.

How to fix it: Explicitly ask it that trivial commits (typos, routine dependency updates, formatting) go to "Internal".

Error 4: PR descriptions aren't leveraged

Symptom: The changelog reproduces the literal commit messages without context.

Why it happens: You're not fetching PR descriptions.

How to fix it: As the script shows, fetch the PR descriptions via the API and pass them to the model. That's where the human context is that the commit message doesn't have.

Error 5: Not versioning tags with consistent semver

Symptom: The script fails because "the previous tag doesn't exist".

Why it happens: Inconsistent tags (release-1, 2.5.0, v3-beta).

How to fix it: Adopt a semver convention with the v prefix: v1.0.0, v1.1.0, v2.0.0-beta.1. Adjust the tags: ['v*'] filter in the workflow.

Diagnosis

Question 1: Does your current changelog include clearly identified Breaking Changes?

If not, stakeholders don't know what requires attention. The Breaking Changes section is the most important.

Question 2: Do you generate the changelog based on commit messages or on real diffs?

Only commit messages = limited. Real diffs (with PR descriptions) = precise.

Question 3: Is the changelog versioned in the repo or only published in releases?

Versioning it in CHANGELOG.md gives accessible history and is an industry standard (Keep a Changelog).

Question 4: Does your workflow trigger on tags or on merge to main?

On tags (semver) is the natural moment — it defines a "release". On every merge would be overkill.

Question 5: How do you handle pre-releases (beta, alpha, rc)?

If the tag has -beta or similar, set prerelease: true in the GitHub Release. It changes visibility and semantically.

Exercises

Exercise 1: Basic workflow (Easy)

Configure a workflow that triggers on v* tags. It does actions/checkout with fetch-depth: 0 and prints git log --oneline $(git describe --abbrev=0 HEAD~1)..HEAD to see the included commits.

Exercise 2: Generate the changelog with Claude (Medium)

Implement the complete pipeline:

Extract commits between the previous and new tag
Pass them to the model with a structured prompt
Save as an artifact
Verify the output with a test release

See checklist

The workflow triggers only on tags
It correctly finds the previous tag
It lists commits between the two tags
It generates a changelog with the 6 categories
It detects breaking changes (test it with a commit that breaks compatibility)

Exercise 3: Auto-PR with the changelog for review (Hard)

Modify the workflow so it:

Instead of pushing CHANGELOG.md directly to main
Creates a new branch with the change
Opens a PR with the changelog for human review
Assigns the PR to the manager or tech lead

Advantage: a human reviews the changelog before it's published as a release.

Summary

Real diffs > commit messages for precise changelogs
6 standard categories: Breaking, Features, Bug Fixes, Internal, Documentation, Security
Breaking change detection is the most valuable part of the automatic change
Trigger by tag (v*) is the natural moment — it defines a release
PR descriptions give human context that commit messages don't have
Truncating diffs avoids high cost and context overflow
Auto-PR for review is a more conservative option than a direct push

Next capsule: 03 — Deployment readiness validation. You already have the changelog. Before the deploy, you have to validate that the system is ready: tests pass, migrations prepared, env vars configured. You'll learn to automate that checklist with Claude Code.

Additional Resources

Keep a Changelog — The format standard
Semantic Versioning — Versioning convention
Conventional Commits — A commit convention that helps categorize
GitHub Releases API — Endpoint for creating releases
git log advanced — Useful filters and formats
Anthropic batch API — For huge repos with many commits

← PreviousModule 4: Deployment Automation Next →Deployment Readiness Validation