Inline Comments via the GitHub API

Overview

This is the capsule that transforms Claude Code from "a tool that comments in general" into a real reviewer: an agent that leaves comments on the exact line where it detects a problem. Inline comments are the difference between feedback that's acted on immediately and feedback that's ignored.

You'll learn the technical difference between general comments and inline comments, how to create the latter via the GitHub API, which commit SHA to use, how to handle common errors (lines out of range, files not in the diff), and how to combine inline comments with a general summary in a single review.

By the end, you'll have a bot that produces human-review-quality inline comments: specific, on the right line, with actionable suggestions.

The Difference: General Comment vs Inline Comment

GENERAL COMMENT (at the end of the PR):
┌─────────────────────────────────────────────┐
│ 🤖 Claude Code Analysis                    │
│                                             │
│ I found 3 problems:                        │
│ - Missing validation in payment.py:42      │
│ - SQL injection in auth.py:87              │
│ - Inconsistent naming in orders.py:120     │
└─────────────────────────────────────────────┘
PROBLEM: the developer has to JUMP between
the comment and each file. Low visibility.

VERSUS

INLINE COMMENTS (on each line):
payment.py
┌──────────────────────────────────────┐
│ Line 42:                              │
│   if amount < 0:                      │
│       process()                       │
│                                       │
│   💬 Claude Code:                     │
│   Missing validation that amount is   │
│   numeric before comparing. If it gets│
│   a string, this compares strings     │
│   (non-deterministic result).         │
└──────────────────────────────────────┘

Visibility and actionability are superior with inline comments. The developer sees the problem in the context of the code, not three clicks away.

The Review Comments API

GitHub has two distinct APIs for "comments" on PRs:

They sound similar but they're different. Capsule 04 of module 1 used the first (issue comments for a general summary). This capsule uses the second.

Creating an inline comment: minimal structure

import requests
import os

token = os.environ["GITHUB_TOKEN"]
repo = os.environ["GITHUB_REPOSITORY"]
pr_number = os.environ["PR_NUMBER"]
commit_sha = os.environ["PR_HEAD_SHA"]

url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/comments"
headers = {
    "Authorization": f"Bearer {token}",
    "Accept": "application/vnd.github+json",
    "X-GitHub-Api-Version": "2022-11-28",
}

payload = {
    "body": "Missing validation that `amount` is numeric.",
    "commit_id": commit_sha,
    "path": "payment.py",
    "line": 42,
    "side": "RIGHT",
}

response = requests.post(url, headers=headers, json=payload)

if response.status_code == 201:
    print(f"Inline comment created: {response.json()['html_url']}")
else:
    print(f"Error: {response.status_code} {response.text}")

Key fields:

Passing the commit SHA to the workflow

In the YAML, expose the correct SHA:

- name: Run review
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    GITHUB_REPOSITORY: ${{ github.repository }}
    PR_NUMBER: ${{ github.event.pull_request.number }}
    PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
  run: python .github/scripts/review_pr.py

pull_request.head.sha is the SHA of the PR's last commit — the one you want to comment on.

The Recommended Pattern: Create a Complete Review

Instead of creating comments one by one, you can create a review with multiple comments in a single call. It's more efficient and atomic.

Endpoint

POST /repos/{repo}/pulls/{number}/reviews

Payload structure

review_payload = {
    "commit_id": commit_sha,
    "body": "## Review summary\n\nI found 3 issues. Details inline.",
    "event": "COMMENT",  # or "REQUEST_CHANGES" or "APPROVE"
    "comments": [
        {
            "path": "payment.py",
            "line": 42,
            "side": "RIGHT",
            "body": "Missing validation that `amount` is numeric.",
        },
        {
            "path": "auth.py",
            "line": 87,
            "side": "RIGHT",
            "body": "SQL injection: use a parameterized query.",
        },
        {
            "path": "orders.py",
            "line": 120,
            "side": "RIGHT",
            "body": "The naming `createOrder` doesn't follow snake_case.",
        },
    ],
}

url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/reviews"
response = requests.post(url, headers=headers, json=review_payload)

The event field

For suggest-only mode (capsule 04), you always use COMMENT. Approving or rejecting automatically requires much more confidence in the model, and capsule 04 develops when it's appropriate.

Multi-Line Comments

Sometimes the problem involves several lines (not just one). GitHub supports multi-line comments:

{
    "path": "payment.py",
    "start_line": 40,    # first line of the range
    "line": 45,          # last line of the range
    "start_side": "RIGHT",
    "side": "RIGHT",
    "body": "This whole function has problems: ...",
}

Use cases:

• A whole function with structural problems
• A duplicated code block
• A sequence of lines with faulty logic

Generating the Inline Comments from Claude

Asking Claude Code to return the comments in a structured JSON format simplifies the integration:

"""Review with structured inline comments."""
import json
import re
import os
from anthropic import Anthropic

client = Anthropic()

# Read the filtered diff (capsule 02)
with open("filtered_diff.txt") as f:
    diff_text = f.read()

prompt = f"""Do a code review of this pull request diff.
Return a JSON with this exact structure:

{{
  "summary": "Review summary (2-3 paragraphs in markdown)",
  "comments": [
    {{
      "path": "path/to/file",
      "line": 42,
      "severity": "critical|warning|suggestion",
      "body": "Comment text in markdown"
    }}
  ]
}}

Rules:
- Only include actionable comments (not "looks good")
- `path` must be exactly as it appears in the diff
- `line` must be a line number present in the diff
- Use severity "critical" only for real bugs or vulnerabilities
- Return ONLY the JSON, with no additional text

Diff:

{diff_text}

"""

response = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=4000,
    messages=[{"role": "user", "content": prompt}],
)

# Parse the JSON
text = response.content[0].text.strip()
text = re.sub(r"^```json\n?", "", text)
text = re.sub(r"\n?```$", "", text)

try:
    review_data = json.loads(text)
except json.JSONDecodeError as e:
    print(f"ERROR: response is not valid JSON: {e}")
    print(f"Text received:\n{text}")
    sys.exit(1)

print(f"Review generated: {len(review_data['comments'])} comments")

Now review_data["comments"] is directly the format the API expects.

Validate Before Publishing

Critical: validate that the comments reference lines that exist in the diff. If Claude suggests commenting on line 200 of a file whose diff only touches lines 40-60, the API responds 422 (Unprocessable Entity).

def validate_comment(comment, files_in_diff):
    """Validate that a comment is publishable."""
    path = comment["path"]
    line = comment["line"]
    
    # Is the file in the diff?
    file_data = next((f for f in files_in_diff if f["filename"] == path), None)
    if not file_data:
        return False, f"File {path} is not in the diff"
    
    # Is the line in any section of the patch?
    patch = file_data.get("patch", "")
    if not patch:
        return False, f"File {path} has no patch (binary or very large)"
    
    # Extract ranges from the patch (format: @@ -X,Y +A,B @@)
    line_in_patch = False
    for hunk in re.finditer(r"@@\s+-\d+,\d+\s+\+(\d+),(\d+)\s+@@", patch):
        start = int(hunk.group(1))
        count = int(hunk.group(2))
        if start <= line < start + count:
            line_in_patch = True
            break
    
    if not line_in_patch:
        return False, f"Line {line} of {path} is not in the diff"
    
    return True, "OK"

# Filter valid comments
valid_comments = []
for c in review_data["comments"]:
    is_valid, reason = validate_comment(c, files_in_diff)
    if is_valid:
        valid_comments.append({
            "path": c["path"],
            "line": c["line"],
            "side": "RIGHT",
            "body": format_severity(c["severity"]) + c["body"],
        })
    else:
        print(f"SKIP: {reason}")

Result: invalid comments are filtered before the API, avoiding 422 errors.

Formatting with Severity

So the comments communicate importance visually and fast, prepend an emoji or tag based on severity:

def format_severity(severity: str) -> str:
    """Visual prefix by severity."""
    return {
        "critical": "🚨 **CRITICAL** — ",
        "warning": "⚠️ **WARNING** — ",
        "suggestion": "💡 **Suggestion** — ",
    }.get(severity, "")

# Usage:
body = format_severity("critical") + "SQL injection: use a parameterized query"
# Result: "🚨 **CRITICAL** — SQL injection: use a parameterized query"

Common Pitfalls

Error 1: Line outside the diff

Symptom: API responds 422 "line must be part of the diff".

Why it happens: The model suggests commenting on a line that wasn't modified in the PR (even though it exists in the file).

How to fix it: Validate against the patch before publishing (the "Validate Before Publishing" section).

Error 2: Path with ./ at the start

Symptom: API responds 422 "path must be relative".

Why it happens: Some models return paths like ./src/payment.py. The API expects src/payment.py.

How to fix it: Clean the paths before sending:

path = comment["path"].lstrip("./")

Error 3: Incorrect commit SHA

Symptom: API responds 422 "commit_id is not the head of the pull request".

Why it happens: You passed github.sha (the workflow run's SHA) instead of github.event.pull_request.head.sha.

How to fix it: On PRs, always pull_request.head.sha. github.sha points to an artificial merge commit that GitHub creates for the workflows, not to the PR's HEAD.

Error 4: Insufficient permissions

Symptom: API responds 403 "Resource not accessible by integration".

Why it happens: The YAML doesn't declare permissions: pull-requests: write.

How to fix it: Add to the workflow:

permissions:
  pull-requests: write
  contents: read

Error 5: Creating comments one by one instead of a review

Symptom: Several comments are published gradually; if the script fails halfway, a partial review is left.

Why it happens: Calling the comments endpoint in a loop instead of the reviews endpoint.

How to fix it: Use POST /pulls/{number}/reviews with all the comments in a single call. Atomicity: either everything is published or nothing.

Diagnosis

Exercises

Exercise 1: Create an inline comment manually (Easy)

Using curl or Postman, create an inline comment on a test PR directly with the API. Verify it shows up on the correct line.

curl -X POST \
  -H "Authorization: Bearer $GH_TOKEN" \
  -H "Accept: application/vnd.github+json" \
  https://api.github.com/repos/OWNER/REPO/pulls/PR_NUMBER/comments \
  -d '{
    "body": "Test comment",
    "commit_id": "COMMIT_SHA",
    "path": "PR_FILE",
    "line": LINE_NUMBER,
    "side": "RIGHT"
  }'

Exercise 2: Create a review with multiple comments (Medium)

Modify the capsule 02 script so it generates a review with 2-3 inline comments on real lines of the diff.

# (assuming you already have review_data from the model)

review_payload = {
    "commit_id": commit_sha,
    "body": review_data["summary"],
    "event": "COMMENT",
    "comments": valid_comments,  # from the validation step
}

url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/reviews"
r = requests.post(url, headers=headers, json=review_payload)
print(f"Review: {r.status_code} — {r.json().get('html_url', r.text)}")

Exercise 3: Robust validation (Hard)

Implement the validate_comment function that:

1. Verifies the file is in the diff
2. Verifies the line is in some hunk of the patch
3. Cleans paths with ./ or trailing slashes
4. Handles files with no patch (binaries)

Use the implementation from the "Validate Before Publishing" section as a base and extend it.

Summary

• Inline comments > general comments — visibility and actionability
• Two distinct APIs: issue comments (general) vs review comments (inline)
• Creating a review with multiple comments in one call is more atomic than loops
• Validating the line in the diff before publishing avoids 422 errors
• github.event.pull_request.head.sha is the correct SHA for PRs
• permissions: pull-requests: write is a prerequisite
• Severity with an emoji prefix communicates importance visually and fast

Next capsule: 04 — Review summary and suggest-only mode. You have working inline comments. Now you learn to complement them with a review summary and to decide when the bot blocks merges vs when it only suggests.

Additional Resources

1. GitHub PR Reviews API — Reviews endpoint
2. GitHub PR Review Comments API — Individual comments endpoint
3. Octokit — Official JS/TS client
4. PyGithub — Python client
5. GitHub Actions: GITHUB_TOKEN permissions — Default permissions
6. Anthropic structured output — Force JSON output from the model