AI code review bot for pull requests

A GitHub Actions workflow that reviews diffs with an LLM and posts capped, line-anchored comments.

Run it as a prompt

Paste this into any AI agent, or fetch it: curl -s https://uplift.page/api/v1/prompts/ai-pr-review-bot/raw

prompt.md

# AI code review bot for pull requests

Post at most 10 inline comments per PR within 3 minutes of a push, keep false positives under 25% on a 30-PR replay set, and reach 60% or more of comments accepted or resolved over 4 weeks.

## Recommended stack

- **GitHub Actions** - Runs the reviewer on pull_request with pull-requests: write. No extra infrastructure, and model keys stay in repository secrets.
- **AI SDK 7** - Model call with typed output. generateText with output: Output.object({ schema }) returns validated findings from any provider.
- **@octokit/rest 22** - Fetch the diff and create one review with inline comments. The create-review endpoint takes a comments array with path, line, and side in one call.
- **zod 4** - Finding schema. Forces path, line, severity, category, message, and optional suggestion on every finding.

## Build steps

1. Run lint, typecheck, and tests first; start the review job only if they pass, so the model does not repeat what tools already catch.
2. Fetch the PR diff; skip lockfiles, generated code, vendored code, and files over 1,000 changed lines. Build one request per file with 30 lines of context and the repo's contributor or agent conventions file.
3. Ask for specific categories only (correctness, security, error handling, concurrency, data loss) and allow an empty list when nothing is worth flagging.
4. Validate every finding: the path is in the diff, the line is inside a changed hunk on the new side, it is not a duplicate of an existing bot comment. Keep the top 10 by severity.
5. Post one review with event COMMENT, a 1-2 line summary body (required for COMMENT), commit_id set to the PR head SHA, and the comments array (path, line, side: RIGHT); include a ```suggestion block only for fixes of 5 lines or fewer.
6. Use the pull_request trigger, never pull_request_target with a checkout of the PR head; set permissions: { contents: read, pull-requests: write } and gate the job with if: github.event.pull_request.draft == false && github.event.pull_request.head.repo.full_name == github.repository (fork PRs get no secrets and a read-only token).
7. Build a replay set of 30 merged PRs with known bugs, measure precision and recall, and tune the prompt; review thumbs-up/down reactions weekly.

## Watch out for

- pull_request_target plus checking out PR code exposes secrets to untrusted contributors.
- Wrong line mapping returns 422 errors; compute lines from hunk headers on the new side and use line + side, not position.
- Style nits and speculative warnings make people ignore the bot. Restrict categories and cap the count.
- Re-reviewing the whole PR on every push duplicates comments. Review only commits since the last reviewed SHA.

## Definition of done

- A test PR with a planted null dereference gets an inline comment on that line within 3 minutes.
- A docs-only PR gets no comments.
- A PR from a fork runs without access to the model key (job is skipped, visible in logs).
- The replay script prints precision of 75% or higher.

The full prompt

Post at most 10 inline comments per PR within 3 minutes of a push, keep false positives under 25% on a 30-PR replay set, and reach 60% or more of comments accepted or resolved over 4 weeks.

Recommended stack

GitHub Actions - Runs the reviewer on pull_request with pull-requests: write. No extra infrastructure, and model keys stay in repository secrets.
AI SDK 7 - Model call with typed output. generateText with output: Output.object({ schema }) returns validated findings from any provider.
@octokit/rest 22 - Fetch the diff and create one review with inline comments. The create-review endpoint takes a comments array with path, line, and side in one call.
zod 4 - Finding schema. Forces path, line, severity, category, message, and optional suggestion on every finding.

Build steps

Run lint, typecheck, and tests first; start the review job only if they pass, so the model does not repeat what tools already catch.
Fetch the PR diff; skip lockfiles, generated code, vendored code, and files over 1,000 changed lines. Build one request per file with 30 lines of context and the repo's contributor or agent conventions file.
Ask for specific categories only (correctness, security, error handling, concurrency, data loss) and allow an empty list when nothing is worth flagging.
Validate every finding: the path is in the diff, the line is inside a changed hunk on the new side, it is not a duplicate of an existing bot comment. Keep the top 10 by severity.
Post one review with event COMMENT, a 1-2 line summary body (required for COMMENT), commit_id set to the PR head SHA, and the comments array (path, line, side: RIGHT); include a ```suggestion block only for fixes of 5 lines or fewer.
Use the pull_request trigger, never pull_request_target with a checkout of the PR head; set permissions: { contents: read, pull-requests: write } and gate the job with if: github.event.pull_request.draft == false && github.event.pull_request.head.repo.full_name == github.repository (fork PRs get no secrets and a read-only token).
Build a replay set of 30 merged PRs with known bugs, measure precision and recall, and tune the prompt; review thumbs-up/down reactions weekly.

Watch out for

pull_request_target plus checking out PR code exposes secrets to untrusted contributors.
Wrong line mapping returns 422 errors; compute lines from hunk headers on the new side and use line + side, not position.
Style nits and speculative warnings make people ignore the bot. Restrict categories and cap the count.
Re-reviewing the whole PR on every push duplicates comments. Review only commits since the last reviewed SHA.

Definition of done

A test PR with a planted null dereference gets an inline comment on that line within 3 minutes.
A docs-only PR gets no comments.
A PR from a fork runs without access to the model key (job is skipped, visible in logs).
The replay script prints precision of 75% or higher.

Served from the uplift.page library and refreshed within 5 minutes of every update.