[Zach Dunn (zachdunn)]

Dead-lettered GitHub webhook events were acked and forgotten. The log kept a compact event, but not the GitHub delivery id, and nothing told the operator. This keeps the full event, sends one email per batch, and adds a script that puts those payloads back on the queue. It covers the burst Zach Dunn (@zachdunn) recovered in #1338.

What it does / what it is not

• When uploads-github-webhook-dlq receives a batch, each event is logged as JSON, including X-GitHub-Delivery, and then acked.
• One batch sends one email to OPERATOR_ALERT_EMAIL on uploads-auth (the same address as the storage FYI). Unset, the mail is skipped. The log line is still written.
• apps/api/scripts/replay-github-webhook-dlq.mjs reads that JSON and pushes it onto uploads-github-webhook.
• This does not redeliver from GitHub, and it does not retry the dead-letter queue itself. A replay is a new message on the main queue.

How to try it

After this deploys, save the JSON from the operator email (or the event field from the log line) and run:

node --env-file=.env apps/api/scripts/replay-github-webhook-dlq.mjs --dry-run events.json
node --env-file=.env apps/api/scripts/replay-github-webhook-dlq.mjs events.json

CLOUDFLARE_API_TOKEN needs Queues Write. --dry-run prints the delivery ids and does not send.

Technical notes

The delivery id rides on the compact WebhookEvent. Processing ignores it. The auth worker sends the mail from POST /internal/webhook-dlq/notify, on the same service-binding path as the storage FYI. The script uses the Queues HTTP API because wrangler has no producer-send command.

Test plan

• pnpm --filter @uploads/api exec vitest run src/github-webhook-queue.test.ts src/routes/github-webhook-route.test.ts scripts/replay-github-webhook-dlq.test.ts src/github-webhook.test.ts
• pnpm --filter @uploads/auth exec vitest run src/internal-routes.test.ts -t webhook-dlq
• pnpm --filter @uploads/email exec vitest run src/webhook-dlq.test.ts
• Confirm a dead-letter batch in production writes the structured log and one email to OPERATOR_ALERT_EMAIL

Closes #1338

  

[changeset-bot]

⚠️ No Changeset found

Latest commit: f0d9230

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 2

• coderabbit:review
• review

🚫 Excluded labels (none allowed): 1

• wip

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: ac60f97d-aef9-426f-85c7-a2879eb1b619

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[uploads-sh]

No uploads on this PR yet

This PR changes UI files (packages/email/src/index.ts, packages/email/src/webhook-dlq.test.ts and packages/email/src/webhook-dlq.ts). Screenshots help reviewers see what changed.

• From the CLI: uploads put before.png after.png --pr 1340
• From an agent: ask it to attach screenshots with uploads.sh
• By hand: drag images into the PR description and they're imported automatically

Tips for agents

Label each shot so reviewers know what they're seeing. Labels apply to every file in a call, so run one put per shot:

uploads put after.png --pr 1340 --meta path=/settings --state after --caption "Saved settings toast"
uploads put chat.png --pr 1340 --app ios --screen "Chat" --state after

No CLI? With the Uploads MCP, call sign_upload_url for files on disk (or put) with repo, pr: 1340 and the same screen, state and caption fields.

This comment updates with each upload, so you don't need to post the images yourself. A shot or two in the PR description is still welcome. Check what's attached with uploads list --pr 1340. More in the github-screenshots skill or the agent docs.

_{Workspace admins can turn this comment off in Settings → GitHub · docs}