09 / 13
Notification webhooks
Three events, outbox retry semantics and the delivery log
Events
backup_failed: a backup job failed.backup_expired: a database's protection went stale — both "newest success older than the freshness threshold" and "never succeeded" (paused databases stop generating new expiry events).verification_failed: restore verification failed (the job itself is still succeeded).
Each event is delivered to every matching webhook as one JSON POST each (a
single outbox pass may POST to several targets, and a failed round re-visits
targets that already succeeded in the previous round), with
X-Supabackup-Event and X-Supabackup-Event-ID headers (a stable dedup key).
Delivery semantics (outbox)
- Transactional in the normal path: the notification commits with the terminal state; compensation paths (fallback enqueue plus the scheduler reconciliation sweep) provide the durable no-lost-alert guarantee.
- At-least-once: retries back off 30/60/120/240s, up to 5 attempts; exhaustion
marks the delivery
dead(visible in the UI, never silently dropped). - Single instance: leases and dedup assume one running instance (enforced by a process lock).
Testing and diagnosing
- "Send test" performs a real delivery immediately (outside the outbox) and reports delivered/failed with an error category.
- The delivery log shows state,
attempts(cumulative FAILED rounds — a success does not increment it, and it is not an HTTP request count) and last error per notification (errors are classified and redacted; URLs are never stored there). - Webhook URLs may point at LAN/loopback receivers (a supported scenario) but
link-local and cloud-metadata endpoints are refused — including IPv6
fd00:ec2::254and NAT64 encodings.
Last updated