Getting StartedWebhooks

Delivery and Retries

Understand webhook delivery behavior, retry logic, and duplicate handling

Overview

Webhook delivery in Fynapse is designed to be reliable and fault-tolerant. If a delivery attempt fails, the system automatically retries according to a defined policy.

Webhook delivery follows an at-least-once model, meaning your endpoint may receive the same event more than once.

Delivery Lifecycle

Each webhook delivery progresses through one of the following states:

  • PENDING — waiting to be delivered or scheduled for retry
  • DELIVERED — successfully received (2xx response)
  • DEAD_LETTER — failed permanently (non-retryable error or retries exhausted)
  • CANCELLED — subscription was inactive when processing started

Response Semantics

Your endpoint’s HTTP response determines whether a delivery is considered successful or retried.

Success

  • 2xx → delivery is marked as successful
  • No further retries are attempted

Non-retryable failures

The following responses are treated as permanent failures:

  • 400, 401, 402, 405, 406, 410, 413

Result:

  • Delivery is moved to DEAD_LETTER
  • No retries are attempted

Retryable failures

The following conditions trigger retries:

  • 408 (timeout)
  • Any other non-2xx status not listed above
  • Network errors or timeouts

Retry Behavior

Retries occur in two stages to balance fast recovery and long-term resilience.

Micro-retries (short-term)

  • Up to 2 attempts total
  • Occur within a single execution
  • Delay: randomized between 200 ms and 2000 ms

Used for:

  • transient network issues
  • brief service interruptions

Queue-level retries (long-term)

  • Up to 20 attempts total
  • Retry window: up to 72 hours
  • Delay increases over time using exponential backoff

Backoff characteristics:

  • Starts at approximately 30 seconds
  • Grows up to approximately 4 hours
  • Multiplier: 3x per attempt
  • Includes jitter (~±20%) to avoid retry spikes

Retry-After support

If your endpoint returns a Retry-After header:

  • Fynapse uses that value as the base delay
  • Adds a small positive jitter (~+10%)

This allows your system to control retry timing during overload.

Idempotency

Webhook delivery is at-least-once, not exactly-once. This means the same event may be delivered multiple times.

Why duplicates happen

Duplicates can occur when:

  • A retry is triggered after a failed delivery
  • A timeout occurs and the delivery outcome is unknown
  • Network interruptions prevent confirmation of success

Idempotency Key

Each request includes an Idempotency-Key header:

  • Uniquely identifies the delivery
  • Should be used to detect and ignore duplicate requests

How to handle duplicates

Your system should:

  • Store processed idempotency keys
  • Reject or ignore requests with previously seen keys
  • Ensure processing logic is safe to execute multiple times

Receiver Best Practices

To ensure reliable integration:

  • Return 2xx only after the message has been safely stored or processed
  • Do not perform long-running work before responding
  • Use asynchronous processing where possible
  • Deduplicate using the Idempotency-Key
  • Monitor failed deliveries and dead-letter events