Idempotency

Idempotency ensures that if you send the same request multiple times (due to network issues, retries, or webhook failures), you won't create duplicate events. This is crucial for maintaining accurate audit logs—you want to know what happened, not how many times your code tried to log it.

Why Idempotency Matters

The problem: Network requests can fail for many reasons:

  • Temporary network issues
  • Timeouts
  • Server errors (500, 502, 503)
  • Webhook retries
  • Scheduled job retries

Without idempotency: If your request fails and you retry it, you might create duplicate events. This makes your audit logs inaccurate and can cause confusion when investigating issues.

With idempotency: You can safely retry failed requests. If the original request actually succeeded, you'll get back the same event ID. If it failed, a new event will be created. Either way, you never get duplicates.

How It Works

The Idempotency-Key header is optional. When included, Archiva uses it to ensure idempotency:

  1. Compute a hash: The API creates a hash of your request body to identify the exact request
  2. Check for existing event: It looks for an event with the same idempotency key and body hash
  3. If found (original request succeeded): Returns the original event ID with a 200 response (not an error—this is success!)
  4. If not found (original request failed): Creates a new event and returns a 201 response
  5. If key reused with different body: Returns a 409 Conflict error (this indicates a bug—you're trying to reuse a key for a different event)

If no idempotency key is provided: The request will be processed normally and a new event will always be created, even if the same request is sent multiple times. No idempotency protection is applied.

Key insight: A 200 response with an idempotency key means "this request was already processed successfully." This is not an error—it's exactly what you want when retrying!

Using Idempotency Keys

The Idempotency-Key header is optional. If you want to protect against duplicate events from retries, include it. If omitted, each request will create a new event regardless of whether the same request was sent before.

Basic Usage

Include an Idempotency-Key header with a unique value (optional):

curl -X POST https://api.archiva.app/api/ingest/event \
  -H "Authorization: Bearer pk_test_your_api_key_here" \
  -H "Idempotency-Key: idem_550e8400-e29b-41d4-a716-446655440000" \
  -H "Content-Type: application/json" \
  -d '{
    "actionKey": "invoice.update",
    "entityType": "invoice",
    "entityId": "inv_12345",
    "actorType": "user",
    "actorId": "usr_123"
  }'

Generating Idempotency Keys

Generate a unique idempotency key for each request. Common patterns:

UUID-based:

const idempotencyKey = `idem_${crypto.randomUUID()}`;

Timestamp + Random:

const idempotencyKey = `idem_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;

Request-based:

const idempotencyKey = `idem_${requestId}_${hash(body)}`;

Response Scenarios

Successful Creation (201)

If the idempotency key is new and the event is created:

Status Code: 201 Created

{
  "eventId": "550e8400-e29b-41d4-a716-446655440000",
  "receivedAt": "2024-01-15T10:30:00.000Z"
}

Idempotent Replay (200)

If the same idempotency key and body are sent again:

Status Code: 200 OK

{
  "eventId": "550e8400-e29b-41d4-a716-446655440000",
  "receivedAt": "2024-01-15T10:30:00.000Z"
}

The eventId will be the same as the original request, and receivedAt will be the original timestamp (not the current time).

Conflict (409)

If the same idempotency key is used with a different request body:

Status Code: 409 Conflict

{
  "error": "Idempotency key conflict: same key used with different body"
}

This indicates that you're trying to reuse an idempotency key with different data, which could indicate a bug in your application.

Best Practices

1. Generate Unique Keys

Each request should have a unique idempotency key. Don't reuse keys across different requests or events.

// ❌ Bad: Reusing the same key for different events
const idempotencyKey = "static-key-123";

// ✅ Good: Unique key per request
const idempotencyKey = `idem_${crypto.randomUUID()}`;

2. Store Keys with Events

If you store events locally before sending, store the idempotency key alongside them:

const event = {
  actionKey: "invoice.update",
  entityType: "invoice",
  entityId: "inv_12345",
  // ... other fields
};

const idempotencyKey = `idem_${event.entityId}_${Date.now()}`;

// Store both locally
localStorage.setItem(`event_${event.entityId}`, JSON.stringify({
  event,
  idempotencyKey,
}));

3. Retry with Same Key

When retrying a failed request, use the same idempotency key:

async function createEventWithRetry(eventData, maxRetries = 3) {
  const idempotencyKey = `idem_${crypto.randomUUID()}`;

  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await fetch('https://api.archiva.app/api/ingest/event', {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${apiKey}`,
          'Idempotency-Key': idempotencyKey, // Same key for all retries
          'Content-Type': 'application/json',
        },
        body: JSON.stringify(eventData),
      });

      if (response.ok) {
        return response;
      }

      // Don't retry on 409 conflicts
      if (response.status === 409) {
        throw new Error('Idempotency key conflict');
      }

      // Retry on other errors
      await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempt) * 1000));
    } catch (error) {
      if (attempt === maxRetries - 1) throw error;
    }
  }
}

4. Handle 200 Responses

Treat 200 responses (idempotent replays) the same as 201 responses - they both indicate success:

const response = await createEvent(eventData, idempotencyKey);
const result = await response.json();

if (response.status === 200 || response.status === 201) {
  console.log(`Event created: ${result.eventId}`);
  // Handle success
} else {
  // Handle error
}

Key Expiration

Idempotency keys expire after 24 hours. After expiration, using the same key will create a new event instead of returning the original event ID.

If you need to retry after 24 hours, generate a new idempotency key.

Idempotency vs Deduplication

Idempotency ensures that the same request (same key + same body) returns the same result. This is useful for:

  • Network failures and retries
  • Webhook retries
  • Scheduled job retries

Deduplication prevents duplicate events even when the request body differs slightly (e.g., timestamps). The API does not currently support deduplication beyond exact idempotency.

Common Patterns

Webhook Retries

When retrying failed webhooks, use the webhook's unique identifier as the idempotency key:

async function sendWebhookEvent(webhookId, eventData) {
  const idempotencyKey = `webhook_${webhookId}_${webhookData.eventId}`;

  return fetch('https://api.archiva.app/api/ingest/event', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Idempotency-Key': idempotencyKey,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(eventData),
  });
}

Scheduled Jobs

For scheduled jobs that might run multiple times, use a deterministic idempotency key:

function generateIdempotencyKey(jobName, entityId, occurredAt) {
  // Deterministic key based on job and entity
  return `job_${jobName}_${entityId}_${occurredAt}`;
}

Batch Processing

For batch processing, include the batch ID and item index:

function generateIdempotencyKey(batchId, itemIndex) {
  return `batch_${batchId}_${itemIndex}`;
}