> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chunkr.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Polling, Webhooks & Production

> Async runs, webhooks, data retention, and limits on Extend

This page covers the operational side of the migration: how you wait for results, how you receive notifications, how long data lives, and what limits apply.

## Sync, polling, or webhooks

Chunkr gives you two ways to get a result: poll the task, or receive a webhook. Extend keeps both and adds a third, synchronous endpoints that hold the HTTP request open until the run finishes.

| Approach | Extend call | Best for | Trade-off |
| :- | :- | :- | :- |
| Synchronous | `client.parse(...)`, `client.extract(...)` | Testing, small files, low volume | Hard 5-minute timeout |
| SDK polling | `client.parse_runs.create_and_poll(...)` | Most production integrations | Process must stay alive for the run |
| Webhooks | `client.parse_runs.create(...)` plus an endpoint | High volume, event-driven backends | Needs a public endpoint |

<Warning>
  Never use the synchronous endpoints for production traffic. Large PDFs and spreadsheets routinely exceed 5 minutes. Chunkr's 1-hour task timeout has no equivalent on async runs; they run until complete.
</Warning>

## Polling

### Replacing a polling loop

If you followed our [task handling guide](/pages/task-system/task-handling#robust-polling-with-retry-logic), you have a `tenacity` or `p-retry` loop around `tasks.parse.get`. `create_and_poll` replaces it, including backoff.

<CodeGroup>
  ```python Chunkr theme={"system"}
  from tenacity import retry, retry_if_result, stop_after_attempt, wait_fixed

  @retry(
      retry=retry_if_result(lambda t: not t.completed),
      stop=stop_after_attempt(1500),
      wait=wait_fixed(3),
  )
  def get_task(task_id):
      return client.tasks.parse.get(task_id=task_id)

  task = client.tasks.parse.create(file=url)
  task = get_task(task.task_id)
  ```

  ```python Extend theme={"system"}
  from extend_ai import PollingOptions

  run = client.parse_runs.create_and_poll(
      file={"url": url},
      polling_options=PollingOptions(max_wait_ms=30 * 60 * 1000),  # Optional cap
  )
  ```

  ```typescript Extend (TypeScript) theme={"system"}
  const run = await client.parseRuns.createAndPoll(
    { file: { url } },
    { maxWaitMs: 30 * 60 * 1000 } // Optional cap
  );
  ```
</CodeGroup>

The helper polls every second for the first 30 seconds, then backs off to a maximum of 30 seconds between polls. Without `max_wait_ms` it polls indefinitely; with it, a `PollingTimeoutError` is raised when exceeded.

### Separate create and retrieve

If you store IDs and poll from a separate worker, the two-step pattern still exists.

<CodeGroup>
  ```python Python theme={"system"}
  # Producer
  run = client.parse_runs.create(file={"url": url})
  enqueue(run.id)

  # Consumer
  run = client.parse_runs.retrieve(run_id)
  if run.status in ("PROCESSED", "FAILED", "CANCELLED"):
      handle(run)
  ```

  ```typescript TypeScript theme={"system"}
  // Producer
  const run = await client.parseRuns.create({ file: { url } });
  enqueue(run.id);

  // Consumer
  const result = await client.parseRuns.retrieve(runId);
  if (["PROCESSED", "FAILED", "CANCELLED"].includes(result.status)) {
    handle(result);
  }
  ```
</CodeGroup>

### Async Python

Chunkr's `AsyncChunkr` client has an equivalent: `AsyncExtend`. Method names and arguments are identical to the sync client; add `await`.

```python theme={"system"}
from extend_ai import AsyncExtend

client = AsyncExtend()
run = await client.parse_runs.create_and_poll(file={"url": url})
```

***

## Webhooks

### What changes

| | Chunkr | Extend |
| :- | :- | :- |
| Provider | Svix | Extend-native |
| Setup | Chunkr dashboard, Webhooks section | Extend dashboard, Developers, Webhook endpoints (or the API) |
| Events | `task.parse.updated` on **every** status change | `parse_run.processed`, `parse_run.failed`, `extract_run.processed`, `extract_run.failed` on **terminal** states only |
| Signature | Svix headers, verified with Svix libraries | `x-extend-request-signature` + `x-extend-request-timestamp`, HMAC-SHA256, verified with the SDK |
| Large payloads | Not applicable | Optional signed download URL delivery |
| Local testing | Svix Play | Any tunnel (ngrok, Cloudflare Tunnel) or the `extend` CLI |

<Note>
  Chunkr fired a webhook for `Starting` and `Processing` as well as completion. If your handler ignored non-terminal events, nothing changes. If it relied on them for progress tracking, poll `retrieve` for intermediate status instead.
</Note>

<Note>
  Extend only emits run events for runs created through the API. Runs started from the Extend dashboard do not trigger webhooks.
</Note>

### Create an endpoint

You can do this in the dashboard or programmatically. The `signingSecret` is returned **once**; store it in your secrets manager.

<CodeGroup>
  ```python Python theme={"system"}
  endpoint = client.webhook_endpoints.create(
      url="https://example.com/webhooks/extend",
      name="Production",
      enabled_events=[
          "parse_run.processed",
          "parse_run.failed",
          "extract_run.processed",
          "extract_run.failed",
      ],
      api_version="2026-02-09",
  )

  print(endpoint.signing_secret)  # "wss_..." — returned only once
  ```

  ```typescript TypeScript theme={"system"}
  const endpoint = await client.webhookEndpoints.create({
    url: "https://example.com/webhooks/extend",
    name: "Production",
    enabledEvents: [
      "parse_run.processed",
      "parse_run.failed",
      "extract_run.processed",
      "extract_run.failed",
    ],
    apiVersion: "2026-02-09",
  });

  console.log(endpoint.signingSecret); // "wss_..." — returned only once
  ```
</CodeGroup>

### Verify and handle events

Replace your Svix verification with the SDK helper. Pass the **raw** request body; re-serializing JSON changes whitespace and breaks the signature.

<CodeGroup>
  ```python Python theme={"system"}
  from extend_ai.wrapper.errors import WebhookSignatureVerificationError

  @app.post("/webhooks/extend")
  def handle_webhook(request):
      try:
          event = client.webhooks.verify_and_parse(
              body=request.body.decode(),
              headers=dict(request.headers),
              signing_secret=os.environ["EXTEND_WEBHOOK_SECRET"],
          )
      except WebhookSignatureVerificationError:
          return {"error": "invalid signature"}, 401

      if event.event_type == "parse_run.processed":
          run = client.parse_runs.retrieve(event.payload.id)
          handle_parse(run)
      elif event.event_type == "extract_run.processed":
          run = client.extract_runs.retrieve(event.payload.id)
          handle_extract(run)
      elif event.event_type.endswith(".failed"):
          alert(event.payload.id, event.payload.failure_reason)

      return {"status": "ok"}
  ```

  ```typescript TypeScript theme={"system"}
  import express from "express";

  // Capture the raw body on the webhook route so the signature matches
  app.post("/webhooks/extend", express.raw({ type: "*/*" }), async (req, res) => {
    let event;
    try {
      event = client.webhooks.verifyAndParse(
        req.body.toString(),
        req.headers,
        process.env.EXTEND_WEBHOOK_SECRET!
      );
    } catch (err) {
      return res.status(401).send("invalid signature");
    }

    switch (event.eventType) {
      case "parse_run.processed": {
        const run = await client.parseRuns.retrieve(event.payload.id);
        await handleParse(run);
        break;
      }
      case "extract_run.processed": {
        const run = await client.extractRuns.retrieve(event.payload.id);
        await handleExtract(run);
        break;
      }
      case "parse_run.failed":
      case "extract_run.failed":
        await alert(event.payload.id, event.payload.failureReason);
        break;
    }

    res.status(200).send("ok");
  });
  ```
</CodeGroup>

Return a `2xx` promptly, as you did for Chunkr. Extend retries failed deliveries and the dashboard lets you inspect and re-send any message. For manual verification steps and the Go SDK, see [Extend's webhook guide](https://docs.extend.ai/webhooks/configuration#verifying-webhook-requests).

### Event payloads

Chunkr's payload was `{ event_type, task_id, status, message }`. Extend's `payload` depends on the event: `parse_run.*` events carry a minimal status object (`id`, `status`, `failureReason`, `failureMessage`, `metadata`) with no output, while `extract_run.*` events carry the full run. Treat the webhook as a notification and fetch the run by `payload.id`, as the handler above does; that keeps both paths identical and avoids depending on payload size. For very large payloads you can configure the endpoint to deliver a signed download URL instead; `verify_and_parse` handles that when called with `allow_signed_url=True`. The full catalogue is on Extend's [events page](https://docs.extend.ai/webhooks/events).

***

## Data retention

Chunkr stored outputs indefinitely unless you set `expires_in`. Extend has no per-run expiry parameter. To use Extend as a pure processing engine, delete runs and files once you have retrieved results.

<CodeGroup>
  ```python Python theme={"system"}
  run = client.parse_runs.create_and_poll(file={"id": uploaded.id})
  store(run.output)

  client.parse_runs.delete(run.id)
  client.files.delete(uploaded.id)
  ```

  ```typescript TypeScript theme={"system"}
  const run = await client.parseRuns.createAndPoll({ file: { id: uploaded.id } });
  await store(run.output);

  await client.parseRuns.delete(run.id);
  await client.files.delete(uploaded.id);
  ```
</CodeGroup>

Extend's default retention is documented in their [data handling policy](https://docs.extend.ai/security/data-handling).

### Presigned URLs

Chunkr's `base64_urls=True` option embedded assets inline so they would not expire. On Extend, figure images (`details.imageUrl`), `outputUrl`, and file `presignedUrl` all expire (15 minutes for parse outputs and file downloads). Download anything you need to keep as soon as the run completes.

***

## Limits

| | Chunkr | Extend |
| :- | :- | :- |
| Rate limit | 10 files per second, `429` on excess | Per-organization limits, `429` on excess. See [rate limits](https://docs.extend.ai/general/rate-limits). |
| Retry on `429` | SDK retried automatically | `create_and_poll` backs off automatically. Add your own retry (`tenacity`, `p-retry`) around other calls. |
| Task timeout | 1 hour | None on async runs; 5 minutes on sync endpoints |
| File size | No hard limit; 1 GB for base64 | Upload via `files.upload` or URL; base64 not supported |
| Pages per file | 2,000 soft limit | Use `advancedOptions.pageRanges` to process part of a large file; you are billed only for processed pages |
| Response size | Inline | Inline by default; `responseType=url` for large parse outputs |

### Error handling

Extend returns structured errors with a `code`, a `retryable` flag, and a `requestId` to quote to support.

| Code | Retryable | Meaning |
| :- | :- | :- |
| `INVALID_REQUEST` | No | Bad body or parameters (schema validation errors land here) |
| `UNAUTHORIZED` | No | Missing or invalid API key |
| `RATE_LIMIT_EXCEEDED` | Yes | Back off and retry |
| `USAGE_BLOCKED` | No | Out of credits |
| `INTERNAL_ERROR` | Yes | Server error |

Failed runs (`status: "FAILED"`) carry `failureReason` and `failureMessage`. See [Extend's error reference](https://docs.extend.ai/api-reference/error-handling).

## Next Steps

<Columns cols={2}>
  <Card title="FAQ" href="/pages/migrate-to-extend/faq" icon="circle-question">
    Accounts, deployment, legacy API, and feature differences.
  </Card>

  <Card title="Extend async processing guide" href="https://docs.extend.ai/general/async-processing" icon="arrow-up-right-from-square">
    Full polling options and webhook recommendations.
  </Card>
</Columns>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.