`@fedify/netlify` for Netlify Async Workloads

[dahlia]

Background

Fedify can serve federation requests from Netlify Functions through the standard Request and Response APIs. Persistent state also does not need a Netlify-specific adapter: Netlify Database exposes a regular PostgreSQL connection string, so applications can use PostgresKvStore from @fedify/postgres.

The missing piece is message processing. Netlify Functions are request-scoped and cannot host the long-running MessageQueue.listen() loop used by queues such as PostgresMessageQueue. Netlify Async Workloads provides durable event delivery, delayed execution, configurable retries, and dead-letter handling on top of Netlify Functions.

This resembles the existing Cloudflare Queues integration. WorkersMessageQueue enqueues messages but does not consume them through listen(). A Cloudflare queue handler receives each message and passes it to Federation.processQueuedTask().

Add an @fedify/netlify package that applies the same model to Netlify Async Workloads.

Proposed API

The package should initially export:

• NetlifyMessageQueue, a MessageQueue implementation that sends Async Workloads events;
• createNetlifyQueueHandler(), a helper that turns an Async Workload event into a call to Federation.processQueuedTask();
• the public types used to configure those APIs.

An illustrative API follows. Exact names can change during implementation.

import {
  AsyncWorkloadsClient,
  type AsyncWorkloadConfig,
} from "@netlify/async-workloads";
import { getConnectionString } from "@netlify/database";
import {
  createNetlifyQueueHandler,
  NetlifyMessageQueue,
} from "@fedify/netlify";
import { PostgresKvStore } from "@fedify/postgres";
import postgres from "postgres";
import { builder } from "../../src/federation.ts";

const sql = postgres(getConnectionString());
const kv = new PostgresKvStore(sql);

const queue = new NetlifyMessageQueue({
  client: new AsyncWorkloadsClient(),
  eventName: "fedify:queue",
  orderingKv: kv,
});

export default createNetlifyQueueHandler({
  queue,
  federation: () => builder.build({ kv, queue }),
  contextData: (event) => ({
    deployId: event.request.headers.get("x-nf-deploy-id"),
  }),
});

export const asyncWorkloadConfig: AsyncWorkloadConfig = {
  events: [queue.eventName],
};

createNetlifyQueueHandler() should return the function produced by asyncWorkloadFn(), so it can be used directly as the default export of a file under netlify/functions/.

The federation factory should be called for each workload invocation. This avoids assuming that database connections or other mutable resources survive across Netlify Function invocations.

Event format

Queue messages should be wrapped in a small Netlify-specific envelope:

interface NetlifyQueueEventData {
  readonly message: Message;
  readonly orderingKey?: string;
}

Message remains opaque to applications. The adapter only stores and restores it for Federation.processQueuedTask().

The default event name should be namespaced, such as fedify:queue, and applications should be able to override it when multiple Fedify queues share one Netlify site.

Implementation plan

Package scaffold

Create the standard package structure:

packages/netlify/
├── src/
│   └── mod.ts
├── test/
│   ├── handler.test.ts
│   └── mq.test.ts
├── deno.json
├── package.json
├── README.md
└── tsdown.config.ts

Add packages/netlify to the root deno.json workspace and pnpm-workspace.yaml. Add @netlify/async-workloads to the pnpm catalog.

@netlify/async-workloads should be a peer dependency of @fedify/netlify. @fedify/fedify should use the same workspace peer dependency convention as the other integration packages.

The first version should target Netlify Functions. Supporting direct queue submission from Netlify Edge Functions can be considered separately after verifying that AsyncWorkloadsClient and its dependencies work in the edge runtime.

Queue producer

Implement NetlifyMessageQueue.enqueue() with these rules:

1. Wrap the Fedify message and optional orderingKey in NetlifyQueueEventData.

2. Convert MessageQueueEnqueueOptions.delay to the absolute millisecond timestamp expected by delayUntil:

   const delayUntil = options?.delay == null
     ? undefined
     : Date.now() + options.delay.total("milliseconds");

3. Call AsyncWorkloadsClient.send(eventName, { data, delayUntil }).

4. Throw if the returned sendStatus is not "succeeded" so that the caller does not treat a rejected event as enqueued.

5. Set nativeRetrial = true. Async Workloads should own retries and dead-letter handling.

enqueueMany() should initially call client.send() concurrently for each message. The public AsyncWorkloadsClient does not expose a batch method, so the adapter should not depend on the router's internal batch API.

listen() should throw a TypeError directing users to createNetlifyQueueHandler(), following the WorkersMessageQueue precedent.

Workload consumer

createNetlifyQueueHandler() should:

1. wrap its handler with asyncWorkloadFn();
2. validate that event.eventData contains a Fedify queue envelope;
3. acquire an ordering lock when the envelope contains orderingKey;
4. resolve the Federation instance and context data;
5. call federation.processQueuedTask(contextData, message);
6. release the ordering lock in a finally block.

Transient errors from processQueuedTask() must escape the handler. Since the queue has nativeRetrial = true, Fedify already leaves retry handling to the backend. Async Workloads will retry when the workload throws or rejects.

Malformed envelopes should throw ErrorDoNotRetry from @netlify/async-workloads, since retrying data that cannot be decoded will not repair it.

Ordering keys

Async Workloads uses FIFO routing but may run many events concurrently. FIFO therefore does not provide the per-key serialization required by MessageQueueEnqueueOptions.orderingKey.

NetlifyMessageQueue should accept a CAS-capable KvStore through orderingKv. A lock can use the following shape:

const token = crypto.randomUUID();
const acquired = await orderingKv.cas?.(
  ["fedify", "netlify", "ordering", orderingKey],
  undefined,
  token,
  { ttl: orderingLockTtl },
);

If the lock is already held, the handler should throw ErrorRetryAfterDelay with a configurable short delay. On completion, it should release the lock with another CAS operation:

await orderingKv.cas?.(lockKey, token, undefined);

Using the token prevents an expired workload from deleting a lock later acquired by another invocation.

If orderingKey is supplied without a KvStore that implements cas(), the adapter should fail clearly instead of silently weakening ordering guarantees. PostgresKvStore backed by Netlify Database is the recommended configuration.

Documentation and example

Add a Netlify section to docs/manual/deploy.md covering:

• provisioning Netlify Database and Async Workloads;
• creating PostgresKvStore with getConnectionString();
• defining the queue producer used by the web application;
• exporting the Async Workload consumer;
• configuring retry count and backoff through asyncWorkloadConfig;
• keeping federation delivery disabled on deploy previews unless preview federation is explicitly intended;
• the 500 KB Async Workloads event payload limit.

Add NetlifyMessageQueue to docs/manual/mq.md next to WorkersMessageQueue.

Add an example under examples/netlify-astro/ showing @fedify/astro, @astrojs/netlify, @fedify/netlify, and @fedify/postgres together. Existing pages may remain prerendered while ActivityPub endpoints run through the Netlify adapter.

Astro version compatibility and changes to @fedify/astro peer dependencies should remain a separate issue.

Tests and release integration

Unit tests should use a small AsyncWorkloadsClient-compatible fake rather than invoking Netlify services.

Cover at least:

• the default and custom event names;
• event envelope serialization;
• immediate and delayed enqueueing;
• enqueueMany() failure propagation;
• unsuccessful sendStatus;
• nativeRetrial;
• the unsupported listen() path;
• successful calls to processQueuedTask();
• transient error propagation;
• malformed envelopes mapped to ErrorDoNotRetry;
• ordering lock acquisition, contention, expiry, and token-safe release;
• rejection of orderingKey without CAS support.

Add the package to the normal package checks and verify it with:

mise run check-each netlify
mise run test-each netlify
mise run build

Add an @fedify/netlify section to CHANGES.md and include the package in the npm and JSR publishing workflow.

Out of scope

The package should not implement another Netlify-specific KvStore. Netlify Database is PostgreSQL and can be used through @fedify/postgres.

Automatic provisioning of Netlify Database or installation of the Async Workloads extension is also out of scope.

The initial package does not need to wrap Netlify's HTTP request handling. Federation.fetch() and framework integrations such as @fedify/astro already cover that boundary.

Acceptance criteria

• @fedify/netlify is published to npm and JSR.
• NetlifyMessageQueue supports immediate, delayed, and multiple-message enqueueing.
• Async Workloads owns retry and dead-letter behavior through nativeRetrial.
• createNetlifyQueueHandler() processes events through Federation.processQueuedTask().
• Invalid event data is not retried.
• orderingKey is enforced with a CAS-capable KvStore.
• listen() fails with actionable guidance.
• Package tests cover the producer, consumer, retries, and ordering locks.
• The message queue and deployment manuals document the Netlify setup.
• A working Astro and Netlify example is included.
• CHANGES.md and release metadata are updated.

References

• Fedify message queue manual
• WorkersMessageQueue
• Netlify Async Workloads
• Sending Async Workload events
• Async Workloads lifecycle and retries
• Async Workloads limitations
• Netlify Database with PostgreSQL drivers