Request Handoff: Optimizing Long-Running Backend Requests

EDIT (2026-10-06): retitled from "Pending Responses: Editing Headers You'll Never Hold" to use Fastly's name for this mechanism, request handoff, and added that name to the post.

Every proxy in this series has had the same shape. Send the request, wait for the response, then pass it on to the client with send-downstream. The program sits in the middle for as long as the backend takes to answer, holding a sandbox open for nothing more than a handoff.

http-resp has a way to skip the waiting, and a set of functions that exist only because of it: they change the headers of a response that the program will never actually hold.

Handing off a request and sending down its response while it's still pending

Two Requests at Once, by Hand introduced send-async, which starts a request and returns a pending-response right away. Every example that has used it since has waited on that handle with await-response. The alternative is to give it away:

/// Send a pending response back downstream in reply to an incoming request.
///
/// This will cause Compute to wait in the background for the pending response to
/// resolve into its headers and body, and then forward them back downstream.
///
/// If the pending request fails while sending and a response never materializes,
/// Compute will generate a 5XX response to send in its stead.
send-downstream-pending: func(
  handle: pending-response
) -> result<_, error>;

The handle is owned, so it's consumed. Compute takes over the wait and forwards whatever arrives, and the program can return from http-incoming.handle immediately. If the request fails, the client still gets an answer, a 5XX made up by the platform instead of an error your code never saw.

Fastly calls this request handoff (opens in a new window), and pitches it for long-running backend requests, such as the ones AI agents make.

That leaves a problem. A proxy usually wants to change something about the response on its way through: add a header that marks where it came from, strip one the origin shouldn't expose. The program won't be around when the response arrives, so it has to say what to change in advance.

Changing a response in advance

Three functions queue changes on a pending-response, and they mirror the header functions from Headers:

/// Sets a response header to the given value on a pending response.
///
/// This will discard any previous values for the given header name, including any
/// pending header changes previously queued via other `insert-header-pending` or
/// `append-header-pending` calls for the same header name.
///
/// The `target` argument controls under which conditions this header is applied.
insert-header-pending: func(
  pending: borrow<pending-response>,
  name: string,
  value: list<u8>,
  target: pending-response-kind,
) -> result<_, error>;

/// Adds a response header with given value to a pending response.
///
/// Unlike `insert-header-pending`, this does not discard existing values for
/// the same header name, but will instead result in multiple headers of the
/// same name in the final response, each with their own values.
///
/// The `target` argument controls under which conditions this header is applied.
append-header-pending: func(
  pending: borrow<pending-response>,
  name: string,
  value: list<u8>,
  target: pending-response-kind,
) -> result<_, error>;

/// Removes all response headers of the given name from a pending response.
///
/// This will also remove any changes previously queued via `insert-header-pending`
/// or `append-header-pending` for the same header name.
///
/// The `target` argument controls under which conditions this header is applied.
remove-header-pending: func(
  pending: borrow<pending-response>,
  name: string,
  target: pending-response-kind,
) -> result<_, error>;

They borrow the handle, so you can queue as many changes as you like before giving it away. Each doc comment spells out how it interacts with changes already queued for the same name.

The new part is target:

/// Control for under which conditions an operation should be applied to the result of
/// a `pending-response`.
///
/// This is primarily useful for allowing consumers of `send-downstream-pending` to
/// control whether a requested header change (`insert-header-pending` et al.) are
/// applied to only the actual origin response, the synthetic 5XX response generated
/// in lieu of an `error-with-detail`, or both.
enum pending-response-kind {
  /// Apply the queued change to the response sent downstream, regardless of whether
  /// it originally came from the origin or was a synthetic 5XX response.
  any,

  /// Only apply the queued change to an actual response to the request.
  response,

  /// Only apply the queued change to the synthetic response made when the request fails.
  error,
}

When you queue a change, you don't know whether the request will succeed. target lets you queue changes for both outcomes at once, and the platform picks the set that applies. A Retry-After belongs on the made-up 5XX and not on a real 200, and an x-outcome: origin marker the other way around.

Putting it together

Full working code: full example on GitHub (opens in a new window).

One function starts the request, queues changes for every outcome, and hands the pending response off:

imports (6 lines)
mod bindings;

use bindings::{
    exports::fastly::compute::http_incoming,
    fastly::compute::{backend, http_body, http_req, http_resp},
};

use http_resp::PendingResponseKind::{Any, Error, Response};

/// Starts a request, queues header changes for whatever comes back, and
/// hands the pending response to Compute to forward. Returns without
/// waiting for the backend.
fn forward(backend: &backend::Backend, uri: &str) -> Result<(), ()> {
    let request = http_req::Request::new().map_err(|_| ())?;
    request.set_uri(uri).map_err(|_| ())?;
    let pending = http_req::send_async(request, http_body::new().map_err(|_| ())?, backend)
        .map_err(|_| ())?;

    // On every response, whichever kind it turns out to be.
    http_resp::insert_header_pending(&pending, "x-via", b"compute", Any).map_err(|_| ())?;1
    // Only on a response that came from the backend.
    http_resp::append_header_pending(&pending, "x-outcome", b"origin", Response)
        .map_err(|_| ())?;1
    http_resp::remove_header_pending(&pending, "x-internal", Response).map_err(|_| ())?;1
    // Only on the 5XX that Compute makes up if the request fails.
    http_resp::append_header_pending(&pending, "x-outcome", b"synthetic", Error)
        .map_err(|_| ())?;2
    http_resp::append_header_pending(&pending, "retry-after", b"5", Error).map_err(|_| ())?;2

    http_resp::send_downstream_pending(pending).map_err(|_| ())
}

The handler calls it with three different requests. /ok asks http-me for a response that carries an x-internal: secret header, using its header= directive. /fail sends to no-such-host.invalid, a name that can never resolve. /slow asks http-me to wait two seconds before answering.

Running it

From a real Compute service, /ok:

HTTP/1.1 200
Content-Length: 0
x-outcome: origin
content-type: text/plain; charset=utf-8
age: 2
accept-ranges: bytes
x-cache: HIT
x-via: compute

And /fail:

HTTP/1.1 502
Content-Length: 11
content-type: text/plain
x-via: compute
x-outcome: synthetic
retry-after: 5

Bad Gateway

Viceroy applies the same headers in both cases. Its 502's body is different, though: instead of Bad Gateway, it's Viceroy's full error message, starting with "Backend connection error for 'broken'" and followed by the chain of causes. That's handy for debugging and a problem for a test that checks the body, and it's tracked as fastly/Viceroy#729 (opens in a new window).

/slow shows what the handoff buys. Viceroy's log says the program was done in under a millisecond:

INFO request{id=2}: handling request GET http://127.0.0.1:7676/slow
INFO request{id=2}: guest completed in 875µs

The client waited just over two seconds for http-me, and the sandbox wasn't part of that wait.

Fastly's Rust SDK has the same operations on its PendingRequest: set_response_header, append_response_header, and remove_response_header each take a PendingResponseKind target, and builder-style variants default the target to Any. send_to_client hands the request off. Its documentation adds a detail the WIT leaves out: when the response is being cached, the changes are "applied after the cache has been updated," so they change what the client sees, not what goes into the cache.

Beyond the WIT

The Rust SDK's send_to_client doesn't always return right away. Its documentation says the sandbox can finish early when the request is "a direct pass or hit-for-pass request, and no cache insertions are necessary." Otherwise, it waits.

That's because the SDK can do its own caching in the guest, as Caching: Host Cache vs. Guest Cache described. While the SDK is still looking up the cache, or waiting to decide whether a backend response should be stored, there's no host pending-response to queue changes on. The request is in the guest's hands. So the SDK keeps its own list of queued header changes, each with its target, and applies them to the response itself once it has one, skipping the ones meant only for the synthetic error. Only when a request goes straight through to the backend does the SDK pass the changes to the host's insert-header-pending and its siblings.

The general point for a binding is that queuing an operation on a host handle only works when the host handle exists and will be the thing that finishes the job. A binding that puts its own layer in front of the host, whether guest-side caching, retries, or a request that might be answered locally, has to keep its own record of the queued changes and replay them wherever the response ends up. The target semantics have to come along too. Something queued for error must never land on a real response because the binding forgot what it was for.

The SDK uses the host's pending edits for itself, too. When a guest-cached request turns into a plain forward, it queues X-Cache: MISS and X-Cache-Hits: 0 for the real response, and removes Surrogate-Key and Surrogate-Control unless the program asked to keep them. Those are headers the platform's own caching adds or strips, and the SDK uses pending-response-kind's response target so that a guest-cached miss looks the same to the client as a host-cached one.

Next: backends again. Dynamic backends have many more options than the handful we used, and a static backend will tell you most of its settings if you ask.