Core Cache: Transactions

EDIT (2026-08-29): the example's lookup-options now sets always-use-requested-range to true, matching the rest of the Core Cache posts. It changes nothing this example prints, since nothing here asks for a byte range.

Last time, both insert and lookup carried a doc comment insisting they were non-request-collapsing, and lookup's went a step further: it returns "without waiting for any request collapsing that may be ongoing." The Fastly folks who wrote these doc comments went out of their way to get across to you what these functions don't do.

Today's topic is the half that these functions were distancing themselves from. They deal with the same resource, in the same key space, with the same option records, but have a completely different contract about who does the work when fifty instances miss at once.

At a glance

Before discussing the individual functions, here's the shape of the whole thing: several instances ask for the same key at once, one of them is told to go produce the object, and the rest wait for what it stores.

One extra word in the signature

/// The entrypoint to the request-collapsing cache transaction API.
///
/// This operation always participates in request collapsing and may return stale objects. To
/// bypass request collapsing, use `entry.lookup` or `insert` instead.
transaction-lookup: static func(
  key: list<u8>,
  options: lookup-options,
) -> result<entry, error>;

The function signature is identical to that of lookup. Same parameters, same lookup-options, same result<entry, error> coming back. Nothing in the types distinguishes them.

What separates them is entirely in the doc comment, and it says two things: this one always collapses requests; and it may hand you a stale object.

Neither of these concepts is expressible in WIT, which is why you need to understand what's going on underneath: even though the only machine-readable difference is the name, the ABI gives you two functions that have vastly unequal runtime behavior.

The obligation

A cache lookup that misses normally just tells you it missed. A transactional one can hand you a job.

/// The status of this lookup (and potential transaction)
flags lookup-state {
  /// A cached object was found
  found,
  /// The cached object is valid to use (implies found)
  usable,
  /// The cached object is stale (but may or may not be valid to use)
  stale,
  /// This client is requested to insert or revalidate an object
  must-insert-or-update,
  /// The cached object is only usable if revalidation has failed.
  /// If usable-if-error is set, either must-insert-or-update will be set
  /// (in which case the client must revalidate before use)
  /// or usable will be set (in which case revalidation has already failed).
  usable-if-error,
}

This is a flags, not an enum. The distinction is the whole design: these aren't five mutually exclusive states, they're five bits that combine, and a real lookup routinely comes back with two or three set at once. usable implies found. stale can be set alongside usable, or not. And must-insert-or-update can arrive on its own, with nothing found at all.

That must-insert-or-update is the one that changes how you write code against this interface. It means the cache has picked you. The cache has missed, and out of every instance that asked for this key at this given time, you're the one being told to produce the object. And the others are waiting on you. compute.wit has a name for that role, which surfaces later in await-entry's doc comment: you're the leader.

Nothing in the ABI says where you get it from—a backend response is the obvious case, but a computed page, a rendered image, or a value you assembled out of three other stores all settle the same obligation.

transaction-insert is how you settle up:

/// Inserts an object into the cache with the given metadata.
///
/// Can only be used in if the cache handle state includes the `must-insert-or-update` flag.
///
/// The returned handle is to a streaming body that is used for writing the object into
/// the cache.
transaction-insert: func(
  options: write-options,
) -> result<body, error>;

"Can only be used in if the cache handle state includes the must-insert-or-update flag"—typo and all, that's the precondition. You can't insert transactionally unless you were the one who was asked to do so. The flag isn't just advisory information about what happened; it's a capability check on what you're allowed to do next.

transaction-update has a stricter version of the same rule, requiring two flags:

/// Update the metadata of an object in the cache without changing its data.
///
/// Can only be used in if the cache handle state includes both of the flags:
/// - `found`
/// - `must-insert-or-update`
transaction-update: func(
  options: write-options,
) -> result<_, error>;

found plus must-insert-or-update is the revalidation case: the object is there, it's gone stale, and you've been elected to refresh its metadata without re-uploading the bytes. That's why transaction-update returns result<_, error> and not a body. There's nothing to stream, because the data isn't changing.

Walking away

Being the leader is not the same as being able to deliver. Whatever you were going to fetch or compute might fail.

/// Cancel an obligation to provide an object to the cache.
///
/// Useful if there is an error before streaming is possible, for example if a backend is
/// unreachable.
transaction-cancel: func() -> result<_, error>;

transaction-cancel isn't the only door out, either. close-entry ends the interaction outright, and its doc comment spells out what that means when you were still carrying an obligation:

/// Closes an ongoing interaction with the cache.
///
/// If the cache handle state includes the `must-insert-or-update` (and hence no insert or
/// update has been performed), closing the handle cancels any request collapsing, potentially
/// choosing a new waiter to perform the insertion/update.
close-entry: func(handle: entry) -> result<_, error>;

The difference may sound subtle, but it's not. Don't take "potentially choosing a new waiter" lightly: the queue behind you doesn't collapse when you fail. One of the others who was waiting gets promoted to leader, and the work moves on without you.

So request collapsing is more than just a way to deduplicate requests. It turns out it's an election system that can promote fallbacks when necessary.

Streaming to two places at once

There's actually one more insert variant, and it exists to solve a specific (and annoying) problem:

/// Inserts an object into the cache with the given metadata, and return a readable stream of the
/// bytes as they are stored.
///
/// This helps avoid the “slow reader” problem on a teed stream, for example when a program
/// wishes to store a backend request in the cache while simultaneously streaming to a client
/// in an HTTP response.
///
/// The returned body handle is to a streaming body that is used for writing the object *into*
/// the cache. The returned cache handle provides a separate transaction for reading out the
/// newly cached object to send elsewhere.
transaction-insert-and-stream-back: func(
  options: write-options,
) -> result<tuple<body, entry>, error>;

This huge doc comment block may take a few reads to understand. I, for one, can tell you I needed to reach for a coffee the first time I read it.

Imagine this: you're fetching a large object from a backend. You want it in the cache, and you want it going out to the client who asked for it, and you'd rather not buffer the whole thing in memory to do both. How do we do that?

The naïve answer is to tee the stream, but that causes a subtle problem: the slower of the two consumers throttles the faster one.

Fastly's answer to this problem is to hand you a tuple: you get the usual body you write into, and also a separate entry you read out of, both pointing at the same object as it lands.

But, how does this help?

Well, it turns out that entry isn't a private read side built just for you. The moment you call transaction-insert-and-stream-back, every instance collapsed behind you on this key is unblocked as well, each holding an entry of its own on an object that hasn't finished arriving. None of them waits for you to close the body. They read at the rate you write, exactly as you do out of your own entry half of the tuple. So, write as quickly as you can into body.

That's the real payoff. Request collapsing stops meaning "everyone else waits until the object is complete." Still one transfer, but now it fans out to every waiter byte by byte as it lands.

Not waiting to find out

transaction-lookup blocks while request collapsing sorts itself out. There's an asynchronous door too:

/// The entrypoint to the request-collapsing cache transaction API, returning instead of waiting
/// on busy.
///
/// This operation always participates in request collapsing and may return stale objects. To
/// bypass request collapsing, use `entry.lookup` or `insert` instead.
transaction-lookup-async: static func(
  key: list<u8>,
  options: lookup-options,
) -> result<pending-entry, error>;

pending-entry should look familiar, because it isn't a new type:

/// Handle that can be used to check whether or not a cache lookup is waiting on another client.
use async-io.{pollable as pending-entry};

It's a pollable under an alias, the same aliasing trick pending-response used back in Two Requests at Once by Hand. Which means a cache lookup can go into the same select as a backend response and a body read, without the cache interface inventing any concurrency machinery of its own.

Two functions turn a pending-entry back into something useful: await-entry waits for the leader and gives you the entry, and close-pending-entry abandons the attempt before collapsing finishes.

There's a second pollable in here, for after you already hold an entry:

/// Returns a `pollable` representing the next step of work for this `entry`. The `pollable` can be used to wait until this `entry` is unblocked.
///
/// The `entry` may require `step`s to complete. When this returns `none`, the entry is ready.
step: func() -> option<pollable>;

Read that second paragraph carefully: "the entry may require steps to complete." Holding an entry doesn't mean the transaction is done with you. Note the plural, too—one step isn't promised to be enough, so this is a loop rather than a single check:

none is the answer you're waiting for, and it means go ahead: get-state, get-body, whichever call you were about to make. A pollable back instead means the entry has work left, and that handle is what you wait on—in a select next to a body read and a pending backend response, since it's the same pollable everything else in this ABI hands you. Then ask again.

Nothing forces you through that loop. Skip step and call get-state directly, and it waits "for the lookup to complete if necessary," in its own doc comment's words. step is what turns that hidden wait into one you can schedule around.

That's the same lesson transaction-lookup-async teaches from the other end. One of them hands you a pollable before you have an entry, the other while you're holding one, and neither is a mechanism the cache invented.

Reading it directly

Full working code: full example on GitHub (opens in a new window).

imports and the helpers carried over from the insert-and-replace article (42 lines)
mod bindings;

use bindings::{
    exports::fastly::compute::http_incoming,
    fastly::compute::{cache, http_body, http_resp},
};

fn write_options(max_age_ns: u64) -> cache::WriteOptions<'static> {
    cache::WriteOptions {
        max_age_ns,
        request_headers: None,
        vary_rule: None,
        initial_age_ns: None,
        stale_while_revalidate_ns: None,
        surrogate_keys: None,
        length: None,
        user_metadata: None,
        edge_max_age_ns: None,
        sensitive_data: false,
        extra: None,
    }
}

fn lookup_options() -> cache::LookupOptions<'static> {
    cache::LookupOptions { request_headers: None, always_use_requested_range: true, extra: None }
}

fn read_entry_body(entry: &cache::Entry) -> String {
    let options = cache::GetBodyOptions { from: None, to: None, extra: None };
    let Ok(body) = entry.get_body(&options) else {
        return "<no body>".to_string();
    };
    let mut out = Vec::new();
    while let Ok(chunk) = http_body::read(&body, 1024) {
        if chunk.is_empty() {
            break;
        }
        out.extend_from_slice(&chunk);
    }
    let _ = http_body::close(body);
    String::from_utf8_lossy(&out).into_owned()
}

struct CoreCacheTransactions;

impl http_incoming::Guest for CoreCacheTransactions {
    fn handle(_request: http_incoming::Request, _request_body: http_body::Body) -> Result<(), ()> {
        let mut lines = String::new();
        let key = b"transaction-demo".to_vec();

        // A transactional lookup on a key nothing has written yet.
        let entry = cache::Entry::transaction_lookup(&key, &lookup_options()).map_err(|_| ())?;
        let state = entry.get_state().map_err(|_| ())?;
        lines.push_str(&format!("first lookup state: {state:?}\n"));
        lines.push_str(&format!("  found:                {}\n", state.contains(cache::LookupState::FOUND)));
        lines.push_str(&format!("  must-insert-or-update: {}\n", state.contains(cache::LookupState::MUST_INSERT_OR_UPDATE)));

        // The state told us we owe the cache an object. Pay the debt.
        let writing = entry.transaction_insert(&write_options(60_000_000_000)).map_err(|_| ())?;
        http_body::write(&writing, b"written under obligation").map_err(|_| ())?;
        http_body::close(writing).map_err(|_| ())?;

        // A second transactional lookup, now that the object exists.
        let again = cache::Entry::transaction_lookup(&key, &lookup_options()).map_err(|_| ())?;
        let state = again.get_state().map_err(|_| ())?;
        lines.push_str(&format!("\nsecond lookup state: {state:?}\n"));
        lines.push_str(&format!("  found:                {}\n", state.contains(cache::LookupState::FOUND)));
        lines.push_str(&format!("  must-insert-or-update: {}\n", state.contains(cache::LookupState::MUST_INSERT_OR_UPDATE)));
        lines.push_str(&format!("  body:                 {}\n", read_entry_body(&again)));
        cache::close_entry(again).map_err(|_| ())?;

        // The async entrypoint hands back a pollable instead of blocking.
        match cache::Entry::transaction_lookup_async(&key, &lookup_options()) {
            Ok(pending) => {
                lines.push_str("\ntransaction-lookup-async: Ok(pending-entry)\n");
                match cache::await_entry(pending) {
                    Ok(awaited) => {
                        lines.push_str(&format!("await-entry state: {:?}\n", awaited.get_state()));
                        let _ = cache::close_entry(awaited);
                    }
                    Err(e) => lines.push_str(&format!("await-entry: Err({e:?})\n")),
                }
            }
            Err(e) => lines.push_str(&format!("\ntransaction-lookup-async: Err({e:?})\n")),
        }

send the response, unchanged (5 lines)
        let response = http_resp::Response::new().map_err(|_| ())?;
        response.insert_header("content-type", b"text/plain").map_err(|_| ())?;
        let out_body = http_body::new().map_err(|_| ())?;
        http_body::write(&out_body, lines.as_bytes()).map_err(|_| ())?;
        http_resp::send_downstream(response, out_body).map_err(|_| ())?;

        Ok(())
    }
}

bindings::export!(CoreCacheTransactions with_types_in bindings);

Against Viceroy:

first lookup state: LookupState(MUST_INSERT_OR_UPDATE)
  found:                false
  must-insert-or-update: true

second lookup state: LookupState(FOUND | USABLE)
  found:                true
  must-insert-or-update: false
  body:                 written under obligation

transaction-lookup-async: Ok(pending-entry)
await-entry state: Ok(LookupState(FOUND | USABLE))

The whole cycle in nine lines. The first lookup finds nothing and comes back with exactly one bit set: no found, no usable, just the obligation. After transaction_insert settles it, the second lookup has FOUND | USABLE and no obligation at all, because there's nothing left to do. The async entrypoint takes the same path and lands in the same place.

Notice what the flags did to the code. There's no if let Some(...) and no match on an enum, because a flags value isn't a case you match, it's a set you test membership in. wit-bindgen generated a bitflags type with a contains method, and asking "was I elected" is state.contains(LookupState::MUST_INSERT_OR_UPDATE).

This all works locally, which is a change from last time—the replace API was Error::Unsupported under Viceroy, but the transaction API is fully there.

Beyond the WIT

lookup and transaction-lookup have identical signatures. Every parameter, the return type, the error type: the same. A binding that maps WIT functions to language functions mechanically produces two methods that look interchangeable and behave nothing alike, and no type system anywhere will warn a caller who picks wrong.

The consequences aren't symmetrical either, which is what makes this worth designing around rather than documenting around. Reaching for lookup when you wanted collapsing means every instance doing the same work at once. Reaching for transaction-lookup when you wanted a quick peek means blocking until whoever the leader turns out to be is finished, and possibly inheriting an obligation you never checked for and will silently drop on the floor when your handle closes.

Fastly's Rust SDK doesn't leave that to a doc comment. fastly::cache::core exposes lookup() and a separate Transaction type, described in its own docs as what "enables request collapsing and revalidation"—the transactional path isn't a similarly-named function you might pick by accident, it's a different type you have to go and ask for. The obligation, in that shape, has somewhere to live: a type that knows it might owe the cache an object can enforce that in ways a bare entry can't.

That's the real question for any binding here, and it isn't about naming. must-insert-or-update is a runtime flag on a handle, and forgetting to check it is both easy and quiet. A binding can mirror that faithfully and let callers test a bit, or it can spend a type on it, so that "I have an obligation" and "I have a cache hit" are different things the compiler can tell apart. The ABI can't make that distinction for you, because flags are flags. A language with sum types can.

Next: what does it look like to start reading one of these entry objects? We'll be getting to the getters, byte ranges, and the other flags we didn't get to in this post.