Behind the Panic

The Wasm Component Model on Fastly Compute

Simple Cache, Built by Hand

September 2, 2026, by Kats Omuro (@katsuyukiomuro (opens in a new window) on X/Twitter)

The map post established that Simple Cache has no interface in compute.wit at all—it turns out there are zero matches for "simple," anywhere in the file. The Simple Cache is a convenience layer each SDK builds on top of cache, documented by Fastly as a feature and implemented entirely in guest code.

The natural guess about a convenience layer is that it's the shallow end of the pool: fewer knobs, weaker guarantees, the thing you reach for when you'd rather not think about the intricacies of programming the cache.

A peek at the Rust SDK source proves that to be true—at least halfway. get really is a plain lookup. The implementation of get_or_set is actually a full request-collapsing transaction, obligation and all, a bit surprising considering its name and simple function signature.

In this post we'll be building the Simple Cache ourselves against the Core Cache in the compute.wit interface. Everything it needs has been covered by now: entry.lookup and insert from the first Core Cache post, transaction-lookup and the obligation flag from the transactions post, write-options from the options post, and purge-surrogate-key from last time.

What there is to build

We'll be referring to the Rust SDK a bit more than usual in this post as it's our reference implementation of the machinery. Keep in mind that during this exploration we are looking at SDK code and not our code that is build directly against compute.wit.

Fastly's Rust SDK puts five functions in fastly::cache::simple (opens in a new window): get, get_or_set, get_or_set_with, purge, and purge_with_opts.

Two of those pairs collapse immediately:

  • purge_with_opts is purge with a scope argument;
  • get_or_set_with returns the current value, or if a value is not set, atomically set one from a closure and return it. get_or_set does the same from a raw value. The closure-taking one is more versatile, so that is the shape this post will build.

So there are three real operations to build: get, get_or_set_with, and purge_with_opts.

get, which really is simple

get is a lookup with a flag check and a body read:

  1. entry.lookup with default options
  2. get-state, and bail unless found is set
  3. get-body with an unbounded range
  4. drain the body, close the entry

The Rust SDK's version is four lines. Look at how core::lookup is used here:

Rust
pub fn get(key: impl Into<CacheKey>) -> Result<Option<Body>, CacheError> {
    let Some(found) = core::lookup(key.into()).execute()? else {
        return Ok(None);
    };
    Ok(Some(found.to_stream()?))
}

That's entry.lookup, the non-collapsing entrypoint whose doc comment you may recall as having gone out of its way to say it "returns a result without waiting for any request collapsing that may be ongoing." There's nothing transactional here, no obligation, and none of the machinery from the transactions post. A miss is just None: nobody is elected, nobody waits, and nothing is owed.

Which is the whole of what separates this function from the next one. Here, a miss is an answer. None comes back, you act on it, and the story ends: the flow isn't resumable, the session isn't in the experiment, the page isn't warm so send a 404. Those are Fastly's own examples for the module, near enough—it names "the state required to resume an authentication flow" and "flags that have been set for A/B testing in a session" as what this interface is for, and neither is something you manufacture on the spot when it's missing. You look, you don't find it, and that settles it.

So don't reach for get when what you mean is "and then fill it." A miss you intend to close is not an answer, it's a job, and asking about it separately buys you a race and nothing else. That's the next function's entire reason for existing, and it's why this one can afford to be four lines.

Great! Now, fasten your seatbelts, because our ride is about to get wilder.

get_or_set_with, a full cache transaction

So: a miss that is a job. That turns out to cost far more than an insert bolted onto the end of a lookup, because the moment you intend to fill a gap, so does everyone else who just missed the same key. Here is the whole of get_or_set_with in the Rust SDK:

Rust
pub fn get_or_set_with<F>(
    key: impl Into<CacheKey>,
    make_entry: F,
) -> Result<Option<Body>, CacheError>
where
    F: FnOnce() -> Result<CacheEntry, anyhow::Error>,
{
    let key = key.into();
    let lookup_tx = Transaction::lookup(key.clone()).execute()?;1
    if !lookup_tx.must_insert_or_update() {2
        if let Some(found) = lookup_tx.found() {
            // the value is already present, so just return it
            return Ok(Some(found.to_stream()?));
        } else {
            // we're not in the insert-or-update case, but there's no found?
            return Err(CacheError::InvalidOperation);3
        }
    }
    // run the user-provided closure to produce the entry, tagging it as a user error if something
    // goes wrong
    let CacheEntry { value, ttl } = make_entry().map_err(CacheError::GetOrSet)?;4
    // perform a standard insert-and-read-back
    let (mut insert_body, found) = lookup_tx
        .insert(ttl)
        .surrogate_keys([
            surrogate_key_for_cache_key(&key, PurgeScope::Pop).as_str(),
            surrogate_key_for_cache_key(&key, PurgeScope::Global).as_str(),
        ])
        .execute_and_stream_back()?;5
    insert_body.append(value);
    insert_body.finish()?;
    Ok(Some(found.to_stream()?))
}

Every line of that is machinery that basically comes straight of the transactions post. The Simple Cache does all this in the module so that it's simple for the caller:

  1. Transaction::lookup is entry.transaction-lookup. This call participates in request collapsing, which means it can block: if another instance is already producing this object, you wait for it here, inside what looked like a get-or-set helper.
  1. must_insert_or_update() is the must-insert-or-update flag. When it's set, the cache has picked you, and everyone else who asked for this key is waiting on you.
  1. The else branch is the state the SDK doesn't believe in. Its own comment ends in a question mark, and the error it returns is InvalidOperation, whose doc comment says outright: "This should not arise during use of this API. If encountered, please report it as a bug."
  1. make_entry() runs after the election, not before. This is the whole reason the closure form exists, and the doc comment is precise about it in a way the summary line isn't: "The closure is only run when no value is present for the key, and no other client is in the process of setting it." That second clause is the transaction talking.
  1. execute_and_stream_back is transaction-insert-and-stream-back, the variant that hands back a read side along with the write side. The transactions post read the WIT as leaving open whether plain transaction-insert releases waiters the same way, and only this one gives you a way to watch it happen. The SDK needs that read side for a plainer reason: it owes the caller a value, and this is where the value comes from.

Then look at what comes back on both paths. It's found.to_stream(), the object as the cache now holds it. On a hit, that's somebody else's object. On a miss, it's the read side of the tuple transaction-insert-and-stream-back just handed you.

purge_with_opts and the secret of the two keys

Simple Cache's purge is actually quite clever; it stops being just a wrapper and starts being an invention.

As we saw last time, the purge interface takes a surrogate key and nothing else, so Simple Cache has to tag every object it writes with something it can recompute later. An example of this was the SHA-256 derivation from last time. But Simple Cache doesn't derive just one surrogate key value. It derives two, visible in the surrogate_keys call above, and here is the function behind them in the Rust SDK, in full:

Rust
fn surrogate_key_for_cache_key(key: &CacheKey, scope: PurgeScope) -> String {
    let mut sha = Sha256::new();
    sha.update(key);
    if let PurgeScope::Pop = scope {
        // if the POP string is empty for some reason, this will amount to a global purge
        // for now which is the safer choice
        let pop = crate::compute_runtime::pop();
        sha.update(pop);
    }
    let mut sk_str = String::new();
    for b in sha.finalize() {
        write!(&mut sk_str, "{b:02X}").expect("writing to a String is infallible");
    }
    sk_str
}

There is no scope parameter on purge-surrogate-key. There is no scope concept anywhere in compute.wit. The scoping is achieved entirely by which of two tags you choose to purge, and the POP-local tag is unguessable from any other POP because that POP's name is inside the hash.

The surrogate-keys field being a space-delimited string rather than a single value is what makes it possible at all: one object, two tags, one field. The shape that looked like a wire-format shortcut in the write-options post actually bears some load here.

compute-runtime.get-pop turns out to be one dependency of the caching layer. The interface that post treated as environmental trivia is what makes POP-scoped purging work.

And that comment about an empty POP string is a failure mode worth knowing about. If get-pop ever comes back empty, both derivations produce the same digest, and a purge that was scoped to one POP silently becomes global. The SDK calls that the safer choice, which it is, but "safer" here means over-purging rather than under-purging.

If you thought being able to purge an object at the POP level or the global level was a platform feature, you should be forgiven—but it turns out it's nothing more than cache entries tagged with two surrogate keys. A Simple Cache purge defaults to the local POP, and going global is the opt-in.

By the way, note the 02X. The digest is uppercase hex, and that isn't cosmetic. A surrogate key is matched as a string, so a lowercase digest is simply a different key, and the last section comes back to what that costs you.

Reading it directly

Below, then, is our reimplementation of the Simple Cache in Rust, against compute.wit.

Full working code: full example on GitHub (opens in a new window).

Rust

mod bindings;

use bindings::{
    exports::fastly::compute::http_incoming,
    fastly::compute::{cache, compute_runtime, http_body, http_resp, purge},
};
use sha2::{Digest, Sha256};
use std::cell::Cell;

fn lookup_options() -> cache::LookupOptions<'static> {
    cache::LookupOptions { request_headers: None, always_use_requested_range: true, extra: None }
}

fn drain(body: &http_body::Body) -> Vec<u8> {
    let mut out = Vec::new();
    while let Ok(chunk) = http_body::read(body, 1024) {
        if chunk.is_empty() {
            break;
        }
        out.extend_from_slice(&chunk);
    }
    out
}

/// Reads a found entry's body to the end. Both operations below need it.
fn read_body(entry: &cache::Entry) -> Option<Vec<u8>> {
    let options = cache::GetBodyOptions { from: None, to: None, extra: None };
    let body = entry.get_body(&options).ok()?;
    let out = drain(&body);
    let _ = http_body::close(body);
    Some(out)
}

/// Every SDK's Simple Cache derives *two* surrogate keys from one cache key, so
/// that `purge` can find the object again without anything being tracked in
/// between. The POP-scoped one folds this POP's name into the hash; the global
/// one hashes the key alone. Uppercase hex, because that is what the SDKs emit,
/// and a digest that doesn't match theirs byte for byte purges nothing.
enum PurgeScope {
    Pop,
    Global,
}

fn surrogate_key_for(key: &[u8], scope: PurgeScope) -> String {
    let mut sha = Sha256::new();
    sha.update(key);
    if let PurgeScope::Pop = scope {
        sha.update(compute_runtime::get_pop());
    }
    sha.finalize().iter().map(|b| format!("{b:02X}")).collect()
}

/// `get`: hand back the cached bytes, or nothing at all. Non-collapsing, on
/// purpose: a miss is a miss, and nobody is elected to do anything about it.
fn simple_get(key: &[u8]) -> Option<Vec<u8>> {
    let entry = cache::Entry::lookup(key, &lookup_options()).ok()?;
    let state = entry.get_state().ok()?;
    let bytes =
        if state.contains(cache::LookupState::FOUND) { read_body(&entry) } else { None };
    let _ = cache::close_entry(entry);
    bytes
}

/// `get_or_set`: a *transaction*, not a lookup with an insert bolted on. The
/// value arrives as a closure, and the closure only runs if the cache elects
/// this instance to produce the object.
fn simple_get_or_set<F>(key: &[u8], ttl_ns: u64, fill: F) -> Result<Vec<u8>, cache::Error>
where
    F: FnOnce() -> Vec<u8>,
{
    // Collapsing lookup: either the object is already here, or we get the job.
    let entry = cache::Entry::transaction_lookup(key, &lookup_options())?;
    let state = entry.get_state()?;

    if !state.contains(cache::LookupState::MUST_INSERT_OR_UPDATE) {
        // Someone else produced it, possibly while we were waiting. Read theirs.
        let found = read_body(&entry).ok_or(cache::Error::GenericError);
        let _ = cache::close_entry(entry);
        return found;
    }

    // Elected. Only now is the value worth producing.
    let value = fill();
    let options = cache::WriteOptions {
        max_age_ns: ttl_ns,
        request_headers: None,
        vary_rule: None,
        initial_age_ns: None,
        stale_while_revalidate_ns: None,
        // The space-delimited field earns its shape here: two tags, one object.
        surrogate_keys: Some(format!(
            "{} {}",
            surrogate_key_for(key, PurgeScope::Pop),
            surrogate_key_for(key, PurgeScope::Global),
        )),
        length: None,
        user_metadata: None,
        edge_max_age_ns: None,
        sensitive_data: false,
        extra: None,
    };

    // Write and read back at once, so every instance collapsed behind us is
    // released as the bytes land rather than after they finish.
    let (writing, reading) = entry.transaction_insert_and_stream_back(&options)?;
    http_body::write(&writing, &value).map_err(|_| cache::Error::GenericError)?;
    http_body::close(writing).map_err(|_| cache::Error::GenericError)?;
    let stored = read_body(&reading).ok_or(cache::Error::GenericError);
    let _ = cache::close_entry(reading);
    stored
}

/// `purge`: reachable only because `get_or_set` tagged the object on the way in.
/// POP scope is the default, matching the SDKs; global is the opt-in.
fn simple_purge(key: &[u8], scope: PurgeScope) -> Result<(), cache::Error> {
    let options = purge::PurgeOptions { soft_purge: false, extra: None };
    purge::purge_surrogate_key(&surrogate_key_for(key, scope), &options)
}

struct SimpleCacheByHand;

impl http_incoming::Guest for SimpleCacheByHand {
    fn handle(_request: http_incoming::Request, _request_body: http_body::Body) -> Result<(), ()> {
        let mut lines = String::new();
        let key = b"greeting";

        let show = |v: Option<Vec<u8>>| match v {
            Some(bytes) => format!("Some({:?})", String::from_utf8_lossy(&bytes)),
            None => "None".to_string(),
        };

        // Counts how many times a fill closure was run.
        let fills = Cell::new(0);

        lines.push_str(&format!("get before anything:  {}\n", show(simple_get(key))));

        let first = simple_get_or_set(key, 60_000_000_000, || {
            fills.set(fills.get() + 1);
            b"hello".to_vec()
        })
        .map_err(|_| ())?;
        lines.push_str(&format!(
            "get_or_set (elected): {:<12} fills run: {}\n",
            format!("{:?}", String::from_utf8_lossy(&first)),
            fills.get()
        ));

        // The second call isn't elected, so the closure is never run at all: the
        // value it would have written is never even produced.
        let second = simple_get_or_set(key, 60_000_000_000, || {
            fills.set(fills.get() + 1);
            b"REPLACED".to_vec()
        })
        .map_err(|_| ())?;
        lines.push_str(&format!(
            "get_or_set (found):   {:<12} fills run: {}\n",
            format!("{:?}", String::from_utf8_lossy(&second)),
            fills.get()
        ));

        lines.push_str(&format!("get after set:        {}\n", show(simple_get(key))));

        lines.push_str(&format!("\npop:        {:?}\n", compute_runtime::get_pop()));
        lines.push_str(&format!("pop key:    {}\n", surrogate_key_for(key, PurgeScope::Pop)));
        lines.push_str(&format!("global key: {}\n", surrogate_key_for(key, PurgeScope::Global)));

        lines.push_str(&format!("\npurge (pop scope): {:?}\n", simple_purge(key, PurgeScope::Pop)));
        lines.push_str(&format!("get after purge:      {}\n", show(simple_get(key))));

        // Nothing is cached now, so the same rejected closure gets elected.
        let third = simple_get_or_set(key, 60_000_000_000, || {
            fills.set(fills.get() + 1);
            b"REPLACED".to_vec()
        })
        .map_err(|_| ())?;
        lines.push_str(&format!(
            "get_or_set (elected): {:<12} fills run: {}\n",
            format!("{:?}", String::from_utf8_lossy(&third)),
            fills.get()
        ));


        let response = http_resp::Response::new().map_err(|_| ())?;
        response.insert_header("content-type", b"text/plain").map_err(|_| ())?;
        let out_body = http_body::new().map_err(|_| ())?;
        http_body::write(&out_body, lines.as_bytes()).map_err(|_| ())?;
        http_resp::send_downstream(response, out_body).map_err(|_| ())?;

        Ok(())
    }
}

bindings::export!(SimpleCacheByHand with_types_in bindings);

Against Viceroy:

Terminal output
get before anything:  None
get_or_set (elected): "hello"      fills run: 1
get_or_set (found):   "hello"      fills run: 1
get after set:        Some("hello")

pop:        "XXX"
pop key:    1C1B505EE00816D744F927BF4A6C707E5EC6C7238E9BD47DBA2A6D574BC6403E
global key: 18F6B0200B6FD32CE4E85B6C841F72247964195B8E1CD7C52E046DC51E48F779

purge (pop scope): Ok(())
get after purge:      None
get_or_set (elected): "REPLACED"   fills run: 2

The fills run column is what matters here, and it's what the lambda bought. The second call was handed a closure that would have produced "REPLACED", and the counter never moved, because must-insert-or-update came back clear and the code returned before it ever reached fill(). If you pass a value, you've already paid for it by the time the cache tells you it wasn't needed. If you pass a lambda, the cache decides whether you pay at all.

Then the purge empties the cache and the election runs again. The same closure that was refused a moment ago is now the one that gets picked, the counter moves to 2, and "REPLACED" is what lands. Nothing about the closure changed between those two calls. Only the cache's answer did.

The two derived keys are visibly different hashes of the same cache key, which is the POP name going into one of them. Viceroy reports its POP as "XXX", a placeholder rather than a real datacenter code, so locally the POP-scoped key is a hash of greeting plus that stub. On a real service it would be a hash of greeting plus something like LHR, and every POP would compute a different one.

transaction-insert-and-stream-back works under Viceroy, which isn't something this arc has been able to take for granted. get-hits, the replace API, get-stale-while-revalidate-ns and purge-surrogate-key-verbose have all come back Error::Unsupported along the way.

Beyond the WIT

Most of this series has been about bindings that wrap the ABI. This post is about building a feature out of other parts of the ABI. The feature has two important parts:

The first is behavior that isn't specified in compute.wit. The ABI knows nothing about what a purge scope is: there's no enum, no option field, no second function. The concept exists entirely in guest code, implemented by hashing a POP identifier into one of two tags, but it reaches SDK users as though it were a platform capability. In fact, from where they sit, it is one.

The second is the reverse: behavior that is specified but in a way the ABI can't express. A call to get_or_set can block because the cache elected another instance of your service to fill that key, and that is exactly the right outcome. Someone reaching for a "simple" API came here precisely so they wouldn't have to think about the intricacies of the cache, and collapsing is the module doing that thinking on their behalf. Nobody wants a hundred instances rendering the same page at the same moment.

This is actually not the SDK's own idea—Fastly's Caching Concepts (opens in a new window) page states it in its description. It describes the Simple Cache as having "always-on request collapsing, so if two operations attempt to populate the same cache key at the same time, the setter callback will only be executed once". And its comparison table sets "always-on" for simple caching, against "heuristic" for the readthrough cache, and "manual control" for the Core Cache.

So there are three documented interfaces, with three collapsing policies.

Which I find to be the more interesting problem. That three-way distinction is a promise Fastly makes about its product, and compute.wit cannot express any part of it: there is no collapsing flag and no marker on a function. That's a limit of the form rather than an oversight. An interface definition language describes shapes, and "this call may wait on another instance" isn't a shape. So the guarantee gets stated in documentation, and every SDK arrives at it from there.

The consequences are worth weighing before copying either pattern into a new binding. The scoping trick works, it needs no coordination and no state, and every SDK that implements it identically stays interoperable. But the derivation itself is unversioned and unspecified: change the hash, or the input to it, or even the case of the hex digest, and every object written by the old code becomes unpurgeable by the new code, silently, with no error anywhere. It's a compatibility contract that every implementation has to arrive at the same way, on its own.

The Rust SDK anticipates the need. surrogate_key_for_cache_key carries a doc comment describing it as a convenience for implementors who want to write a Simple Cache-compatible surrogate key by hand through the Core Cache API, which is exactly the case above. It's a private fn rather than a pub one, though, and its cross-reference points at a delete() that isn't in that module. The seam is described but not exposed.

For a new SDK, that convention is effectively a compatibility requirement, and reading source is the only way to find it. Match it and Simple Cache objects are mutually purgeable across languages. Deviate, even improve on it, and you get an isolated cache whose objects only your SDK can invalidate. That's worth knowing before deciding your hash should include a version prefix.

Next: back to HTTP, and the cache that already ran on every send you've made in this series.