Core Cache: Write Options

Every write to the Core Cache so far has passed the same record with a bunch of Nones in it. insert takes it, transaction-insert takes it, transaction-update takes it, and every post in this arc has quietly folded it away, almost as though it was boilerplate.

You'll be glad to know it isn't boilerplate. It's where a cached object's entire lifecycle gets decided, and it's the last thing standing between here and purging.

One common record used by four functions

/// Configuration for several functions that write to the cache:
/// - `insert`
/// - `transaction-insert`
/// - `transaction-insert-and-stream-back`
/// - `transaction-update`
///
/// Some options are only allowed for certain of these hostcalls; see the comments
/// on the fields.
record write-options {
  /// this is a required field
  max-age-ns: duration-ns,
  /// a full request handle, but used only for its headers
  ///
  /// Only allowed for non-transactional `insert`;
  /// in a transaction, the request headers are passed at `lookup` time, and cannot be changed later.
  request-headers: option<borrow<request>>,
  /// a list of header names separated by spaces
  vary-rule: option<string>,
  /// The initial age of the object in nanoseconds (default: 0).
  ///
  /// This age is used to determine the freshness lifetime of the object as well as to
  /// prioritize which variant to return if a subsequent lookup matches more than one vary rule
  initial-age-ns: option<duration-ns>,
  stale-while-revalidate-ns: option<duration-ns>,
  /// a list of surrogate keys separated by spaces
  surrogate-keys: option<string>,
  length: option<object-length>,
  user-metadata: option<list<u8>>,
  edge-max-age-ns: option<duration-ns>,
  sensitive-data: bool,

  /// Additional options may be added in the future via this resource type.
  extra: option<borrow<extra-write-options>>,
}

As we can see from the comment at the very top, this is a common record type that's shared by four functions, and "some options are only allowed for certain of these hostcalls." So it's a record whose validity depends on where you pass it: certain fields are only going to be used in the operation(s) that they're relevant for.

request-headers is the field that spells it out: "Only allowed for non-transactional insert; in a transaction, the request headers are passed at lookup time, and cannot be changed later." Which makes sense once you remember that a transaction starts at lookup, and the vary rule needs request headers to match against. By the time you're inserting, that decision is already made. If you set the field anyway, you're passing something the host has no legal way to use.

Half the fields have no doc comment at all. stale-while-revalidate-ns, length, user-metadata, edge-max-age-ns, and sensitive-data are bare declarations. For length and user-metadata you can work it out; edge-max-age-ns takes a trip through another SDK before it makes sense, which we'll do below; and sensitive-data as a bare bool is the kind of field whose entire meaning is in its name.

Time, in four flavors

Four of the ten fields are time durations, all measuring different things.

max-age-ns is how long the object stays fresh. This also happens to be the only field in the record that's absolutely required; all other fields in the entire record are optional fields.

stale-while-revalidate-ns is the window after expiry during which the object can still be served while somebody refreshes it. That's the stale and usable flag combination from the transactions post, seen from the writing end.

initial-age-ns is a strange one, and its doc comment gives it two jobs: determining freshness lifetime, and breaking ties when "a subsequent lookup matches more than one vary rule." It lets you write an object that is already partway through its life. If you fetched something from an origin that told you it was thirty seconds old, you can say so, and the cache's freshness maths starts from there instead of from now.

edge-max-age-ns has no doc comment, and its name doesn't actually help that much either. To find out what it actually does, I had to do some investigation into Fastly's SDKs. It turns out the Rust SDK has a builder function called deliver_node_max_age (opens in a new window), whose body writes to a field of its own named edge_max_age, and whose doc comment explains what that means: "the maximum time the cached item may live on a deliver node in a POP." We must remember that a POP is not one machine but a group of nodes, and this value bounds the object's life on the individual node that handles a request.

The two space-delimited strings

Curiously, these two fields are option<string> where you might expect a list:

/// a list of header names separated by spaces
vary-rule: option<string>,

/// a list of surrogate keys separated by spaces
surrogate-keys: option<string>,

The WIT language supports list<string>, and it's even used elsewhere in compute.wit. For some reason, these two fields don't use it, and instead carry a delimiter convention in a doc comment.

I would speculate the real reason for this to be that both of these correspond to things that travel as HTTP header values. A vary rule is a Vary header; surrogate keys arrive and leave as Surrogate-Key. Both are space-delimited in the wire format, so the ABI passes through what the wire already looks like rather than parsing and re-serializing at the boundary.

That's defensible, but it still pushes work onto every caller. It also creates a side effect: a key containing a space is unrepresentable.

What a vary rule is actually for

So what does vary-rule do?

Well, it is the reason request-headers exists on three separate records.

One cache key does not have to hold one object. A vary rule names request headers whose values become part of object selection, so a single key can hold several variants of the same resource and a lookup picks the one matching the request in hand.

compute.wit half-confirms this itself, surprisingly in the doc comment for initial-age-ns, passingly mentioning breaking ties when "a subsequent lookup matches more than one vary rule."

To get a clear understanding of the rule, I again consulted Fastly's Rust SDK, which calls the field vary_by (opens in a new window) rather than a vary rule. It documents it as "the list of headers that must match when looking up this cached item," with the matching spelled out:

A lookup will succeed when there is at least one cached item that matches lookup's cache key, and all of the lookup's headers included in the cache items' vary_by list match the corresponding headers in that cached item.

Which is where request-headers comes in, on both sides of the exchange. The writer decides which headers matter, by naming them in the vary rule. Every caller has to supply the headers to be compared, and that is the only thing request-headers is for. Its doc comment says as much, in the same words on all three records: "a full request handle, but used only for its headers." You hand over a whole request and the cache reads a few header values off it, ignoring the method, the URL, and the body entirely.

That also explains the transactional restriction from earlier. write-options.request-headers is "only allowed for non-transactional insert" because a transaction began at transaction-lookup, which took its own lookup-options with its own request-headers. The comparison already happened. Offering a different set at insert time would be answering a question the cache stopped asking.

surrogate-keys is also the field the whole next post depends on. A purge doesn't take a cache key, it takes a surrogate key, so whatever you write here is the only handle you'll ever have for invalidating this object later. Write nothing, and the object is unreachable by purge for as long as its max-age-ns lasts.

Reading it directly

Full working code: full example on GitHub (opens in a new window).

imports, decode and lookup_options, unchanged from the reading-a-hit article (18 lines)
mod bindings;

use bindings::{
    exports::fastly::compute::http_incoming,
    fastly::compute::{cache, http_body, http_resp},
};

fn decode(v: Result<Option<Vec<u8>>, cache::Error>) -> String {
    match v {
        Ok(Some(bytes)) => format!("{:?}", String::from_utf8_lossy(&bytes)),
        Ok(None) => "Ok(None)".to_string(),
        Err(e) => format!("Err({e:?})"),
    }
}

fn lookup_options() -> cache::LookupOptions<'static> {
    cache::LookupOptions { request_headers: None, always_use_requested_range: true, extra: None }
}

struct CoreCacheWriteOptions;

impl http_incoming::Guest for CoreCacheWriteOptions {
    fn handle(_request: http_incoming::Request, _request_body: http_body::Body) -> Result<(), ()> {
        let mut lines = String::new();
        let key = b"write-options-demo".to_vec();

        // Every field the record has, set to something distinguishable.
        let options = cache::WriteOptions {
            max_age_ns: 60_000_000_000,          // the only required field
            request_headers: None,               // non-transactional insert only
            vary_rule: Some("accept-encoding user-agent".to_string()),
            initial_age_ns: Some(5_000_000_000),
            stale_while_revalidate_ns: Some(30_000_000_000),
            surrogate_keys: Some("catalog product-42".to_string()),
            length: Some(11),
            user_metadata: Some(b"origin=example".to_vec()),
            edge_max_age_ns: Some(10_000_000_000),
            sensitive_data: false,
            extra: None,
        };

        let writing = cache::insert(&key, &options).map_err(|_| ())?;
        http_body::write(&writing, b"hello world").map_err(|_| ())?;
        http_body::close(writing).map_err(|_| ())?;

        // Which of them can be read back off the entry?
        let entry = cache::Entry::lookup(&key, &lookup_options()).map_err(|_| ())?;
        lines.push_str(&format!("max-age-ns:  {:?}\n", entry.get_max_age_ns()));
        lines.push_str(&format!("age-ns:      {:?}\n", entry.get_age_ns()));
        lines.push_str(&format!("length:      {:?}\n", entry.get_length()));
        lines.push_str(&format!("metadata:    {}\n", decode(entry.get_user_metadata(256))));
        lines.push_str(&format!("swr-ns:      {:?}\n", entry.get_stale_while_revalidate_ns()));
        cache::close_entry(entry).map_err(|_| ())?;

        // sensitive-data on a second key, to see whether it changes what comes back.
        let secret = b"write-options-sensitive".to_vec();
        let mut options = cache::WriteOptions {
            max_age_ns: 60_000_000_000,
            request_headers: None,
            vary_rule: None,
            initial_age_ns: None,
            stale_while_revalidate_ns: None,
            surrogate_keys: None,
            length: None,
            user_metadata: None,
            edge_max_age_ns: None,
            sensitive_data: true,
            extra: None,
        };
        options.user_metadata = Some(b"secret=yes".to_vec());
        let writing = cache::insert(&secret, &options).map_err(|_| ())?;
        http_body::write(&writing, b"do not log me").map_err(|_| ())?;
        http_body::close(writing).map_err(|_| ())?;

        match cache::Entry::lookup(&secret, &lookup_options()) {
            Ok(entry) => {
                lines.push_str("\n-- sensitive-data: true --\n");
                lines.push_str(&format!("state:    {:?}\n", entry.get_state()));
                lines.push_str(&format!("metadata: {}\n", decode(entry.get_user_metadata(256))));
                let _ = cache::close_entry(entry);
            }
            Err(e) => lines.push_str(&format!("\nsensitive lookup: Err({e:?})\n")),
        }

send the response, unchanged (5 lines)
        let response = http_resp::Response::new().map_err(|_| ())?;
        response.insert_header("content-type", b"text/plain").map_err(|_| ())?;
        let out_body = http_body::new().map_err(|_| ())?;
        http_body::write(&out_body, lines.as_bytes()).map_err(|_| ())?;
        http_resp::send_downstream(response, out_body).map_err(|_| ())?;

        Ok(())
    }
}

bindings::export!(CoreCacheWriteOptions with_types_in bindings);

Against Viceroy:

max-age-ns:  Ok(Some(60000000000))
age-ns:      Ok(Some(5000070709))
length:      Ok(Some(11))
metadata:    "origin=example"
swr-ns:      Err(Error::Unsupported)

-- sensitive-data: true --
state:    Ok(LookupState(FOUND | USABLE))
metadata: "secret=yes"

Look at the age. The object was written moments ago and reports itself as five seconds and seventy microseconds old, which is initial-age-ns of five seconds plus the real elapsed time since the insert. The field isn't advisory metadata; it moves the object's clock. Write something with an initial-age-ns past its max-age-ns and it is stale before you finish the request.

sensitive-data: true changed nothing observable from inside the guest. The state is a normal hit and the user metadata reads back exactly as written. Whatever that flag governs, it isn't the writing instance's own view of the object, which fits a field meant to keep content out of logs or off disk rather than to restrict guest access.

And get-stale-while-revalidate-ns is Error::Unsupported again, this time on an object that explicitly set a thirty-second window. That's the third getter on this list, after get-hits and the whole replace API.

Beyond the WIT

The record's doc comment is the interesting artifact here. It names four functions and then says some fields are only legal for some of them.

That's a union type wearing a record's clothes. insert accepts ten fields; transaction-insert accepts nine, because request-headers is meaningless once a transaction has started; transaction-update writes metadata only, so the fields governing body storage have nothing to act on. One WIT type covers all three, and the constraint that separates them lives in prose.

WIT does have the union type, as it happens. A variant is exactly that, and this series already took one apart in bot-category. So the question isn't whether the ABI could have said this. It's why it didn't.

Two reasons, and both are good ones. The first is that the record mirrors what actually crosses the boundary. The aside above showed the lookup side of that, a flat struct plus a mask; the write side has the same shape, one CacheWriteOptions that all four of these functions pass. A variant here would have the WIT describing a contract the layer underneath doesn't have, and every binding would flatten it straight back into one struct.

The second is that WIT gives you no way to share fields between types. No extension, no spread, no defaults. A variant with one case per call site means copying nine or ten declarations three times over and maintaining the copies by hand, and the first person to update three of the four cases ships a worse bug than the one they set out to prevent. Adding an optional field to a record is also backwards-compatible in a way that adding a case to a variant isn't, which matters a great deal for a record carrying an extra escape hatch whose entire purpose is adding things later.

Fastly's Rust SDK doesn't reproduce that. The builders in fastly::cache::core are split by call site: InsertBuilder, TransactionInsertBuilder, and TransactionUpdateBuilder are distinct types. A method that doesn't apply to a transactional insert simply isn't on the builder you're holding, so the doc comment's caveat becomes a compile error instead of a runtime surprise.

That's the case for builders that the insert-and-replace post left open. Splitting one over-general record into several precise types is something a builder does well and a record can't do at all. The cost is still real, and it's the same cost as before: max-age-ns is required by the record and optional in any builder that lets you call .execute() without it, so you trade a runtime error about an illegal field for a runtime error about a missing one.

Which suggests the split isn't builders versus records at all. It's whether your language can express "these ten fields, minus these two, with this one mandatory" as a type. Rust can, with three structs and no builders in sight. Most languages can, with enough boilerplate that nobody does it. The ABI declined to try, which is a defensible choice for a wire format and a poor one to copy into an API.

Next: we finally do something about all those surrogate keys, and find out how little of the cache a purge actually needs to know about.