compute.wit Changed: Filling In an extra

Two changes in Fastly's Viceroy repository have updated compute.wit since the copy this series has been quoting: pull request #659 (opens in a new window) on August 25, and a commit titled "Update to the latest WITs" (opens in a new window) on September 10. Viceroy's copy of compute.wit is the public one, the file every example in this series vendors, so a change there is a change to the ABI this series has been reading.

Compared with the copy the earlier posts quoted, the new file differs in five places:

1. log's endpoint resource has a new clone method.
2. backend's backend resource has one too.
3. extra-cache-override-details, which was empty, has a constructor and a method, set-lookup-timeout. This one came from #659, and the September 10 commit brought the rest.
4. purge-surrogate-key and purge-surrogate-key-verbose each gained a sentence in their doc comments.
5. Two other doc comments were corrected: await-entry's, in the Core Cache, and a stray > at the end of the link in override-host's.

The third change is the interesting one. Send Errors ended by promising to explain why compute.wit is full of empty extra resources once one of them got filled in. This is that one.

Two kinds of extra

compute.wit has sixteen places where it reserves room to grow with an extra resource, and they come in two shapes.

The first is a case at the end of a variant, the way send-error-detail ends with extra(extra-send-error-detail) and bot-category ends with extra(extra-bot-category). The host creates the resource and hands it to you, so these are for the host to say something your bindings don't have a name for yet.

The second is a field at the end of a record, for options you pass in. Here's the one that changed, in http-req:

/// The fields for the `override` arm of `cache-override`.
///
/// The origin response’s cache control headers will be used for ttl and
/// `stale-while-revalidate` if `none`.
record cache-override-details {
  ttl: option<u32>,
  stale-while-revalidate: option<u32>,
  pci: bool,
  surrogate-key: option<string>,

  /// Additional options may be added in the future via this resource type.
  extra: option<borrow<extra-cache-override-details>>,
}

The field is option<borrow<...>>. You make the resource, keep it, and lend it to the call. Until this update, there was nothing to make it with: the resource was declared as resource extra-cache-override-details {}, with an empty body. Now it has a constructor and a method:

/// Extensibility for `cache-override-details`
resource extra-cache-override-details {
  new: static func() -> extra-cache-override-details;
  /// Set an upper bound on the amount of time that can be spent waiting for a cache lookup,
  /// including waiting for a request collapse leader to return a result.
  set-lookup-timeout: func(timeout-ms: u32);
}

Three of the other record-shaped extra resources already had a constructor(): the Core Cache's extra-lookup-options, extra-write-options, and extra-replace-options. You could create those, but there was nothing to set on them. extra-cache-override-details is the first record-shaped extra with an option in it. (It's also spelled differently from those three: new: static func() instead of constructor(). Both produce a new resource, and bindings generators usually turn either into a constructor.)

Why a resource and not a field

The question Send Errors left open is why compute.wit does it this way. cache-override-details is a record. If Fastly wanted a lookup timeout, why not add lookup-timeout: option<u32> to it?

Because a record's fields are part of its type. set-cache-override takes a cache-override, which contains a cache-override-details. Add a field to the record and the type of set-cache-override changes with it. A component's imports are checked against what the host provides when the component is instantiated, so a program built against the old record would be asking for a function the host no longer offers in that form. Every guest built before the change would stop working.

A resource's methods, on the other hand, are separate functions. set-lookup-timeout is a new import that a program either uses or doesn't. A program built against the old WIT doesn't import it, doesn't know it exists, and keeps working. The record's type stayed exactly the same, because its extra field always had this type. What changed is what you can do with the resource it points to.

The same change looks different on Fastly's older, pre-component ABI, which Viceroy still implements alongside the WIT. There, the equivalent of the record is a set of arguments to a hostcall, and the way to add a field was a new function: cache_override_v3_set, next to the existing cache_override_set and cache_override_v2_set. It arrived in the same pull request as the WIT change, #659. #659's description explains why the older ABI needed a new function at all: "The existing cache override hostcalls didn't support adding additional fields," so the v3 hostcall is "structured so as to allow future extensibility." The WIT didn't need a third version of anything. The room was already there.

Using the lookup timeout

Request collapsing came up in Core Cache: Transactions: when two requests want the same thing and nothing is cached, one of them goes to the backend and the other waits for it. That's usually what you want. It isn't if the leader is slow and the follower would rather give up than wait. set-lookup-timeout puts a limit on the wait.

Full working code: full example on GitHub (opens in a new window).

The example's /collapse path starts two requests at once for the same URL, one that http-me takes two seconds to answer and marks cacheable. Only the second request gets the lookup timeout:

/// Starts a cacheable request for a slow URL, optionally with a cache
/// lookup timeout set through extra-cache-override-details.
fn start(
    backend: &backend::Backend,
    uri: &str,
    lookup_timeout: Option<u32>,
) -> Result<http_req::PendingResponse, ()> {
    let request = http_req::Request::new().map_err(|_| ())?;
    request.set_uri(uri).map_err(|_| ())?;

    if let Some(ms) = lookup_timeout {
        let extra = http_req::ExtraCacheOverrideDetails::new();
        extra.set_lookup_timeout(ms);
        request
            .set_cache_override(&http_req::CacheOverride::Override(
                http_req::CacheOverrideDetails {
                    ttl: None,
                    stale_while_revalidate: None,
                    pci: false,
                    surrogate_key: None,
                    extra: Some(&extra),
                },
            ))
            .map_err(|_| ())?;
    }

    http_req::send_async(request, http_body::new().map_err(|_| ())?, backend).map_err(|_| ())
}

extra lives only as long as that if block. The record borrows it for the one call to set-cache-override, and the setting stays on the request after the resource is gone.

The rest of the path waits for both requests with select and reports when each finished. Each run uses a fresh URL, so nothing is already in the cache. From a real Compute service, with no timeout:

run a1790491332, lookup timeout on the second request: None ms
second finished at  2104 ms: status Ok(200), x-cache HIT
first  finished at  2104 ms: status Ok(200), x-cache MISS

The two requests collapsed. One went to http-me, a MISS, and the other waited for it and was answered from the cache, a HIT, at the same moment. With a 500-millisecond lookup timeout on the second:

run b1790491332, lookup timeout on the second request: Some(500) ms
second finished at   501 ms: Error::GenericError, detail Some(SendErrorDetail::HttpResponseTimeout)
first  finished at  2076 ms: status Ok(200), x-cache MISS

The second request stopped waiting at 501 milliseconds and failed, with the same http-response-timeout detail Send Errors saw for a slow origin. It didn't go to the backend on its own. The first request carried on and was the only one to reach http-me.

That's what Fastly's Rust SDK documents for its Request::set_lookup_timeout: "If this timeout is exceeded, the request is cancelled with an error, without having been sent to the origin," and "the request is lost; it is not retried." A follower that times out needs a plan of its own, whether that's retrying, serving something stale, or reporting the error.

Under Viceroy, the timeout does nothing you can see, because Viceroy has no HTTP cache for requests to collapse in. #659 says so itself: "the override is a no-op." Both requests go to http-me and finish together.

clone

The two other additions are small, and they're the same method on two resources:

/// Returns a new owned handle to the same endpoint.
clone: func() -> endpoint;

/// Returns a new owned handle to the same backend.
clone: func() -> backend;

A handle to a resource is owned by exactly one place in your program, and dropping it releases it. Until now, the way to get a second owned handle to the same log endpoint or backend was to open it again by name. clone makes that direct.

The example's /clone-backend and /clone-log paths clone each one, drop the original, and use the clone. On a real service, both work:

backend clone: name "http_me", send through it: Ok(Ok(200))

log endpoint: wrote through the clone after dropping the original

Under Viceroy, the backend clone works too, but cloning a log endpoint traps:

Caused by:
    cannot lower a `borrow` resource into an `own`

Viceroy's endpoint.clone returns the very handle it was called on, which it only borrowed, as though it were a new one of its own. The runtime refuses to hand out a borrowed handle as an owned one, and the request fails with a 500. It's tracked as fastly/Viceroy#731 (opens in a new window), and until it's fixed, a program that clones log endpoints can't be tested locally.

The doc comment changes

The rest of the diff is doc comments, and one of them is worth repeating. purge-surrogate-key and purge-surrogate-key-verbose now both say:

/// Despite the name, the `surrogate-keys` argument only holds one key.

Caching: Three Caches, One Purge Surface and Purging by Surrogate Key both pointed out that the plural parameter name takes a single key. Now the WIT says so too.

await-entry's doc comment used to say it continues a lookup transaction, "waiting for the leader transaction if request collapsed." Core Cache: Transactions described it the same way, as the function that waits for the leader. The new doc comment drops the waiting:

/// Continues the lookup transaction from which the given busy handle was returned,
/// as a cache handle instead of a busy handle.
///
/// Note that cache handles also proceed asynchronously; the returned cache handle may not be ready
/// to read.
await-entry: func(
  handle: pending-entry,
) -> result<entry, error>;

Getting an entry back doesn't mean the leader has finished, or that the entry is ready to read. If you read that post's description of await-entry, this is the correction to carry with it.

Beyond the WIT

For a binding, an update like this is additive at the ABI level. Regenerate the bindings and the new methods appear, and every program built against the old ones keeps working, because nothing it imports has changed. That's the whole point of the extra resources, and this update is the first time one of the record-shaped ones has paid off.

Additive at the ABI level isn't the same as additive in a binding's API, though. A binding that modeled cache-override-details as a plain struct has to decide where a lookup timeout goes, and the easy answer, another field on the struct, is exactly the change the WIT avoided. That's fine for a binding, whose users recompile against it. What the binding has to handle is the translation: when the timeout is set, create an extra-cache-override-details, set it, and lend it to the call. When it isn't, pass none and create nothing.

Fastly's Rust SDK did add it as a field, on its CacheOverride type, in its 0.13.1 release, dated August 25, the same day as #659. Its set_cache_override still calls the older ABI's cache_override_v3_set, which Viceroy's component adapter translates onto the WIT call (#703 (opens in a new window)). Two ABIs, one feature, and an adapter in between. That's the situation any binding with a long history is in: the ABI it was built on and the ABI it's moving to both have to express every feature, and the new one is the one designed to grow.

The empty extra resources are the part of compute.wit that says the most about where it's going, and the least about what's there now. There's no telling which one gets filled in next, but whichever it is, you can rest assured that it won't break what you build today.