[Jamesr51d]

Stacks on making a file the origin does not have (#1309). The additions that follow the seven pull requests open here are listed in #777.

Compare split/22-creating with split/23-prepare

to see only this pull request's change.

What this pull request does

A middleware whose decision needs the request's services, or has to wait on the store or the network, answers Middleware::prepare with a future. The request's handler runs it before the document is processed, as the middleware's module with the permissions it declares, and what it answers is kept under the middleware's name and phase for create in that phase to take.

fn prepare<'a>(&'a self, call: ModuleCall<'a>, context: &'a MiddlewareContext<'a>) -> Option<Preparation<'a>> {

    Some(Box::pin(async move {

        let lines = self.lines_for(call, context.request_host).await?;

        Ok(Box::new(lines) as Box<dyn Any + Send>)

    }))

}



fn create(&self, context: &MiddlewareContext<'_>) -> MiddlewareAction {

    let lines: Vec<String> = context.take_prepared(MODULE).unwrap_or_default();

    MiddlewareAction { stream: Some(Box::new(AppendLines::new(lines))), ..MiddlewareAction::pass() }

}

A preparation may answer a Refusal instead, with a status, an optional Retry-After in whole seconds rounded up, and a message, and the request is answered with that status, no document is processed, and nothing is stored or served from the store.

Why

create is synchronous, is given no services, and is asked inside the synchronous streaming paths that the Fastly adapter needs. A middleware that makes a well known file from what the store holds or a service answers, as robots.txt does, could not decide there, and could not fail closed when it had nothing to give, which for robots.txt matters because an empty file tells crawlers to crawl everything. The preparation runs where the handler is asynchronous on every adapter, and only its answer, a Send value, travels to the streaming paths, so create, the plan and every streaming path are as they were.

The rules

• A middleware that prepares nothing is asked create as before, and nothing of it changes.

• On a document fetched for the request that will not be stored, both phases are prepared, fetch then serve. On a document that may be stored, only the fetch phase is prepared, so the serve middleware of the reader it was fetched for find nothing prepared. On a copy read from the store, the serve phase alone. The preparations of a phase run in the order the entry names the middleware.

• The first refusal ends the preparations of a document, and the response it makes is marked private and unstorable.

• A fetch preparation of a document that may be stored is handed the request's services and nothing of the reader, neither its permissions, location nor Edge Cookie, because its answer is stored with the document.

What breaks

MiddlewareContext, HtmlProcessorConfig and ReaderDocument each gain a required pub field prepared, so every struct literal of them outside this repository stops compiling.

How it was verified

CI passed on 1c0c244f7 on the fork. Not exercised: no preparation has run against a real store or network, because the fixtures answer from what they are called with. The preparation on a copy read from the store has no test, because the test services have no template cache.

References #1281.

[jwrosewell]

Moved into this repository as #1364, one branch of the stack, so the review continues there. This pull request is closed in its favor and its comments stay here for the record.