Repository navigation
Request 51Degrees client hints from one middleware - #1386
Open
jwrosewell wants to merge 60 commits into
Open
jwrosewell wants to merge 60 commits into
jwrosewell wants to merge 60 commits into
Conversation
A browser sends its high entropy client hints only when a response asks for them with Accept-CH, retries its first navigation carrying them only when Critical-CH names them, and lets a third party read them only when the page delegates them, by a Delegate-CH tag in the page as served or a Permissions-Policy header. Nothing in the stack wrote any of that for a deployment whose device detection is 51Degrees, so every first page view of a session was priced on the User-Agent alone. The 51degrees.client_hints serve middleware, in crates/middleware/51degrees, writes all of it where a [[serve]] entry names it. As it filters a page request it asks the 51Degrees service which hints it wants from that browser, through the cache the 51Degrees modules share, and leaves the names in the request's state. From that list it writes Accept-CH, Critical-CH and Permissions-Policy through the headers step, each keeping what the response already said under that name, and the Delegate-CH tag in the head. No hint name is written down in the code, so the headers follow the service. The critical switch is in the middleware's own table, [middleware.51degrees.client_hints], on by default. What the 51Degrees modules share is crates/shared/51degrees: the table [shared.51degrees] with endpoint, license_key and timeout_ms, whose presence runs the 51Degrees builder, the license key every server side call presents and the refusal when the table has none, the one call to the service for a request's evidence as a form POST over the platform's HTTP client with its answer held in the loading cache so a question many requests ask at once is asked once, the evidence a request gives, and the type a module reads the hint names from the request's state by. The answers are held in the declared cache 51degrees-answers, selected and sized in [cache.51degrees-answers] as every declared cache is, so a stock build holds them with its one loading cache module without a line in the document. An answer is held for an hour at most. The stock list ships the builder, so every adapter and the ts tool take it, and its tests hold the new row, the fixture settings and the hook order. The unknown-setting test now plants the key in a vendor's shared table for a builder that table runs. Both crates name their maintainers.
Cargo.lock gains the entries for trusted-server-shared-51degrees and trusted-server-middleware-51degrees, which the workspace now holds, and nothing else moves.
The modules crate is gone, so the client hints middleware is listed by each adapter and the ts command after the ads.txt module, describes the middleware it supplies, and the shipped module test plants an unknown key in the vendor's shared table for a builder that table runs.
…inks The adapter's own list now declares the client hints module's cache, so the deploy check reads the same builders startup composes, the adapter's and the deployment's after them.
… and what it costs Review round 1 found that the request filter runs wherever the table is present and not only where an entry covers the page, that the cost to existing pages was not stated, that the timeout of each adapter and the answers on failure were not stated, and that nothing said what each navigation sends to the service. The design now says each, and records that only the Fastly adapter runs request filters, so the other three adapters make no call. It also gives the cache key, the settings, what the headers keep, the untested paths and the questions on consent, and drops a related spec that exists in no branch. The guide, the changelog and the middleware readme no longer say the filter follows the entries.
Review round 2 found that the related specs were named under a pull request that is now closed, that the guide still said the first call to the service waits within timeout_ms when the connection can take up to one second and each gap between bytes gets its own timeout_ms, and that code comments still say more than the design does. The design now names the pull request that holds the design documents, the guide matches the section on failure, and section 3 lists the comments left for a later change.
A browser keeps the hints an origin asked for with Accept-CH until a response from that origin names a different set, and only an empty Accept-CH clears them (Client Hints Infrastructure, section 3.2). Chrome 155 behaves the same way: after one response with Accept-CH, the high entropy hints arrive on every later navigation and in new tabs, through pages that send no Accept-CH. So the middleware now leaves Accept-CH and Critical-CH off the response to a page request that already carries every hint the service names. It still writes them when the response names an Accept-CH set of its own, because that set would replace the one the browser holds, and when any name is missing, which is how a hint the service starts asking for reaches the browser. Chromium sends three low entropy hints unasked, so only the whole list counts, and a hint counts whatever its value, because Sec-CH-UA-Model is an empty string on a desktop. Permissions-Policy and the Delegate-CH tag apply to their own page only, so every page still gets them. The filter leaves a marker beside the names when the request carried them all, and response_headers reads it. The guide, the configuration table, the crate README, the design note and the changelog say the same.
The request filter is registered by [shared.51degrees] alone, so it called the service on every page navigation once the table existed, whatever the [[serve]] entries and their paths said. A deployment with the table and no entry naming the middleware, or an entry for /news/ only, still sent each navigation's evidence to the service and kept the page to the origin path as private, no-store, with no headers written. The filter now asks only when a [[serve]] entry whose path covers the request path names 51degrees.client_hints, which is the rule the headers step already uses (PhaseEntries::for_path), so the call is made for exactly the pages whose responses the middleware writes on. Tests: a table with no entry makes no call, and an entry with path = "/news/" calls for /news/today and not for / or /newsletter. cargo test -p trusted-server-middleware-51degrees, 40 passed.
Each navigation sent the service the visitor's address, the User-Agent and the 51D_ cookies, and the answer cache was keyed on all three, so a held answer served only a request from the same address with the same cookies. The address and cookies were there so a future 51Degrees module could share the answer, and no such module exists. The evidence is now the User-Agent and every Sec-CH-UA request header (Sec-CH-UA and each header named Sec-CH-UA-...), by lower case name in name order, sent under the bare header names the service reads. No address, no cookie and no other header is sent. The cache key is built from the fields sent, as before, so one held answer now serves every reader whose browser sends the same values. The filter no longer reads the client address, and the cookie and address handling in the shared crate is removed with BROWSER_EVIDENCE. The answer lifetime stays one hour. Its comment gave the address as the reason, which no longer applies, so the comment now says only what the code does. Tests: no address and no cookie reaches the request the client builds (shared crate) or the request the filter causes (middleware crate), two readers on the same browser at different addresses make one call, and the evidence holds only the User-Agent and the Sec-CH-UA hints, in name order. Each guard test was checked to fail when every header is sent. cargo test: shared 47 passed, middleware 41 passed.
The middleware read only the first Accept-CH, Critical-CH and Permissions-Policy line a response carried, and core then wrote each header with HeaderMap::insert (write_response_headers in crates/trusted-server-core/src/middleware.rs), which replaces every line of that name. A response sending a header on two lines lost the second. The middleware now reads every line under each name, joins them into one list in order, and adds the hints after it, so nothing the response said is dropped. Critical-CH is read by the same code, so it keeps every line too. Test: a response sending each of the three headers twice keeps both lines of each, in order, with the hints after them. cargo test -p trusted-server-middleware-51degrees, 42 passed. cargo clippy on both 51Degrees packages with --all-targets -- -D warnings is clean.
… merging The design, the guide page, the changelog entry and the two crate readmes now say what the code does after the three fixes before this commit. - The request filter asks the service only for a page navigation whose path a [[serve]] entry naming the middleware covers, by the rule the headers use, so an entry's path limits the call as well as the headers and the tag, and a table with no entry makes no call. - The call sends the User-Agent and the Sec-CH-UA hints alone, with no address and no cookie, and one held answer serves every reader whose browser sends the same values. - Accept-CH, Critical-CH and Permissions-Policy keep every line the response carries. The design moves those three out of section 3 and out of the open questions, updates section 4 with the new tests (six untested paths remain), and adds a revision record row. The permission question stays open, now with what was found. The permission model names a data use by an IAB Tech Lab Privacy Taxonomy Data Use, none of which names device detection or client hints, and the nearest general ones are granted everywhere, consent gated in the GDPR groups, or denied everywhere in the sample policy, so choosing one is a policy decision. The hour an answer is held was chosen for the address, which is no longer sent, so whether it should be longer is added as a second open question. Prettier --check passes on every changed Markdown file.
The comment on client_hint_headers broke a sentence across two short lines when the merging fix rewrote its start. It now reads as one paragraph wrapped near 80 columns. No code changes.
… declares The push check now reads the secret settings of a vendor's shared table, so the test that plants [shared.51degrees] holding only no_such_setting was refused for its empty license_key before the unknown setting was reached, and every_linked_module_rejects_a_setting_it_does_not_know failed on the 51degrees builder. The planted table now names a key for each secret setting the builder declares, read from the builder itself so that a later module with secrets is covered too, which leaves the unknown setting as the only fault in the table.
jwrosewell
changed the title
jwrosewell
marked this pull request as ready for review
October 19, 2026 20:24
The shared crate's README, the module comments of engine_cache.rs, client.rs and evidence.rs, and the comments on CloudPipeline and accept_ch said that a question many requests ask at once is asked of the cloud service once. That holds only where a deployment links a loading cache module, and a stock build links none, so each now says so. The test services' comment calls the cache they use the 51Degrees module's rather than the stock one. The client hints middleware's tests link the 51Degrees loading cache as a deployment would, so the helper stock_builders() is linked_builders(), and the test that the shared table runs the middleware and a linked cache holds its answers is named and worded for a linked cache rather than the stock one.
The 51Degrees guide said that a stock build holds the service's answers with its one loading cache module, and that only the first call for a browser's values costs a round trip. A stock build links no loading cache module, so it asks the service on every page navigation the middleware is placed for, and the guide, the configuration guide, the example settings and the changelog now say that the answers are held only where a deployment links a loading cache module.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
crates/shared/51degreescrates/middleware/51degreesdocs/superpowersdocs/guideCargo.lockcrates/trusted-server-adapter-axum/teststrusted-server.example.tomlCargo.tomlcrates/trusted-server-adapter-axum/srccrates/trusted-server-adapter-cloudflare/srccrates/trusted-server-adapter-fastly/srccrates/trusted-server-adapter-spin/srccrates/trusted-server-cli/src.cargo/config.tomlCHANGELOG.mdcrates/trusted-server-adapter-axum/Cargo.tomlcrates/trusted-server-adapter-cloudflare/Cargo.tomlcrates/trusted-server-adapter-fastly/Cargo.tomlcrates/trusted-server-adapter-spin/Cargo.tomlcrates/trusted-server-cli/Cargo.tomlCloses
Closes #1333
Spec
Spec: 51degrees-client-hints
Test plan
CI on this pull request at 63b6bb3 passed all 20 checks.
cargo test-fastly && cargo test-axumcargo clippy-fastly && cargo clippy-axumcargo fmt --all -- --checkcd crates/trusted-server-js/lib && npx vitest runcd crates/trusted-server-js/lib && npm run formatcd docs && npm run formatcargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1fastly compute serveChecklist
unwrap()in production code (clippy deniesunwrap_usedin CI)logmacros as AGENTS.md asks (notprintln!)On top of the modules crate removal (#1384). Issue: #1333.
What changed after the 51Degrees PR review
Agreed at the review on 23 October, point 2. The 51Degrees loading cache is pluggable through modules, and the publisher asset feature becomes a module.
Agreed at the review on 23 October, point 1. No configuration or permission policy is compiled into the WASM, and a WASM without a policy does not start. Since #1345 a loaded domain carries its permission policy and every services graph needs one, so the tests below give theirs the sample policy.
Fixes found in review
Kept current
mainat 7aa5c99, the changes made lower in the stack andfiftyone-caching4.5.13 (Move the loading cache module to the released caching crate #1372).What this adds
What changes.
A deployment using 51Degrees gets client hints from one middleware.
crates/middleware/51degreesis the51degrees.client_hintsserve middleware, andcrates/shared/51degreesis what every 51Degrees module shares: the vendor's table, the one call to the service and its held answer, the evidence the service wants and the hint names a device module reads.How it works.
[[serve]]entry naming the middleware covers, sending theUser-Agentand theSec-CH-UAhints and nothing else of the request. It loads the answer through the declared cache51degrees-answers, which holds it only where the build links a loading cache module, and leaves the names in the request state under the vendor's id for a device module to read. On each page such an entry covers it writes theDelegate-CHtag in the head and, through the headers step,Accept-CH,Critical-CH(thecriticalswitch from its own table, on by default) andPermissions-Policyfor the delegation by header. Each keeps every line the response already carries, andAccept-CHandCritical-CHare left off where the request already carries every hint the service names and the response names noAccept-CHset of its own.[shared.51degrees]holdsendpoint,timeout_msandlicense_key, declared as a secret setting, and refuses a key it does not know. The service is called over the platform's HTTP client in the wire shape the vendor's engine sends, with no discovery round trip.51degrees-answers, selected and sized in[cache.51degrees-answers]as every declared cache is, and only where the build links a loading cache module. No stock build links one, so a stock build asks the service on every page navigation the middleware is placed for.Why
Anyone taking the stack with 51Degrees should get the benefit of client hints by naming one entry, as a deployment using InMobi Choice gets its consent prompt by naming one module. The code is the vendor's to maintain and lives in the vendor's crates, and core gains nothing, because the middleware uses the serve phase, the headers step and the middleware table the stack already has.
What breaks
A deployment without
[shared.51degrees]is unchanged. So is one that has the table and no[[serve]]entry naming51degrees.client_hints, which makes no call to the service and writes no header and no tag.A deployment on the Fastly adapter that adds the table and an entry changes the pages that entry covers.
User-AgentandSec-CH-UAhint headers the browser sent, and nothing else of the request (no address, no cookie and no other header). Answers are held only where the build links a loading cache module, so a stock build asks the service on every such navigation. Where one is linked, one held answer serves every browser that sends the same values, and an answer is held for an hour at most.private, no-store. A browser the service names no hints for gets its pages as before.criticalon, the browser repeats its first navigation to carry the hints, so that page is built twice.On the Axum, Cloudflare and Spin adapters the table is accepted and the license key is still required, but nothing is asked of the service and nothing is written, because only the Fastly adapter runs request filters.
The commits
tscommand list the middleware's builder after the ads.txt module, the crate describes the middleware it supplies, and the shipped module test plants an unknown key in the vendor's shared table for a builder that table runs.Accept-CHandCritical-CHoff the response to a page request that already carries every hint the service names, unless the response names anAccept-CHset of its own.Permissions-Policyand theDelegate-CHtag are still written on every page the entry covers.[shared.51degrees]alone, so it called the service on every page navigation once the table existed, whatever the[[serve]]entries said, and kept the page to the origin path as private and not stored with no header written. It now asks only when a[[serve]]entry whose path covers the request path names51degrees.client_hints, the rule the headers step uses.51D_cookies, and the held answer was keyed on all three. It now sends theUser-Agentand everySec-CH-UArequest header, in name order, and no address, cookie or other header, so one held answer serves every reader whose browser sends the same values.Accept-CH,Critical-CHandPermissions-Policyline a response carried, and core replaces every line of a name with the one line written, so a header sent on two lines lost the second. The middleware now joins every line in order and adds the hints after it.[shared.51degrees]gives the table a key for each secret setting the builder declares, so the unknown setting is the table's only fault.[cache.51degrees-answers] module = "51degrees"is refused unless the build linkscrates/loading-cache/51degrees.What is not in this pull request
A 51Degrees device module, so nothing reads the names the filter leaves yet. Request filters on the Axum, Cloudflare and Spin adapters, which run none, so on those adapters the middleware asks the service nothing and writes nothing. Whether the call to the service needs a permission, and how long an answer is held. The design page lists both as open questions.