Repository navigation

Make robots.txt a middleware core supplies - #1313

Closed

Jamesr51d wants to merge 615 commits into

IABTechLab:mainfrom

jwrosewell:split/25-robots-txt-middleware

Closed

Make robots.txt a middleware core supplies#1313
Jamesr51d wants to merge 615 commits into
IABTechLab:mainfrom
jwrosewell:split/25-robots-txt-middleware

This pull request is big! We're only showing the most recent 250 commits

Commits on Oct 8, 2026
Commits on Oct 9, 2026
- Write a rewritten script back as the rewriter gave it
  jwrosewell
  committed
  68cb6c3
- Write script text held for a marker in the order it came
  jwrosewell
  committed
  9500037
- A page change can be a middleware that an entry places
  jwrosewell
  committed
  0b21fe3
- A page change can run for each reader, as a serve middleware
  jwrosewell
  committed
  4600b1c
- The Fastly adapter's probe tests cover a serve entry at startup
  jwrosewell
  committed
  9b4cb4c
- Two entries for the same pages are told to share one list
  jwrosewell
  committed
  e0bf6f2
- Helpers for a module's own tests of its middleware
  jwrosewell
  committed
  e17e058
- The GPT bootstrap script reaches a page with one set of line endings
  jwrosewell
  committed
  47ab129
- Testlight's page change is a middleware
  jwrosewell
  committed
  04c7bf3
- Permutive's page change is a middleware
  jwrosewell
  committed
  1e36ef2
- lockr's page change is a middleware
  jwrosewell
  committed
  b37c64b
- Didomi's page change is a middleware
  jwrosewell
  committed
  33d6790
- Sourcepoint's page changes are one middleware
  jwrosewell
  committed
  b97c5b4
- Google Tag Manager's page changes are one middleware
  jwrosewell
  committed
  c64c9c4
- DataDome's page changes are two middleware, one in each phase
  jwrosewell
  committed
  a6c4be1
- Prebid's page changes are one middleware
  jwrosewell
  committed
  4d6e7b2
- A registration states the attributes it puts on the bundle's tag
  jwrosewell
  committed
  60732a7
- APS states its bundle tag attribute and changes no page
  jwrosewell
  committed
  99066ff
- GPT's page changes are one middleware
  jwrosewell
  committed
  cb72186
- GPT diagnostics writes into a reader's copy as a serve middleware
  jwrosewell
  committed
  847a02d
- Next.js's page changes are one middleware
  jwrosewell
  committed
  d81b56b
- The JavaScript asset proxy changes a page as a middleware
  jwrosewell
  committed
  d4d5fe1
- Core's deferred and payload stand-ins change a page as middleware
  jwrosewell
  committed
  0b6c1e3
- Core's request stand-in writes into a reader's copy as a serve middleware
  jwrosewell
  committed
  072e362
- Test the two attributes core rewrites that nothing asserted on
  jwrosewell
  committed
  a00b005
- The four page hook traits are removed
  jwrosewell
  committed
  7e039ae
- The stock list says which middleware its modules supply
  jwrosewell
  committed
  3352f8d
- `ts audit` places the page changes of the modules it selects
  jwrosewell
  committed
  21623bb
- What was named for a hook is named for the middleware
  jwrosewell
  committed
  c4c4076
- A Fastly sandbox keeps an application for each app-config key
  jwrosewell
  committed
  448f41d
- A `__KEY` selector with `{host}` names an app-config blob for each host
  jwrosewell
  committed
  e802375
- A harness serves two publishers from one local Viceroy instance
  jwrosewell
  committed
  0e4f153
- The guides say how one service serves several publishers
  jwrosewell
  committed
  64750a4
- A build reports when and from what it was made
  jwrosewell
  committed
  12301a6
- Signed evidence of who operates a deployment and which build it runs
  jwrosewell
  committed
  4edaec8
- Every adapter serves the attestation pages
  jwrosewell
  committed
  94ae6a2
- The attestation guide, for an operator, a relying party and a reviewer
  jwrosewell
  committed
  dffb3b2
- What a key schedule may hold is tested
  jwrosewell
  committed
  896942c
- A harness checks the evidence as a verifier would
  jwrosewell
  committed
  a4cebd8
- The permissions inspector's lock file takes the two signing crates
  jwrosewell
  committed
  1060456
- ts keys rotates and retires the signing keys through the host
  jwrosewell
  committed
  e95ca11
- ts cache purge asks the host to purge
  jwrosewell
  committed
  95f4346
- The Edge Cookie lookups under /_ts/admin are removed
  jwrosewell
  committed
  492f172
- Key rotation leaves the publisher's domain
  jwrosewell
  committed
  86c9344
- The signing store ids leave the settings
  jwrosewell
  committed
  f0b4379
- The service holds no token for its own platform's API
  jwrosewell
  committed
  b4a22e9
- The cache purge route is removed
  jwrosewell
  committed
  57fa6dc
- No request is asked for a password
  jwrosewell
  committed
  3720762
- Bot protection is asked about the publisher's admin paths
  jwrosewell
  committed
  a5a8704
- A module can add rules to robots.txt, and core writes the file
  jwrosewell
  committed
  31dd42d
- The seam probe supplies a robots.txt contributor
  jwrosewell
  committed
  0188f5d
- Every adapter serves /robots.txt and the refusal's response header
  jwrosewell
  committed
  3701fbc
- The guides describe [robots-txt]
  jwrosewell
  committed
  6d27af2
- Core can remove the headers that describe a deployment
  jwrosewell
  committed
  f1b6fef
- Every adapter strips the diagnostic headers before it finalizes
  jwrosewell
  committed
  506727f
- The guide says which headers leave a response
  jwrosewell
  committed
  5b632c4
- Auction telemetry is a module [analytics] selects
  jwrosewell
  committed
  7505ff7
- The guides describe [analytics]
  jwrosewell
  committed
  75fc697
- The Model Terms cookie name is declared by the module that reads it
  jwrosewell
  committed
  f208663
Commits on Oct 10, 2026
- The InMobi Choice consent prompt is a module
  jwrosewell
  committed
  97a31f2
- CI runs a module's page script tests with Node
  jwrosewell
  committed
  8fb826d
- The guides describe [cmp.inmobi]
  jwrosewell
  committed
  4e1b379
- A middleware may run on a media type other than HTML
  jwrosewell
  committed
  a0a48b7
- The guides describe entries for other media types
  jwrosewell
  committed
  218636c
- The sellers a deployment adds to the publisher's ads.txt are a module
  jwrosewell
  committed
  1e20c83
- The guides describe [ads-txt]
  jwrosewell
  committed
  e660d2c
- A consent prompt runs in the phase the publisher chooses
  jwrosewell
  committed
  7b08d94
- The guides say a consent prompt's phase is the publisher's choice
  jwrosewell
  committed
  9cb281d
- A file the origin does not have is made by the middleware named for its path
  jwrosewell
  committed
  11706a2
- The guides say a file the origin does not have is made by its middleware
  jwrosewell
  committed
  303e923
Commits on Oct 11, 2026
Commits on Oct 12, 2026
Commits on Oct 13, 2026
- A cache table naming no module takes the one loading cache module the build links
  jwrosewell
  committed
  4bd093a
- The serve phase of a document that may be stored is prepared for its reader
  jwrosewell
  committed
  f393547
- Merge 23-prepare into 24-loading-cache
  jwrosewell
  committed
  c0a048a
- Merge 24-loading-cache into 25-robots-txt-middleware
  jwrosewell
  committed
  58c3456
- The Fastly adapter no longer writes X-TS-Version on every response
  jwrosewell
  committed
  eacc120
- The attestation evidence names the tag the build was made from
  jwrosewell
  committed
  a656298
- A serve middleware may write a response header, and is told the platform
  jwrosewell
  committed
  5cc8fe7
- Core supplies a middleware that writes the service version where named
  jwrosewell
  committed
  0991047
- Merge upstream main d2f0d8cc0 into split/1-ec-provider
  Jamesr51d
  committed
  6844050
- Merge 12-attestation into 13-removals
  jwrosewell
  committed
  7819409
- Merge 13-removals into 14-robots-txt
  jwrosewell
  committed
  1f1e27a
- Merge 14-robots-txt into 15-diagnostic-headers
  jwrosewell
  committed
  50c10da
- Merge 15-diagnostic-headers into 16-analytics
  jwrosewell
  committed
  04ee8ad
- Merge 16-analytics into 17-cookie-constant
  jwrosewell
  committed
  891a43c
- Merge 17-cookie-constant into 18-inmobi
  jwrosewell
  committed
  71bf6a3
- Merge 18-inmobi into 19-media-types
  jwrosewell
  committed
  3ea9c1f
- A serve middleware's headers reach a response that is not HTML
  jwrosewell
  committed
  6b5b486
- Merge 19-media-types into 20-ads-txt
  jwrosewell
  committed
  2cf9fbc
- Merge 20-ads-txt into 21-cmp-phases
  jwrosewell
  committed
  d8f020c
- Merge split/1-ec-provider into split/2-device-geo
  Jamesr51d
  committed
  138faf1
- Merge 21-cmp-phases into 22-creating
  jwrosewell
  committed
  ce608fd
- Merge 22-creating into 23-prepare
  jwrosewell
  committed
  95d4c4e
- Merge 23-prepare into 24-loading-cache
  jwrosewell
  committed
  1eae9e4
- Merge 24-loading-cache into 25-robots-txt-middleware
  jwrosewell
  committed
  5fb7c64
- Merge split/2-device-geo into split/3-permissions
  Jamesr51d
  committed
  a028c00
- Merge split/3-permissions into split/4-client-resolve
  Jamesr51d
  committed
  a70e9d2
- Merge split/4-client-resolve into split/5-response-hook-docs
  Jamesr51d
  committed
  61b1399
- Merge split/5-response-hook-docs into split/7-integration-seam-impl
  Jamesr51d
  committed
  b826d6e
- Lock the web-time dependency of the DataDome crate
  Jamesr51d
  committed
  af04ad6
- Merge split/7-integration-seam-impl into split/8-module-context
  Jamesr51d
  committed
  3f25457
- Merge split/8-module-context into split/9-visibility-endpoints
  Jamesr51d
  committed
  bf58e02
- Merge split/9-visibility-endpoints into split/10-page-middleware
  Jamesr51d
  committed
  b76c5b9
- Merge split/10-page-middleware into split/11-config-by-host
  Jamesr51d
  committed
  5ae3b03
- Merge split/11-config-by-host into split/12-attestation
  Jamesr51d
  committed
  b76c010
- Merge split/12-attestation into split/13-removals
  Jamesr51d
  committed
  2c7f32f
- Merge split/13-removals into split/14-robots-txt
  Jamesr51d
  committed
  8038607
- Merge split/14-robots-txt into split/15-diagnostic-headers
  Jamesr51d
  committed
  49940a8
- Merge split/15-diagnostic-headers into split/16-analytics
  Jamesr51d
  committed
  50a9d3d
- Merge split/16-analytics into split/17-cookie-constant
  Jamesr51d
  committed
  9680cdf
- Merge split/17-cookie-constant into split/18-inmobi
  Jamesr51d
  committed
  b431bf0
- Merge split/18-inmobi into split/19-media-types
  Jamesr51d
  committed
  33ba0bc
- Merge split/19-media-types into split/20-ads-txt
  Jamesr51d
  committed
  f18aed1
- Merge split/20-ads-txt into split/21-cmp-phases
  Jamesr51d
  committed
  0287372
- Merge split/21-cmp-phases into split/22-creating
  Jamesr51d
  committed
  33f6298
- Merge split/22-creating into split/23-prepare
  Jamesr51d
  committed
  dc3584d
- Merge split/23-prepare into split/24-loading-cache
  Jamesr51d
  committed
  92826a3
- Merge split/24-loading-cache into split/25-robots-txt-middleware
  Jamesr51d
  committed
  b04af3d
- Keep the compiled-in git version out of the legacy settings loader and run the Cloudflare auction test with no demand modules
  Jamesr51d
  committed
  d3bbc32
- Merge split/7-integration-seam-impl into split/8-module-context
  Jamesr51d
  committed
  bc28b3e
- Merge split/8-module-context into split/9-visibility-endpoints
  Jamesr51d
  committed
  497d553
- Merge split/9-visibility-endpoints into split/10-page-middleware
  Jamesr51d
  committed
  0b0320f
- Merge split/10-page-middleware into split/11-config-by-host
  Jamesr51d
  committed
  b8314cb
- Merge split/11-config-by-host into split/12-attestation
  Jamesr51d
  committed
  196da17
- Allow the host clock in the build script's build time
  Jamesr51d
  committed
  59e09b6
- Merge split/12-attestation into split/13-removals
  Jamesr51d
  committed
  f735e0f
- Merge split/13-removals into split/14-robots-txt
  Jamesr51d
  committed
  35eb5ae
- Merge split/14-robots-txt into split/15-diagnostic-headers
  Jamesr51d
  committed
  e4de006
- Merge split/15-diagnostic-headers into split/16-analytics
  Jamesr51d
  committed
  c4f0302
- Merge split/16-analytics into split/17-cookie-constant
  Jamesr51d
  committed
  2018369
- Merge split/17-cookie-constant into split/18-inmobi
  Jamesr51d
  committed
  27c720c
- Merge split/18-inmobi into split/19-media-types
  Jamesr51d
  committed
  cca2847
- Merge split/19-media-types into split/20-ads-txt
  Jamesr51d
  committed
  192f34b
- Merge split/20-ads-txt into split/21-cmp-phases
  Jamesr51d
  committed
  9945df2
- Merge split/21-cmp-phases into split/22-creating
  Jamesr51d
  committed
  5db9a1c
- Merge split/22-creating into split/23-prepare
  Jamesr51d
  committed
  03053fa
- Merge split/23-prepare into split/24-loading-cache
  Jamesr51d
  committed
  f4e81af
- Merge split/24-loading-cache into split/25-robots-txt-middleware
  Jamesr51d
  committed
  c56c240
- No adapter writes a version on a response
  jwrosewell
  committed
  a43a05a
- Merge 15-diagnostic-headers into 16-analytics
  jwrosewell
  committed
  7b19336
- Merge 16-analytics into 17-cookie-constant
  jwrosewell
  committed
  af8af38
- Merge 17-cookie-constant into 18-inmobi
  jwrosewell
  committed
  badd6fb
- Merge 18-inmobi into 19-media-types
  jwrosewell
  committed
  149280a
- Merge 19-media-types into 20-ads-txt
  jwrosewell
  committed
  de85e19
- Merge 20-ads-txt into 21-cmp-phases
  jwrosewell
  committed
  938be5d
- Merge 21-cmp-phases into 22-creating
  jwrosewell
  committed
  7723c5e
- Merge 22-creating into 23-prepare
  jwrosewell
  committed
  d9fb81a
- Merge 23-prepare into 24-loading-cache
  jwrosewell
  committed
  844b01f
- Merge 24-loading-cache into 25-robots-txt-middleware
  jwrosewell
  committed
  1624877

Commits

Commits on Oct 8, 2026

Commits on Oct 9, 2026

Commits on Oct 10, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026