From c5698c322f037424709aaacfa3456676b3e7679f Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Fri, 23 Oct 2026 13:19:45 -0500 Subject: [PATCH 1/2] docs: make first detection and monitor mode clear --- README.md | 38 ++++++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index ea1abd9..71769e8 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,15 @@ # Stealth-scraper tripwires for Node.js -**Catch scrapers with honeypot paths, not fingerprinting.** Deterministic, zero-false-positive bot detection for Express, Fastify, and Next.js — **no account, no API key, three lines of code.** +**Turn requests to decoy paths into evidence you can act on.** WebDecoy adds tripwires and local rate limits to Express, Fastify, Next.js and Hono — **no account or API key required for local rules.** -Fingerprint- and challenge-based detection loses to purpose-built stealth scrapers ([botasaurus](https://github.com/omkarcloud/botasaurus), undetected-chromedriver, SeleniumBase-UC) that present a genuine browser fingerprint. Tripwires win a different fight: a hidden honeypot path a real user can never reach, so **any request for it is automated by construction** — it detects *intent*, which a better fingerprint can't spoof away. +Real-browser scrapers can resemble ordinary browsers. A hidden decoy link gives you another signal: a request for a path outside your normal user journey. Inspect the request and your legitimate traffic before choosing to block it. [![npm version](https://img.shields.io/npm/v/@webdecoy/node.svg)](https://www.npmjs.com/package/@webdecoy/node) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) - +[Install the Express adapter](#framework-integration-still-no-api-key) · [Verify your first detection](#verify-your-first-detection) · [Express documentation](https://docs.webdecoy.com/sdk-plugins/express/?utm_source=github&utm_medium=readme&utm_campaign=node_sdk_install) + +The middleware starts in **monitor mode**: it records verdicts without blocking requests. A local rules setup needs no cloud connection; an API key is needed to send detections to the WebDecoy dashboard. ## Quick start — no API key @@ -31,11 +32,11 @@ const wd = new WebDecoy({ }); // Inject the invisible, rel=nofollow decoy link into your HTML. Real users -// never see or follow it; a link-following scraper requests it and is blocked. +// do not encounter it in the normal UI. Inspect requests to the decoy path. html = html.replace('', `${trap.linkHtml}`); ``` -That's the whole thing. Runs entirely on your server, in memory, with no account and no outbound calls. With the [framework middleware](#framework-integration-still-no-api-key), a tripwire hit becomes a `403` automatically. +This creates the detector and decoy link; use the [framework middleware](#framework-integration-still-no-api-key) to inspect incoming requests. Local rules run on your server without an account or outbound calls. A tripwire hit is recorded in monitor mode and can return `403` in enforce mode. Register your own paths, prefixes, or patterns too: @@ -50,20 +51,20 @@ tripwire({ }); ``` -## Why tripwires beat fingerprinting +## Why add tripwires -Modern scraping frameworks are built specifically to defeat fingerprint- and challenge-based bot management. They run a real Chrome, strip `navigator.webdriver`, spoof canvas/WebGL/audio, and solve CAPTCHAs. Independent data: residential-proxy scrapers bypass major CDNs 80–99% of the time. Fingerprinting is an arms race you re-run every release. +Scraping frameworks can use real browsers and alter browser signals. Tripwires add evidence from requested paths alongside other detection methods. -A tripwire sidesteps the arms race. It doesn't ask *"does this look like a bot?"* — it exploits the one thing automation does that humans don't: **following links and hitting paths a person can never see.** +A tripwire evaluates whether a request matches a decoy or scanner-bait path. A hit is a reason to investigate; legitimate crawlers, link scanners and accidental requests still need consideration. - **Deterministic.** A hit isn't a probability; it's a request for a path that only exists to trap automation. -- **Zero-false-positive by design.** The decoy link is invisible and `rel=nofollow`; the default bait paths (`/.env`, `/.git/config`, `/wp-config.php`, …) are never requested by real traffic. (See [false positives](#what-counts-as-a-false-positive) for the honest edge cases.) -- **Unspoofable.** It catches *intent*, not a fingerprint — so a stealthier browser doesn't help the scraper. +- **Reviewable.** See [false positives](#what-counts-as-a-false-positive) before enforcing; hidden links and `rel=nofollow` do not guarantee that every requester is malicious. +- **Independent of browser appearance.** A matching request still matches when the requester uses a real browser. - **Instant.** Evaluated locally before any scoring, model, or network call. ## Framework integration (still no API key) -Tripwires (and the other [local rules](#more-local-rules)) are enforced automatically by the middleware — a hit returns `403`, no extra code. +The middleware evaluates tripwires and other [local rules](#more-local-rules). The default monitor mode records verdicts without blocking; enable enforcement only after reviewing your traffic. ### Express @@ -92,6 +93,19 @@ app.use( Fastify (`@webdecoy/fastify`), Next.js (`@webdecoy/nextjs`) and Hono (`@webdecoy/hono`) expose the same rule-based middleware. +### Verify your first detection + +After mounting the Express middleware above, log `req.webdecoyDecision` from a route after the middleware, then send a controlled request to your **local development app**: + +```bash +curl -i http://localhost:3000/.env +curl -i http://localhost:3000/ +``` + +The first request should produce a tripwire verdict; inspect `req.webdecoyDecision?.deniedBy('tripwire')` and `.results`. Monitor mode leaves your route’s HTTP response unchanged, so a `200` or `404` alone does not tell you whether a rule matched. Compare the ordinary `/` request too. Review real traffic before choosing `mode: 'enforce'`. + +Need the cloud dashboard? [Create a WebDecoy account](https://app.webdecoy.com/?utm_source=github&utm_medium=readme&utm_campaign=node_sdk_install) and configure your own API key. A keyless local verdict is not a cloud detection. + ### Hono — Workers, Bun, Deno ```bash From 8e77a45e71abdd9f94d886311d0093a55a280244 Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Fri, 23 Oct 2026 13:23:03 -0500 Subject: [PATCH 2/2] docs: align false-positive guidance with request evidence --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 71769e8..0a49ebf 100644 --- a/README.md +++ b/README.md @@ -210,11 +210,11 @@ Set `WEBDECOY_SITE_KEY`, `WEBDECOY_SCANNER_ID` and `WEBDECOY_SENSOR_KEY` from ** ## What counts as a false positive? -The zero-false-positive claim is about *design*, and it holds if you know the edge cases: +Tripwire matching is deterministic, but a match does not guarantee malicious intent. Review these edge cases before enforcing: -- **The honeytoken link** is rendered hidden and `rel=nofollow`, and isn't in your sitemap or nav — so real users and well-behaved crawlers (Googlebot respects `nofollow`) never follow it. A scraper that ignores `nofollow` and follows every link does. +- **The honeytoken link** is rendered hidden and `rel=nofollow`, outside your normal sitemap and navigation. Those properties reduce ordinary discovery; they do not guarantee that legitimate software will never request it, or that a scraper will follow it. - **Aggressive link-prefetchers and unfurl/preview bots** (e.g. some browsers' speculative prefetch, chat link previews) *can* fetch a hidden link. If you serve link previews or use prefetching, start with `dryRun: true` to measure before enforcing, or scope the trap to paths those bots won't touch. -- **The default scanner-bait paths** (`/.env`, `/.git/config`, …) are never hit by legitimate traffic — but *your own* security scanners or uptime checks might. Exclude them or run `dryRun` first. +- **The default scanner-bait paths** (`/.env`, `/.git/config`, …) can also be requested by your own security scanners, uptime checks or users troubleshooting a site. Exclude known legitimate activity or run `dryRun` first. The honest rule of thumb: enforce tripwires you control the surface of, and use `dryRun` to observe any path you're unsure about.