fix: ScanDir SSRF self-allowlist, Flush requeue, remove dead RateLimiter, FailOn contract - #34
Open
Patel230 wants to merge 6 commits into
Open
fix: ScanDir SSRF self-allowlist, Flush requeue, remove dead RateLimiter, FailOn contract#34Patel230 wants to merge 6 commits into
Patel230 wants to merge 6 commits into
Conversation
ScanDir serves the target directory on an ephemeral 127.0.0.1 listener and crawls it through the crawler's private-IP blocking, which is enabled by default. Default-options scans (including the MCP inspect_scan_dir tool) therefore never fetched a page and returned a silently empty report; tests only passed by opting out with WithAllowPrivateIPs. Add a narrowly scoped private-IP allowlist to crawler.Config (PrivateIPAllowlist: exact host:port entries) honored at both SSRF layers - the transport dialer and validateURL - and have ScanDir register its listener address for the duration of that scan only. User-supplied URLs and all other private addresses stay blocked; entries are exact matches, so a different port on the same host is still rejected. Regression tests: ScanDir with default options now analyzes the local page, a loopback server not in the allowlist is still rejected, and crawler-level tests cover both the validateURL and dialer layers.
Flush swapped the buffer out before calling StoreBatch, so any sink error silently dropped the whole batch. Re-queue the failed batch at the front of the buffer (entries added while StoreBatch ran stay behind it) and keep returning the wrapped error, so the next Flush retries the same entries instead of losing them. Keep the re-queue bounded: the buffer is capped at maxBufferEntries (10,000); overflow drops the oldest entries and counts them in the new FindingsStore.Dropped accessor. Tests: failed batch is retained and retried verbatim on the next Flush, re-queue ordering, and the bounded-overflow drop behavior.
The exported RateLimiter (ratelimit.go) had zero non-test callers anywhere in inspect or the hawk workspace: the crawler rate-limits through its own internal per-crawl limiter (internal/crawler/rate.go). It also carried a latent bug - Close() panicked on double-Close. Delete ratelimit.go and ratelimit_test.go, and drop the now-unused golang.org/x/time dependency from go.mod. Crawler rate limiting is unchanged.
…actReport A bare field copy left verify.Report.FailOnSet false, so a user-configured below-critical threshold did not take effect in the contract Failed() gate. Call SetFailOn like sight does.
- go.mod: hawk-core-contracts v0.1.9 -> v0.1.13-0.20260815203243-0f60bf02 (branch fix/audit-sweep-2026-08 of hawk-core-contracts) — needed for verify.Report.SetFailOn/FailOnSet used by ToContractReport; re-pin to the tagged release once hawk-core-contracts#27 merges - go.mod + CI: Go 1.26.6 — 1.26.5 stdlib has reachable vulns that fail govulncheck
The browser submodule's go.mod still pinned hawk-core-contracts v0.1.9; its CI test step failed with 'updates to go.mod needed'. Aligns with the root module's v0.1.13-0.20260815203243-0f60bf02 pin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
host:portprivate-IP allowlist honored at both dialer and URL-validation layers;ScanDirregisters its ephemeral listener. User URLs remain fully protected. Default-optionsScanDirnow works.ToContractReportnow callsSetFailOnso user-configured thresholds take effect at the contract layer.Test plan
go test github.com/GrayCodeAI/inspect/...— all 8 packages pass-raceon root package