Skip to content

fix: truncate input before sanitization to fix 816s JS test timeout - #50810

Draft
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/investigate-slow-javascript-tests
Draft

fix: truncate input before sanitization to fix 816s JS test timeout#50810
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/investigate-slow-javascript-tests

Conversation

Copilot AI commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

sanitizeContentCore was running all expensive operations — Unicode normalization, entity decoding, homoglyph mapping, 3× applyToNonCodeRegions passes, mention neutralization, URL policy — on the full raw input before truncating. A 600k-char input caused the "should truncate long content" test to hang for 816 seconds.

Change

  • Move applyTruncation to the top of sanitizeContentCore, before any other processing. Inputs exceeding maxLength (default 524,288) are cut immediately.
  • Retain the existing truncation call later in the pipeline for cases where normalization (stripping invisible chars) reduces length after the fact.
function sanitizeContentCore(content, maxLength, maxBotMentions) {
  if (!content || typeof content !== "string") return "";

  // Early truncation — avoid running expensive ops on oversized input
  content = applyTruncation(content, maxLength);

  // ... hardenUnicodeText, neutralizeAllMentions, applyToNonCodeRegions, etc.

  // Second pass after normalization (may further reduce length)
  sanitized = applyTruncation(sanitized, maxLength);
  ...
}

Result: compute_text.test.cjs (51 tests) drops from 816,365 ms (1 failed) → 387 ms (all pass).

…ocessing of large inputs

The "should truncate long content" test was timing out at ~816 seconds because
sanitizeContentCore ran all expensive operations (hardenUnicodeText, neutralizeAllMentions,
applyToNonCodeRegions, applyURLSanitizationPolicy, etc.) on the full 600,000-character input
before truncating at line 1382.

Fix: add an early applyTruncation call at the start of sanitizeContentCore so oversized
inputs are cut down to maxLength (default 524,288) before any other processing begins.
A second truncation pass is still applied later (after normalization may reduce length
by stripping invisible chars). This drops the test from ~816s to under 1s.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title fix: apply truncation early in sanitizeContentCore to prevent slow JS tests fix: truncate input before sanitization to fix 816s JS test timeout Aug 6, 2026
Copilot AI requested a review from pelikhan August 6, 2026 07:01
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Triage Summary

Category: bug (perf fix) | Risk: low | Priority score: 55/100

Score breakdown: impact 25 + urgency 15 + quality 15

Recommended action: fast_track

Small, well-scoped fix (5 additions, 1 file) that resolves an 816s test timeout by truncating input before expensive sanitization passes. No CI data available yet; no reviews posted. Low blast radius, high test-suite value.

Generated by 🔧 PR Triage Agent · auto · 33.2 AIC · ⌖ 2.45 AIC · ⊞ 7.9K ·

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Hey @copilot-swe-agent 👋 — thanks for fixing the performance issue in the sanitization pipeline! This early-truncation approach is solid and delivers a massive improvement (816s → 387ms).

One small thing to consider for completeness:

  • Add test coverage for the early truncation — while the existing test suite now runs much faster and passes, it would be good to add a unit test specifically for the early-truncation optimization. This ensures the behavior is explicitly covered if the function is refactored later.

If you'd like to add test coverage, here's a prompt:

Add a unit test to actions/setup/js/sanitize_content_core.test.cjs (or appropriate test file) that verifies:
1. When input exceeds maxLength (e.g., 600k chars), early truncation cuts it to maxLength before expensive operations.
2. The output matches the expected truncated length.
3. Performance: processing a 600k-char input completes in <1 second.

Otherwise, this is ready to go! 🚀

Generated by ✅ Contribution Check · auto · 51.7 AIC · ⊞ 8.7K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants