Skip to content

fix(storage): attach trailing checksum on final chunk of streaming resumable uploads - #18065

Open
chandra-siri wants to merge 4 commits into
mainfrom
fix/streaming-upload-trailing-checksum-471191823
Open

fix(storage): attach trailing checksum on final chunk of streaming resumable uploads#18065
chandra-siri wants to merge 4 commits into
mainfrom
fix/streaming-upload-trailing-checksum-471191823

Conversation

@chandra-siri

@chandra-siri chandra-siri commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Fixes b/471191823.

Root Cause

During streaming resumable uploads where initial content length is unknown (total_bytes starts as None), the SDK previously evaluated (start_byte + len(payload) == self._total_bytes) to determine when to append the trailing x-goog-hash checksum header. Because self._total_bytes remained None, this condition evaluated to False on the terminal chunk, causing the checksum header to be omitted from the final PUT request.

Fix

In ResumableUpload._prepare_request(), when get_next_chunk() detects EOF on an unknown-length stream (indicated by content_range transitioning from bytes start-end/* to bytes start-end/total), we now update self._total_bytes to the parsed total byte size. Consequently, the terminal chunk request properly attaches x-goog-hash.

Verification

  • Added positive and negative unit tests verifying x-goog-hash attachment on streaming uploads.
  • Added live GCS system tests validating trailing CRC32C verification and rejection on corrupted stream checksums.

@chandra-siri
chandra-siri requested a review from a team as a code owner August 11, 2026 13:12
@chandra-siri chandra-siri self-assigned this Aug 11, 2026
@chandra-siri
chandra-siri requested a review from v-pratap August 11, 2026 13:13

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for extracting and setting the total bytes during a resumable streaming upload when the size is initially unknown, enabling trailing checksum validation. It also includes comprehensive unit and system tests to verify this behavior. The review feedback suggests a cleaner and more robust approach to update self._total_bytes by directly using the total_bytes local variable instead of parsing the content_range string.

…ksum to accept BadRequest when trailing checksum header is rejected by GCS
@parthea parthea changed the title fix(storage): attach trailing checksum on final chunk of streaming resumable uploads (b/471191823) fix(storage): attach trailing checksum on final chunk of streaming resumable uploads Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants