fix(storage): attach trailing checksum on final chunk of streaming resumable uploads - #18065
Open
chandra-siri wants to merge 4 commits into
Open
fix(storage): attach trailing checksum on final chunk of streaming resumable uploads#18065chandra-siri wants to merge 4 commits into
chandra-siri wants to merge 4 commits into
Conversation
…sumable uploads (b/471191823)
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds support for extracting and setting the total bytes during a resumable streaming upload when the size is initially unknown, enabling trailing checksum validation. It also includes comprehensive unit and system tests to verify this behavior. The review feedback suggests a cleaner and more robust approach to update self._total_bytes by directly using the total_bytes local variable instead of parsing the content_range string.
v-pratap
approved these changes
Aug 11, 2026
…ksum to accept BadRequest when trailing checksum header is rejected by GCS
… in unit and system tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes b/471191823.
Root Cause
During streaming resumable uploads where initial content length is unknown (
total_bytesstarts asNone), the SDK previously evaluated(start_byte + len(payload) == self._total_bytes)to determine when to append the trailingx-goog-hashchecksum header. Becauseself._total_bytesremainedNone, this condition evaluated toFalseon the terminal chunk, causing the checksum header to be omitted from the final PUT request.Fix
In
ResumableUpload._prepare_request(), whenget_next_chunk()detects EOF on an unknown-length stream (indicated bycontent_rangetransitioning frombytes start-end/*tobytes start-end/total), we now updateself._total_bytesto the parsed total byte size. Consequently, the terminal chunk request properly attachesx-goog-hash.Verification
x-goog-hashattachment on streaming uploads.