Repository navigation
Fix InvalidPart on multipart uploads when a smaller part arrives before a larger lower-numbered one - #170
Open
ServerSideHannes wants to merge 7 commits into
Open
Fix InvalidPart on multipart uploads when a smaller part arrives before a larger lower-numbered one#170ServerSideHannes wants to merge 7 commits into
ServerSideHannes wants to merge 7 commits into
Conversation
A new upload numbers internal parts densely (client part N uses internal N) until a client part needs more than one internal part, then switches to sparse ranges (20 per client part). The switch did not look at parts that had already been allocated. When a short last part reached the proxy before a larger part 1, both were given internal part 2, so Complete listed number 2 twice and the backend answered InvalidPart. When the numbers did not overlap but were out of client order, Complete succeeded and the object could not be decrypted, because reads locate client parts by summing sizes in client order. Record each client part's internal range in the upload state when the part starts, and place every new part in the free window between its neighbours. Dense and sparse placement are kept whenever they are safe; a first single-internal part that arrives out of order reserves sparse room below it. If no safe placement exists the part is rejected up front with InvalidPart instead of corrupting the object at Complete. Complete now refuses internal part numbers that are not strictly ascending in client order rather than sorting them.
Bumps the security group with 3 updates in the / directory: [anyio](https://github.com/agronholm/anyio), [h2](https://github.com/python-hyper/h2) and [hpack](https://github.com/python-hyper/hpack). Updates `anyio` from 4.13.0 to 4.14.2 - [Release notes](https://github.com/agronholm/anyio/releases) - [Commits](agronholm/anyio@4.13.0...4.14.2) Updates `h2` from 4.3.0 to 4.4.1 - [Changelog](https://github.com/python-hyper/h2/blob/master/CHANGELOG.rst) - [Commits](python-hyper/h2@v4.3.0...v4.4.1) Updates `hpack` from 4.1.0 to 4.2.0 - [Changelog](https://github.com/python-hyper/hpack/blob/master/CHANGELOG.rst) - [Commits](python-hyper/hpack@v4.1.0...v4.2.0) --- updated-dependencies: - dependency-name: anyio dependency-version: 4.14.2 dependency-type: indirect dependency-group: security - dependency-name: h2 dependency-version: 4.4.1 dependency-type: indirect dependency-group: security - dependency-name: hpack dependency-version: 4.2.0 dependency-type: indirect dependency-group: security ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [source-map-js](https://github.com/7rulnik/source-map-js) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/7rulnik/source-map-js/releases) - [Changelog](https://github.com/7rulnik/source-map-js/blob/main/CHANGELOG.md) - [Commits](7rulnik/source-map-js@v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: source-map-js dependency-version: 1.2.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [devalue](https://github.com/sveltejs/devalue) from 5.8.1 to 5.9.4. - [Release notes](https://github.com/sveltejs/devalue/releases) - [Changelog](https://github.com/sveltejs/devalue/blob/main/CHANGELOG.md) - [Commits](sveltejs/devalue@v5.8.1...v5.9.4) --- updated-dependencies: - dependency-name: devalue dependency-version: 5.9.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
…12 updates Updates the requirements on [fastapi](https://github.com/fastapi/fastapi), [uvicorn[standard]](https://github.com/Kludex/uvicorn), [cryptography](https://github.com/pyca/cryptography), [pydantic](https://github.com/pydantic/pydantic), [pydantic-settings](https://github.com/pydantic/pydantic-settings), [uvloop](https://github.com/MagicStack/uvloop), [orjson](https://github.com/ijl/orjson), [moto[s3]](https://github.com/getmoto/moto), [ruff](https://github.com/astral-sh/ruff), [mypy](https://github.com/python/mypy), [boto3-stubs[s3]](https://github.com/youtype/mypy_boto3_builder) and [fakeredis](https://github.com/cunla/fakeredis-py) to permit the latest version. Updates `fastapi` from 0.141.1 to 0.146.0 - [Release notes](https://github.com/fastapi/fastapi/releases) - [Commits](fastapi/fastapi@0.141.1...0.146.0) Updates `uvicorn[standard]` to 0.54.0 - [Release notes](https://github.com/Kludex/uvicorn/releases) - [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md) - [Commits](Kludex/uvicorn@0.52.4...0.54.0) Updates `cryptography` from 50.0.1 to 50.0.2 - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@50.0.1...50.0.2) Updates `pydantic` from 2.13.5 to 2.14.1 - [Release notes](https://github.com/pydantic/pydantic/releases) - [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md) - [Commits](pydantic/pydantic@v2.13.5...v2.14.1) Updates `pydantic-settings` from 2.15.0 to 2.16.0 - [Release notes](https://github.com/pydantic/pydantic-settings/releases) - [Commits](pydantic/pydantic-settings@v2.15.0...v2.16.0) Updates `uvloop` from 0.22.1 to 0.23.0 - [Release notes](https://github.com/MagicStack/uvloop/releases) - [Commits](MagicStack/uvloop@v0.22.1...v0.23.0) Updates `orjson` from 3.12.0 to 3.13.0 - [Release notes](https://github.com/ijl/orjson/releases) - [Changelog](https://github.com/ijl/orjson/blob/master/CHANGELOG.md) - [Commits](ijl/orjson@3.12.0...3.13.0) Updates `moto[s3]` to 5.2.4 - [Release notes](https://github.com/getmoto/moto/releases) - [Changelog](https://github.com/getmoto/moto/blob/master/CHANGELOG.md) - [Commits](getmoto/moto@5.2.3...5.2.4) Updates `ruff` from 0.16.5 to 0.17.1 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.16.5...0.17.1) Updates `mypy` from 2.3.1 to 2.4.0 - [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md) - [Commits](python/mypy@v2.3.1...v2.4.0) Updates `boto3-stubs[s3]` to 1.43.114 - [Release notes](https://github.com/youtype/mypy_boto3_builder/releases) - [Commits](https://github.com/youtype/mypy_boto3_builder/commits) Updates `fakeredis` from 2.37.1 to 2.40.0 - [Release notes](https://github.com/cunla/fakeredis-py/releases) - [Commits](cunla/fakeredis-py@v2.37.1...v2.40.0) --- updated-dependencies: - dependency-name: fastapi dependency-version: 0.146.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: uvicorn[standard] dependency-version: 0.54.0 dependency-type: direct:production dependency-group: all-dependencies - dependency-name: cryptography dependency-version: 50.0.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all-dependencies - dependency-name: pydantic dependency-version: 2.14.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: pydantic-settings dependency-version: 2.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: uvloop dependency-version: 0.23.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: orjson dependency-version: 3.13.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: moto[s3] dependency-version: 5.2.4 dependency-type: direct:production dependency-group: all-dependencies - dependency-name: ruff dependency-version: 0.17.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: mypy dependency-version: 2.4.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies - dependency-name: boto3-stubs[s3] dependency-version: 1.43.114 dependency-type: direct:production dependency-group: all-dependencies - dependency-name: fakeredis dependency-version: 2.40.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-dependencies ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.0.16 to 8.3.5. - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.3.5/packages/vite) --- updated-dependencies: - dependency-name: vite dependency-version: 8.3.5 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps the github-actions group with 1 update: [docker/login-action](https://github.com/docker/login-action). Updates `docker/login-action` from 4.6.0 to 4.7.0 - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@v4.6.0...v4.7.0) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: 4.7.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Multipart uploads no longer fail with InvalidPart when a short part reaches the proxy before a larger, lower-numbered part. Each client part now gets its own internal part-number range, kept in client-part order whatever order parts arrive in, so the assembled object can be decrypted. The branch also carries six Dependabot dependency bumps, which are independent of the fix.
Changes
Review focus
Check the placement logic in
s3proxy/state/allocation.py, especially the reservation made when the first part to arrive is a short last part. The dependency commits can be dropped without affecting the fix.Not included
#169 (@sveltejs/kit 2 to 3 and adapter-static 3 to 4) stays open. As shipped it is a breaking major:
npm cifails with ERESOLVE because kit 3 needs typescript ^6, and after bumping typescriptvite buildfails becausesvelte.config.jsis no longer supported, tsconfig must extend$app/tsconfig, and the$libalias was removed. That is a frontend migration and belongs in its own PR.