Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4810 +/- ##
==========================================
+ Coverage 67.72% 67.74% +0.01%
==========================================
Files 255 256 +1
Lines 33711 33750 +39
==========================================
+ Hits 22832 22864 +32
- Misses 10878 10885 +7
Partials 1 1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Replaces pbio.NewDelimitedReader with a reader using the same wire format and error semantics that grows its buffer as bytes arrive instead of allocating the declared length up front, so a peer declaring a large frame but sending little cannot pin a large buffer. Applies the same to dkg/sync readSizedProto, which now also reads the whole frame instead of failing on a short read. Per-protocol read limits and the 128MB maximum are unchanged. Parsigex now rejects partial signature sets larger than the cluster's validator count before decoding them, since each entry is unmarshalled before any of it is authenticated. Adds a fuzz test comparing the reader against pbio and a benchmark. category: bug ticket: none Co-authored-by: Andrei Smirnov <6468078+pinebit@users.noreply.github.com>
OisinKyne
force-pushed
the
oisin/p2p-incremental-read
branch
from
November 4, 2026 01:40
7f7d62d to
f9e5494
Compare
|
OisinKyne
marked this pull request as ready for review
November 4, 2026 01:43
OisinKyne
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hardens the handling of messages received from peers, so a peer cannot make a node allocate or decode far more than it actually sends.
pbio.NewDelimitedReaderallocates the full declared length (up to 128MB) before reading the body. The newp2preader keeps pbio's wire format and errors (io.ErrShortBufferover the limit,io.EOF/io.ErrUnexpectedEOFon truncation, unwrapped stream errors soIsRelayErrorstill matches), but starts with at most 64KB and doubles only as bytes arrive.dkg/sync'sreadSizedProtouses the same helper. It also previously did a singleReadand failed if the message arrived in several reads; it now reads the full frame. This makes the "Allocation size of p2p messages" entry in KNOWN_ISSUES.md accurate.ParSignedDataSetFromProtoSSZ/JSON-decodes each entry (taken from core/parsigex: bound work triggered by received messages #4637).Max message sizes are unchanged: the 128MB default, every
WithReadLimit(consensus, peerinfo, payloadex) and dkg/sync's 32MB.Testing:
FuzzDelimitedReaderchecks that the reader returns the same messages and errors as pbio on arbitrary input, including the max-size boundary, oversized declared lengths, truncated bodies and overlong varints.BenchmarkDelimitedReadershows the same cost up to 4KB, and up to ~2x time and allocation for frames above 64KB from the buffer doubling (about 0.6ms per extra MB).From #4637, the 32MB parsigex read limit was dropped, since it changes a limit and its rationale (up-front allocation) is removed here. The expired-duty check was also dropped, since replaying current duties costs the same. #4639 (resource manager) is out of scope.
category: bug
ticket: none