[common] Fix equals/hashCode contract in value and kv record batches - #3952
[common] Fix equals/hashCode contract in value and kv record batches#3952vbhanuchander-lang wants to merge 1 commit into
Conversation
DefaultValueRecordBatch and DefaultKvRecordBatch override equals() with value semantics but do not override hashCode(). Both extend Object and neither equals() delegates to super.equals(), so hashCode() is the identity hash: two byte-identical batches are equal but hash differently, violating the general contract of Object.hashCode(). The third batch implementation in the same package, DefaultLogRecordBatch, has the identical equals() shape and already hashes the same byte range. Mirror it in both classes so the three siblings stay consistent. This is not a fix for the flakiness in apache#3877; it only removes the contract violation observed while diagnosing it.
|
Could a committer approve the pending workflow runs? All three ( cc @loserwang1024 @luoyuxia — you are the most recent contributors to these files. The change is Details, including why nothing currently depends on the old identity hash, are in #3951. |
Purpose
Linked issue: close #3951
DefaultValueRecordBatchandDefaultKvRecordBatchoverrideequals()with valuesemantics but never override
hashCode(). Both classes onlyimplementstheir interface,so their superclass is
Object, and neitherequals()delegates tosuper.equals()—hashCode()is therefore the identity hash. Two byte-identical batches areequals()butreturn different hash codes, which violates the general contract of
Object.hashCode().The third batch implementation in the same package,
DefaultLogRecordBatch, has theidentical
equals()shape and does hash the same byte range(
DefaultLogRecordBatch.java:316),which is why this reads as an oversight rather than a deliberate choice. The Checkstyle
EqualsHashCodemodule is not enabled intools/maven/checkstyle.xml, so CI does not flagit today.
Brief change log
DefaultValueRecordBatch: addhashCode(), mirroringDefaultLogRecordBatchexactly.DefaultKvRecordBatch: same.Both use
MurmurHashUtils.hashBytes(segment, position, sizeInBytes()).equals()comparesthe byte range
[position, position + sizeInBytes())viasegment.equalTo(that.segment, position, that.position, sizeInBytes), andhashByteshashes exactly that range, so
equals()andhashCode()are consistent by construction.Tests
New in
DefaultValueRecordBatchTest(new file) andDefaultKvRecordBatchTest:testEqualsAndHashCodetestHashCodeDiffersForDifferentContentstestEmptyBatchesAreEqualAndShareHashCodeReplicaManagerTest#testLimitScanPrimaryKeyTableVerified locally with
mvn clean verify -pl fluss-common: 1748 + 186 tests pass, Checkstylereports 0 violations,
spotless:checkandapache-rat:checkpass.API and Format
No API or storage-format change.
hashCode()is additive.The only behavioural change is for a
HashSet/HashMapkeyed on one of these batches,which would previously have used the identity hash. Nothing does that today — the batches
appear only as map value types (e.g.
Map<TableBucket, KvRecordBatch>inServerRpcMessageUtils#getPutKvDataandReplicaManager#putRecordsToKv), wherehashCode()is never consulted.Documentation
No new feature, no documentation change.
Note on #3877
I found this while diagnosing the flaky
ReplicaManagerTest#testLimitScanPrimaryKeyTablein #3877. This PR does not fix that flakiness — why the two batches genuinely differ
there is a separate question. It does make the failure message more useful: neither class
overrides
toString(), soObject.toString()renders the identity hash, and the...@60eebc2evs...@1408822in that CI log cannot tell a reader whether the contentsactually differ (two equal batches would also print different suffixes). With a
content-derived
hashCode(), differing suffixes do mean differing bytes.Generative AI disclosure