executor: skip probe for empty hash joins - #11001
Conversation
ti-chi-bot
Bot
added
the
release-note-none
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
the
size/XL
📝 WalkthroughWalkthroughEmpty-build hash joins now publish probe-skip state, bypass probe processing for eligible join kinds, and prevent pipeline sources from running when transforms can finish without input. Tests cover empty and null-key builds, spilled joins, and anti-semi behavior. ChangesEmpty-build probe skipping
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant BuildSide
participant Join
participant ProbeTransform
participant PipelineExec
participant ProbeSource
BuildSide->>Join: publish empty build state
ProbeTransform->>Join: request shouldSkipProbe()
Join-->>ProbeTransform: return skip decision
PipelineExec->>ProbeTransform: request shouldSkipSource()
ProbeTransform-->>PipelineExec: return true
PipelineExec-->>ProbeSource: skip source lifecycle
ProbeTransform->>Join: finalize skipped probe
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
dbms/src/Interpreters/Join.h (1)
250-258: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse camelCase for newly introduced member names.
The new state variables use snake_case, contrary to the C++ naming guideline. Rename them consistently and update all references:
dbms/src/Interpreters/Join.h#L250-L258,449-449: renamebuild_side_emptytobuildSideEmpty.dbms/src/Interpreters/JoinV2/HashJoin.h#L77-L83,156-156: renamebuild_side_emptytobuildSideEmpty.dbms/src/Interpreters/JoinV2/HashJoin.cpp#L468-L468: update the state publication reference.dbms/src/Flash/Pipeline/Exec/PipelineExec.h#L83-L83: renamesource_prefix_executedtosourcePrefixExecuted.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@dbms/src/Interpreters/Join.h` around lines 250 - 258, Rename the newly introduced state members to camelCase and update every reference: in dbms/src/Interpreters/Join.h lines 250-258 and 449-449, rename build_side_empty to buildSideEmpty; in dbms/src/Interpreters/JoinV2/HashJoin.h lines 77-83 and 156-156, rename build_side_empty to buildSideEmpty; in dbms/src/Interpreters/JoinV2/HashJoin.cpp line 468, update the state publication reference; and in dbms/src/Flash/Pipeline/Exec/PipelineExec.h line 83, rename source_prefix_executed to sourcePrefixExecuted.Source: Coding guidelines
dbms/src/Flash/tests/gtest_join_executor.cpp (1)
443-469: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winConsider adding empty-build coverage for Cross-join and NullAware semi/anti kinds.
Current new tests cover Inner/Semi/RightSemi (skip) and Anti (no-skip) for the standard equi hash join, but there's no test for
Cross_LeftOuterAnti/Cross_LeftOuterorNullAware_*kinds with an empty build side. Given the skip logic inJoin::joinBlock(Join.cpp Lines 1990-1994) runs before the kind-specific dispatch, a test confirming these families are not incorrectly skipped would close the verification gap raised on theJoin.cppchange.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@dbms/src/Flash/tests/gtest_join_executor.cpp` around lines 443 - 469, Add empty-build coverage alongside EmptyBuildAntiSemiJoinStillReadsProbe for Cross_LeftOuterAnti, Cross_LeftOuter, and NullAware semi/anti join kinds. Build requests with an empty build side and non-empty probe side, then verify execution is not skipped and produces the expected probe-side results, covering the pre-dispatch skip logic in Join::joinBlock.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@dbms/src/Flash/tests/gtest_join_executor.cpp`:
- Around line 443-469: Add empty-build coverage alongside
EmptyBuildAntiSemiJoinStillReadsProbe for Cross_LeftOuterAnti, Cross_LeftOuter,
and NullAware semi/anti join kinds. Build requests with an empty build side and
non-empty probe side, then verify execution is not skipped and produces the
expected probe-side results, covering the pre-dispatch skip logic in
Join::joinBlock.
In `@dbms/src/Interpreters/Join.h`:
- Around line 250-258: Rename the newly introduced state members to camelCase
and update every reference: in dbms/src/Interpreters/Join.h lines 250-258 and
449-449, rename build_side_empty to buildSideEmpty; in
dbms/src/Interpreters/JoinV2/HashJoin.h lines 77-83 and 156-156, rename
build_side_empty to buildSideEmpty; in dbms/src/Interpreters/JoinV2/HashJoin.cpp
line 468, update the state publication reference; and in
dbms/src/Flash/Pipeline/Exec/PipelineExec.h line 83, rename
source_prefix_executed to sourcePrefixExecuted.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 33aad696-443c-48c3-b6c6-f48d79afe38e
📒 Files selected for processing (15)
dbms/src/DataStreams/HashJoinProbeExec.cppdbms/src/Flash/Pipeline/Exec/PipelineExec.cppdbms/src/Flash/Pipeline/Exec/PipelineExec.hdbms/src/Flash/Pipeline/Exec/tests/gtest_simple_operator.cppdbms/src/Flash/tests/gtest_join_executor.cppdbms/src/Interpreters/Join.cppdbms/src/Interpreters/Join.hdbms/src/Interpreters/JoinV2/HashJoin.cppdbms/src/Interpreters/JoinV2/HashJoin.hdbms/src/Operators/HashJoinProbeTransformOp.cppdbms/src/Operators/HashJoinProbeTransformOp.hdbms/src/Operators/HashJoinV2ProbeTransformOp.cppdbms/src/Operators/HashJoinV2ProbeTransformOp.hdbms/src/Operators/HashProbeTransformExec.hdbms/src/Operators/Operator.h
|
/retest |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@dbms/src/Flash/tests/gtest_spill_join.cpp`:
- Around line 269-271: Update the test around executeStreams in the spill join
case to verify V1 spill fallback behavior, not just the empty result. Add
assertions that external spilling occurred and that the probe scan or probe
executor was invoked, using the existing test instrumentation and counters,
while preserving the current result assertion.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 871d801b-9eaa-458d-ac64-d4c337559ae7
📒 Files selected for processing (2)
dbms/src/Flash/tests/gtest_join_executor.cppdbms/src/Flash/tests/gtest_spill_join.cpp
| WRAP_FOR_SPILL_TEST_BEGIN | ||
| ASSERT_COLUMNS_EQ_UR(empty_result, executeStreams(request, 10)); | ||
| WRAP_FOR_SPILL_TEST_END |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Assert the spill fallback behavior.
Line 270 checks only the result rows. Both probe execution and incorrect probe skipping produce an empty result for this join. Assert that external spill occurred and that the probe scan or probe executor ran. This makes the V1 spill fallback observable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@dbms/src/Flash/tests/gtest_spill_join.cpp` around lines 269 - 271, Update the
test around executeStreams in the spill join case to verify V1 spill fallback
behavior, not just the empty result. Add assertions that external spilling
occurred and that the probe scan or probe executor was invoked, using the
existing test instrumentation and counters, while preserving the current result
assertion.
|
/retest |
2 similar comments
|
/retest |
|
/retest |
1 participant
What problem does this PR solve?
Issue Number: close #11000
Problem Summary
When the build side of a hash join is empty after filtering, the join result may already be known without reading the probe side. However, the current execution pipeline can still scan the entire probe side and perform unnecessary hash lookups against an empty hash table.
For example:
The filter produces an empty build side, but the profile shows:
This causes unnecessary probe-side I/O, block processing, pipeline scheduling, and hash-table lookup overhead. The same problem also applies to eligible Semi Join cases.
What is changed and how it works
This change allows supported hash joins to terminate the probe side early when the finalized build hash table is empty.
The implementation has two parts:
Track whether the finalized build side is empty.
Skip probe processing at both the join and pipeline levels.
PipelineExecthen avoids reading the probe source and finishes the transform normally, without relying on source EOF or repeatedly producing empty blocks.The optimization is enabled for:
RightSemi Join is not currently supported by Hash Join V2 and therefore falls back to the V1 implementation.
Join types whose output still depends on every probe row, such as AntiSemi Join and LeftOuterSemi Join, are not included in this optimization. V1 also keeps the existing behavior when an actual external join spill has occurred.
Check List
Tests
Side effects
Documentation
Release note
Summary by CodeRabbit
Performance Improvements
Bug Fixes
Tests