fix verify-migration: scope checks to indirectly filtered objects - #73
Open
DmitriiAn wants to merge 2 commits into
Open
fix verify-migration: scope checks to indirectly filtered objects#73DmitriiAn wants to merge 2 commits into
DmitriiAn wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a migration ran with a
~/filters.ini, the source legitimately has moreobjects than the target.
verify-migration.shalready handled directexclusions, but not objects excluded indirectly — where the object's own
name matches nothing in filters.ini, yet pgcopydb could never have created it
because its parent was filtered out.
What changed
All new logic went into
filters-lib.shsopreflight-check.shbenefits too.Five clause helpers, all built from one internal out-of-scope predicate:
sequence_owner_clausefk_target_clauseview_dep_clauseextension_rel_clause/extension_oid_clause[exclude-extension]extension_clausewas previously applied only to the extensions check, soevery other check false-FAILed on extension-owned objects.
The columns check needed the view-dependency clause too —
information_schema.columnsreports view columns as well as table columns, sowithout it a dependency-excluded view reappears there as "missing columns".
This only shows up against a real database.
include-only-tablemode: pgcopydb copies the listed tables plus their ownindexes/constraints/sequences, but not the views, routines or standalone
sequences that merely share those schemas. Checks 6/7/9 now skip with the
reason in the section header instead of comparing.
Filter sanity:
filter_conflicts()existed in the library but verify nevercalled it — now it does. Sections outside the six that filters.ini documents
(
[exclude-table-data],[exclude-index], …) surface as a WARN rather thanbeing half-honoured.
pgcopydb-helpers/filters.iniis unchanged. The four helperspreflight-check.shconsumes keep their exact behaviour.Notes for the reviewer
README.mduntouched;AGENTS.mddocuments the dependency scoping and theinclude-only-tableskips.elsewithout re-indenting ~40 lines each, to keep the diff reviewable. Happy to indent if you'd rather.# shellcheck disabledirectives inverify-migration.shcover pre-existing findings (SC1090 onsource ~/.env, SC2034 on a positionally-readreadvariable), since the file is in scope for the clean bar.slack-migration-alerts.test.shstill fails for unrelated reasons (needs a live webhook) — untouched.