Skip to content

fix verify-migration: scope checks to indirectly filtered objects - #73

Open
DmitriiAn wants to merge 2 commits into
mainfrom
fix/verify-migration-filter-scope
Open

fix verify-migration: scope checks to indirectly filtered objects#73
DmitriiAn wants to merge 2 commits into
mainfrom
fix/verify-migration-filter-scope

Conversation

@DmitriiAn

Copy link
Copy Markdown
Contributor

Problem

When a migration ran with a ~/filters.ini, the source legitimately has more
objects than the target. verify-migration.sh already handled direct
exclusions, but not objects excluded indirectly — where the object's own
name matches nothing in filters.ini, yet pgcopydb could never have created it
because its parent was filtered out.

What changed

All new logic went into filters-lib.sh so preflight-check.sh benefits too.
Five clause helpers, all built from one internal out-of-scope predicate:

helper closes
sequence_owner_clause serial/identity sequence of an excluded table
fk_target_clause a kept table's FK pointing at an excluded table
view_dep_clause view on an excluded table, recursively (view-on-view)
extension_rel_clause / extension_oid_clause tables, columns, indexes, constraints, views, sequences and routines owned by an [exclude-extension]

extension_clause was previously applied only to the extensions check, so
every other check false-FAILed on extension-owned objects.

The columns check needed the view-dependency clause too
information_schema.columns reports view columns as well as table columns, so
without it a dependency-excluded view reappears there as "missing columns".
This only shows up against a real database.

include-only-table mode: pgcopydb copies the listed tables plus their own
indexes/constraints/sequences, but not the views, routines or standalone
sequences that merely share those schemas. Checks 6/7/9 now skip with the
reason in the section header instead of comparing.

Filter sanity: filter_conflicts() existed in the library but verify never
called it — now it does. Sections outside the six that filters.ini documents
([exclude-table-data], [exclude-index], …) surface as a WARN rather than
being half-honoured.

pgcopydb-helpers/filters.ini is unchanged. The four helpers
preflight-check.sh consumes keep their exact behaviour.

Notes for the reviewer

  • Nothing changes for users of the tool — this is a false-positive fix. README.md untouched; AGENTS.md documents the dependency scoping and the include-only-table skips.
  • Dependency exclusions are silent by design — no per-object "skipped" accounting in the output.
  • The wrapped bodies of checks 6/7/9 sit inside else without re-indenting ~40 lines each, to keep the diff reviewable. Happy to indent if you'd rather.
  • Two # shellcheck disable directives in verify-migration.sh cover pre-existing findings (SC1090 on source ~/.env, SC2034 on a positionally-read read variable), since the file is in scope for the clean bar.
  • slack-migration-alerts.test.sh still fails for unrelated reasons (needs a live webhook) — untouched.

@DmitriiAn
DmitriiAn requested a review from teknogeek0 August 11, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant