fix(packaging): correct declared dependencies and publish the tests extra - #1277
Draft
ogenstad wants to merge 6 commits into
Draft
fix(packaging): correct declared dependencies and publish the tests extra#1277ogenstad wants to merge 6 commits into
ogenstad wants to merge 6 commits into
Conversation
…xtra The declared dependency surface had drifted from what the SDK actually needs. pydantic>=2.0.0 admitted 2.0 and 2.0.2, on which `import infrahub_sdk` raises SchemaError because their regex engine rejects the `\_` escape in the generated schema model patterns. anyio, typing-extensions and packaging were imported by the shipped package but only ever arrived as transitive dependencies of httpx, pydantic and the test tooling, so a constrained resolution could install the SDK unusable. The `tests` extra is described in the installation guide but was never published, so `pip install 'infrahub-sdk[tests]'` warned and installed nothing beyond the base package. It now exists and carries what `infrahub_sdk.testing` and the bundled pytest plugin import. `all` becomes self-referential so it cannot drift from the extras it aggregates, which is how it had already lost mdxify. numpy and mdxify were declared but never imported. pyarrow declares numpy itself on the releases that need it, and mdxify only builds the docs, so it moves to a docs dependency group. Each floor was established by installing the candidate version and exercising the import rather than read off release notes: anyio.Path appears in 3.3.0 rather than 3.0, and typing-extensions needs 4.4.0 for PEP 696 TypeVar defaults. New unit tests over the packaging metadata keep the declared and imported dependency sets in agreement.
The extra is newly installable, so readers will reach it for the first time. It pulls in the container tooling behind `infrahub_sdk.testing`, which is a lot more than the `ctl` extra beside it.
ogenstad
added
the
type/tech-debt
github-actions
Bot
added
the
type/documentation
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## infrahub-develop #1277 +/- ##
====================================================
+ Coverage 84.16% 84.21% +0.04%
====================================================
Files 147 148 +1
Lines 13047 13081 +34
Branches 1930 1936 +6
====================================================
+ Hits 10981 11016 +35
+ Misses 1503 1499 -4
- Partials 563 566 +3
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
1 issue found across 9 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="pyproject.toml">
<violation number="1" location="pyproject.toml:66">
P2: Custom agent: **Detect conflicting package versions across dependency files**
The new `tests` extra declares `pytest>=7.0`, but the same file's `tests` dependency group declares `pytest>=9.0,<9.1`. Align these pytest specifiers or otherwise avoid declaring conflicting versions in the same dependency metadata.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| tests = [ | ||
| "infrahub-testcontainers>=1.7.3", | ||
| "packaging>=21.0", | ||
| "pytest>=7.0", |
Contributor
There was a problem hiding this comment.
P2: Custom agent: Detect conflicting package versions across dependency files
The new tests extra declares pytest>=7.0, but the same file's tests dependency group declares pytest>=9.0,<9.1. Align these pytest specifiers or otherwise avoid declaring conflicting versions in the same dependency metadata.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pyproject.toml, line 66:
<comment>The new `tests` extra declares `pytest>=7.0`, but the same file's `tests` dependency group declares `pytest>=9.0,<9.1`. Align these pytest specifiers or otherwise avoid declaring conflicting versions in the same dependency metadata.</comment>
<file context>
@@ -43,30 +47,28 @@ infrahubctl = "infrahub_sdk.ctl.cli:app"
+tests = [
+ "infrahub-testcontainers>=1.7.3",
+ "packaging>=21.0",
+ "pytest>=7.0",
+]
+
</file context>
The import scan classified modules with the running interpreter's `sys.stdlib_module_names`. `tomllib` only joined the standard library in 3.11, so on 3.10 the guarded `import tomllib` in ctl/config.py looked like an undeclared third-party package and failed the check. Exempt it by name rather than skipping imports nested under a `sys.version_info` guard: the `tomli` backport sits in the same else branch and is a real declared dependency that must stay verified.
Deploying infrahub-sdk-python with
|
| Latest commit: |
ffb9351
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://b85de03f.infrahub-sdk-python.pages.dev |
| Branch Preview URL: | https://pog-packaging-metadata-tests.infrahub-sdk-python.pages.dev |
ogenstad
marked this pull request as draft
August 26, 2026 07:49
Jinja2, PyYAML and rich sat in the `ctl` extra, but `template/`, `spec/`, `transfer/` and `protocols_generator/` import them at module level. On a plain `pip install infrahub-sdk` those modules raised ModuleNotFoundError: 11 of them, including `infrahub_sdk.template`, which renders Transforms and is nothing to do with the CLI. rich is not merely presentation there either, its Traceback/Frame/Syntax types are carried in the Jinja error model. Moving the three to the core dependencies takes the non-ctl modules that fail to import on a base install from 20 to 9, and the remaining 9 legitimately need an extra: seven want pytest, and async_typer and graphql/plugin.py want the CLI dependencies. It also means the `tests` extra provides a working pytest environment, which it did not: the bundled plugin imports yaml and jinja2, so pytest could not start at all. A base install grows from 19 to 26 packages. ruamel.yaml stays in `ctl`, where its round-trip mode is only used to preserve comments in `schema format`.
The import check compared against the union of every extra, so a module in the base wheel could import a package only `ctl` installed and still pass. That is exactly how `infrahub_sdk.template` came to be broken on a plain install. Each shipped module is now checked against the requirements its own surface implies: base modules against the core dependencies, `pytest_plugin/` and `testing/` against the `tests` extra, and `ctl/` plus the two modules only reachable from it against `ctl`. Against the previous metadata this reports 36 violations. Only imports that run at load time count. An import inside a function is the sanctioned way to reach for an extra, as the JSON importer does for pyarrow, and flagging it would punish the correct pattern. Also collapse separator runs in the name normaliser so it matches PEP 503 as its docstring claims.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 7 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…ports Moving rich into the core dependencies also promoted its `<14` cap from CLI users to everyone, which would conflict with any project already on a newer rich. The cap has no recorded cause: it arrived with the UV conversion, a mechanical commit that postdates rich 14.0, and nothing needs it. The unit suite passes against rich 13.9.4, 14.0.0, 14.2.0 and 15.0.0, and `Traceback._guess_lexer`, the one private API in use, behaves the same on all four. The lock stays on 13.9.4; only the accepted range widens. The import scan also counted `if TYPE_CHECKING:` blocks as running at import time, because it unwound every module-level `if` without reading the guard. Nothing triggered it yet, but annotating a base module against a ctl-only package is precisely a type-checking import, so the check would have rejected the correct pattern. The `else` branch of such a guard is a runtime fallback and still counts. Import classification now has direct tests, since the distinction between import-time, deferred and type-checking-only is subtle enough to regress quietly. Reverting the guard fix fails two of them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The unit-test matrix runs five Python versions but every job installs exactly what
uv.lockpins, so the version ranges declared inpyproject.tomlare never exercised. Checking them by hand turned up several that were wrong, two of which break users outright.Goal: make the declared dependency surface match what the SDK actually needs, and add tests so it stays that way.
Non-goals: no CI jobs that resolve at the declared lower and upper bounds. That is the follow-up work, and it is what will genuinely test the ranges end to end. This PR only covers what can be checked without extra CI time.
Ref IHS-224 (part 1 of 3).
What changed
Behavioral changes:
pip install 'infrahub-sdk[tests]'works. The extra is documented in the installation guide but was never published, so the command warned that no such extra existed and installed nothing beyond the base package.SchemaErroronimport infrahub_sdk. Those versions reject the\_escape in the generated schema model patterns, so they never worked.anyio,typing-extensionsandpackagingare declared directly. All three were imported by the shipped package but only arrived as transitive dependencies, so a constrained resolution could install the SDK unusable.infrahub-sdk[ctl]no longer installsnumpyormdxify, neither of which the SDK imports.infrahub-sdk[all]now coversctlandteststogether, so it pulls in considerably more than before.Implementation notes:
anyio.Patharrives in 3.3.0 rather than 3.0, andtyping-extensionsneeds 4.4.0 for PEP 696TypeVardefaults.allis now["infrahub-sdk[ctl,tests]"]. Hand-duplicating the lists is how it had already lostmdxify, and a self-reference cannot drift.mdxifymoves to a newdocsdependency group, included indev, souv sync --all-groupsstill provides it forinvoke docs-generate.What stayed the same: no runtime code changed. This is packaging metadata, a regenerated lock, one docs note, and a new test module.
How to review
Start with the
pyproject.tomldiff, which is the whole substance of the change. Thentests/unit/test_packaging_metadata.py. Theuv.lockdiff is mechanical, and mostly shrinkage from droppingnumpyandmdxify.Two things worth extra scrutiny:
testsextra shares its name with thetestsdependency group while holding different contents. The name was chosen to match what the docs and README already tell people to type. uv handles the two namespaces without complaint, but the follow-up PR that splits the test groups is the natural place to rename the group.testsextra is heavy, adding roughly 66 packages including Docker, FastAPI, uvicorn and Prefect client libraries, all viainfrahub-testcontainers. Someone who only wants the pytest plugin gets all of it. Splitting the container-based helpers into their own extra is worth considering in the follow-up.How to test
The new tests are non-vacuous: pointing them at the previous
pyproject.tomlfails four of the five checks, each naming a real defect. Therequires-pythoncheck passes on the old metadata too, so it is a regression guard rather than a bug finder.Verified locally:
ruff,tyandmypyclean;docs-validateexits 0 with no committed docs changed;lint-docsbyte-identical to the base branch. The unit suite is 1840 passed with 2 failures, both of which reproduce identically in a pristine worktree of the base commit (macOS-only Rich wrapping of long/private/var/folders/...temp paths, in files this PR does not touch).Impact & rollout
numpy/mdxifyfrom the extras are resolver-visible narrowings. Nothing that worked before stops working, since the removed versions could not import the SDK and the removed packages were never used, but the install-time behavior changes. This targetsinfrahub-developdeliberately so it ships with the next Infrahub version rather than as a patch to the current SDK line.Checklist
Summary by cubic
Fixes the packaging metadata so a plain install imports everything the SDK ships with, and publishes the
testsextra. Old: a plain install failed importingtemplate/,spec/,transfer/andprotocols_generator/,pydantic2.0/2.0.2 crashed on import, andinfrahub-sdk[tests]did nothing; New: those dependencies ship with the SDK, the brokenpydanticversions fail during resolution, and thetestsextra installs. Ref IHS-224.Dependencies
template/,spec/,transfer/andprotocols_generator/, and drop therichupper bound.pydantic>=2.0.3(2.0 and 2.0.2 now fail during resolution).anyio>=3.3.0,typing-extensions>=4.4.0, andpackaging>=21.0as direct requirements.testsextra withinfrahub-testcontainers>=1.7.3,pytest>=7.0, andpackaging>=21.0.allaggregate the other extras viainfrahub-sdk[ctl,tests]to prevent drift.numpyandmdxifyfromctl; movemdxifyto adocsdependency group included indev.Migration
pip install infrahub-sdknow installs Jinja2, PyYAML and rich (19→26 packages).pydanticto 2.0.x, raise it to>=2.0.3.infrahub-sdk[ctl]to installnumpyormdxify, declare them in your project.infrahub-sdk[all]to install more; useinfrahub-sdk[ctl]if you don’t need the testing tools.Written for commit ffb9351. Summary will update on new commits.