Skip to content

test: overhaul test infrastructure for clientless pytest-databases - #859

Draft
cofin wants to merge 20 commits into
mainfrom
pytest_databases_clientless_research
Draft

cofin wants to merge 20 commits into
mainfrom
pytest_databases_clientless_research

Conversation

@cofin

@cofin cofin commented Oct 26, 2026

Copy link
Copy Markdown
Member

Summary

Overhauls SQLSpec's unit and integration test infrastructure on top of the clientless pytest-databases branch (test/clientless-all), decoupling root pytest runs from container and optional database SDK imports, sharding integration CI across parallel service groups for Python 3.11–3.15, and adding live containerized coverage for Oracle Autonomous Database mTLS wallets, multi-cloud object storage (gs://, s3://, az://), AlloyDB Omni 18, MariaDB 11.8, and pure-Go enterprise ADBC drivers.

Key Changes

Plugin Isolation & Clientless Fixtures (tests/conftest.py, tests/integration/conftest.py, tests/integration/fixtures/)

  • Removed global pytest_plugins from root tests/conftest.py so tests/unit and tests/typing execute without importing pytest_databases.docker.* or optional database SDKs, enforced by test_unit_pytest_session_does_not_load_docker_or_integration_plugins.
  • Updated should_run_optional_cloud_suite in tests/conftest.py to accept either CLI opt-in flags (--run-bigquery-tests, --run-spanner-tests) or environment variables (SQLSPEC_ENABLE_BIGQUERY_TESTS=1, SQLSPEC_ENABLE_SPANNER_TESTS=1), while running automatically when CI=true.
  • Added dynamic _FIXTURE_DRIVER_REQUIREMENTS plugin discovery in tests/integration/conftest.py and guarded optional driver imports across sqlite.py, adapter_cases.py, and tests/integration/adapters/sqlite/sqlite/ so partial-extra runs load only installed drivers.
  • Replaced custom tests/plugins/db2.py (and types-docker), custom gizmosql_sqlite_service, dead asyncpg_adk.py, eager tests/integration/fixtures/__init__.py re-exports, and deprecated Spanner shims (spanner_connection, spanner_service.credentials) with clientless pytest_databases.docker.* services (tests/integration/fixtures/db2.py, cockroach.py, spanner.py) and in-container admin commands (exec_psql, sqlplus -L -S, sqlcmd -C).
  • Consolidated triplicated MySQL ADK schema/precision/foreign-key fixtures and tests into MysqlAdkResidualContext (tests/integration/fixtures/mysql.py) and tests/integration/adapters/mysql/test_adk_residuals.py covering aiomysql, asyncmy, mysqlconnector (sync and async), and pymysql with pool teardown cleanup.
  • Replaced s3fs-coupled RustFS bucket creation in tests/fixtures/rustfs.py with stdlib AWS SigV4 HTTP PUT provisioning (ensure_s3_bucket / ensure_rustfs_bucket).

Parallel Execution & Sharded CI (tests/integration/adapters/conftest.py, .github/workflows/ci.yml, Makefile)

  • Implemented ancestor-safe xdist_group normalization (@pytest.hookimpl(tryfirst=True)) in tests/integration/adapters/conftest.py, preserving dedicated secondary-container markers (alloydb, gizmosql_sqlite, mariadb, oracle_18c, oracle_adb, paradedb, pgvector, snowflake) and routing oracle, db2, and snowflake ADBC cases to their respective database families.
  • Relocated CockroachDB retry tests from postgres/test_cockroach_retry.py to tests/integration/adapters/cockroach/test_retry.py under xdist_group("cockroachdb").
  • Partitioned test-integration in .github/workflows/ci.yml across Python 3.11–3.15 and parallel service-group shards (postgres-cockroach, mysql-mariadb-embedded, oracle, mssql-db2, cloud-adbc-storage) with shard-scoped Docker image caches and matching make test-integration-<shard> targets in Makefile.

Containerized Adapter & Multi-Cloud Storage Suites (pyproject.toml, tests/integration/)

  • Pinned pytest-databases>=0.19.0 to litestar-org/pytest-databases@test/clientless-all in pyproject.toml ([tool.uv.sources]) and added gcsfs, adlfs, adbc-driver-oracle, adbc-driver-db2, adbc-driver-gizmosql, and adbc-driver-snowflake to [dependency-groups].extras.
  • Added live Oracle Autonomous Database Free 23ai (oracle_adb_service) Thin-mode mTLS wallet_zip (ewallet.p12 to ewallet.pem), tnsnames.ora alias resolution, and DBMS_CLOUD tests in tests/integration/adapters/oracle/oracledb/test_adb_wallet.py.
  • Added multi-cloud object storage suites across floci_gcp_service (gs://), floci_aws_service (s3://), and azure_blob_service (az://) for ObStoreBackend and FSSpecBackend (tests/integration/storage/test_cloud_backends.py), Arrow/object streaming (tests/integration/storage/test_streaming.py), and GCS-backed Spanner ADK artifact stores (tests/integration/adapters/spanner/spanner/extensions/adk/test_adk_artifact_store.py).
  • Added live AlloyDB Omni 18 columnar GUC tests (tests/integration/adapters/postgres/test_alloydb_omni.py), MariaDB 11.8 compatibility tests (tests/integration/adapters/mysql/test_mariadb.py), and containerized pure-Go ADBC suites for Oracle (oracle/adbc/test_adbc_oracle.py), IBM Db2 (db2/adbc/test_adbc_db2.py), and Snowflake/Embucket (snowflake/adbc/test_adbc_snowflake.py).

Production Adapter Fixes (sqlspec/adapters/asyncpg/, sqlspec/adapters/psycopg/, sqlspec/adapters/psqlpy/, sqlspec/adapters/adbc/)

  • Updated enable_alloydb_columnar across asyncpg, psycopg, and psqlpy to configure the session-settable google_columnar_engine.enable_columnar_scan GUC (rather than POSTMASTER-scoped google_columnar_engine.enabled), aligned pooler_mode="auto" startup GUC behavior, validated invalid pooler_mode values with ImproperConfigurationError, and removed unsupported max_cached_statement_lifetime from PsqlpyPoolParams.
  • Normalized IBM Db2 CLI keyword DSN strings (DATABASE=...;HOSTNAME=...;PORT=...;UID=...;PWD=...;) to db2:// URIs in sqlspec/adapters/adbc/core.py and routed parameterized non-row-returning Db2 statements through Cursor.executemany (AdbcStatementExecuteUpdate) in sqlspec/adapters/adbc/driver.py while preserving Arrow C-stream ingestion.

Documentation & Contributor Guides (docs/, tests/integration/adapters/README.md)

  • Updated docs/usage/testing.rst and docs/examples/patterns/integration_testing.py for bare clientless pytest-databases, Docker and Podman runtime discovery (--container-runtime), --keep-containers, shared-container pytest-xdist --dist=loadgroup, and the database and cloud-storage service catalog.
  • Synchronized docs/contributing/creating_adapters.rst, docs/reference/adapters/db2.rst, docs/reference/adapters/oracledb.rst, and tests/integration/adapters/README.md with _FIXTURE_DRIVER_REQUIREMENTS, preserved secondary-container xdist_group markers, or cloud opt-in flags, and per-shard Makefile targets.

cofin added 18 commits October 25, 2026 15:49
@codecov-commenter

codecov-commenter commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.38144% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.38%. Comparing base (4eae3fc) to head (299f9e6).

Files with missing lines Patch % Lines
sqlspec/adapters/adbc/core.py 80.76% 5 Missing and 10 partials ⚠️
sqlspec/adapters/adbc/driver.py 66.66% 8 Missing and 4 partials ⚠️
sqlspec/adapters/oracledb/core.py 75.55% 7 Missing and 4 partials ⚠️
sqlspec/adapters/oracledb/config.py 87.50% 0 Missing and 1 partial ⚠️
sqlspec/adapters/psqlpy/config.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #859      +/-   ##
==========================================
+ Coverage   81.30%   81.38%   +0.07%     
==========================================
  Files         527      527              
  Lines       88851    89016     +165     
  Branches    16576    16620      +44     
==========================================
+ Hits        72244    72447     +203     
+ Misses      12159    12122      -37     
+ Partials     4448     4447       -1     
Flag Coverage Δ
cloud-adbc-storage 32.53% <14.94%> (?)
integration 60.32% <55.67%> (+0.63%) ⬆️
mssql-db2 34.14% <27.31%> (?)
mysql-mariadb-embedded 40.43% <11.34%> (?)
oracle 33.21% <22.68%> (?)
postgres-cockroach 43.00% <23.19%> (?)
py3.11 79.79% <78.35%> (+0.06%) ⬆️
py3.12 79.79% <78.35%> (+0.04%) ⬆️
py3.13 79.79% <77.83%> (+0.05%) ⬆️
py3.14 80.73% <77.83%> (+0.05%) ⬆️
unit 73.44% <76.80%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sqlspec/adapters/asyncpg/config.py 89.94% <100.00%> (+0.02%) ⬆️
sqlspec/adapters/asyncpg/driver.py 89.88% <100.00%> (ø)
sqlspec/adapters/psqlpy/driver.py 85.78% <100.00%> (ø)
sqlspec/adapters/psycopg/config.py 89.98% <100.00%> (+0.37%) ⬆️
sqlspec/adapters/psycopg/core.py 95.20% <100.00%> (+0.12%) ⬆️
sqlspec/adapters/psycopg/driver.py 84.53% <100.00%> (-0.47%) ⬇️
sqlspec/adapters/oracledb/config.py 90.90% <87.50%> (+0.15%) ⬆️
sqlspec/adapters/psqlpy/config.py 84.83% <75.00%> (+0.09%) ⬆️
sqlspec/adapters/oracledb/core.py 74.46% <75.55%> (+0.58%) ⬆️
sqlspec/adapters/adbc/driver.py 78.99% <66.66%> (+0.39%) ⬆️
... and 1 more

... and 14 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants