Skip to content

Adapt PropertyGraph graph.yaml schema and DDL compiler to Semantic Ontology V1 - #2417

Merged
rafal-hawrylak merged 9 commits into
mainfrom
feat/property-graph-semantic-ontology
Nov 6, 2026
Merged

rafal-hawrylak merged 9 commits into
mainfrom
feat/property-graph-semantic-ontology

Conversation

@rafal-hawrylak

@rafal-hawrylak rafal-hawrylak commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Updates definitions/graph.yaml (PropertyGraphConfig) and the BigQuery CREATE OR REPLACE PROPERTY GRAPH DDL compiler to the Semantic Ontology V1 schema while keeping the compiled dataform.PropertyGraph proto unchanged.

Key Changes

  • Model wrapper and deployment target: Requires top-level version: 0.2.0.dev0/google and a single-entry semantic_model list in V1. Supports optional deployments with a BigQuery property graph resource name (bigquery.googleapis.com/projects/<project>/datasets/<dataset>/propertyGraphs/<graph>, without leading //) and falls back to the project's default database and schema when omitted.
  • Entities (datasets alias) and table sources: Maps semantic_model[].entities (or datasets as an alias) to NODE TABLES. Supports source via Dataform ref ({ ref: "<name>" } or { ref: { name, schema, database, include_dependent_assertions } }), "bigquery:project.dataset.table", "project.dataset.table" and BigQuery table resource names (bigquery.googleapis.com/projects/<project>/datasets/<dataset>/tables/<table>, without leading //). Validates physical column names in primary_key and rejects unique_keys in V1.
  • Relationships (1:N FK and M:N bridge tables): Maps semantic_model[].relationships to EDGE TABLES with NO PROPERTIES. Relationships without an explicit source inherit the from entity's backing table and primary_key as the edge KEY and SOURCE KEY. Relationships with an explicit source require primary_key and to_columns.
  • Derived fields and ai_context: Passes field expression as-is into PROPERTIES (...) (defaulting omitted expression to field.name), ignores informational field metadata (datatype, dimension and label) in DDL and emits PROPERTIES ALL COLUMNS when an entity has no explicit fields or metrics. Maps ai_context.synonyms and ai_context.examples into DEFAULT LABEL OPTIONS(...) before PROPERTIES and NO PROPERTIES.
  • Top-level metrics (MEASURE(...)): Compiles semantic_model[].metrics into MEASURE(<expression>) AS <name> on the target entity's default label, passing metric.expression as-is into MEASURE(...).
  • Packaging and tests: Bundles property_graph_config.proto into @dataform/core and updates unit, VM compilation and BigQuery integration tests.

Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph.ts Resolved
Comment thread core/actions/property_graph.ts Resolved
Comment thread core/main.ts Resolved
Comment thread protos/property_graph_config.proto Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
@rafal-hawrylak
rafal-hawrylak force-pushed the feat/property-graph-semantic-ontology branch from 568dac5 to 78623ab Compare November 4, 2026 08:56
@rafal-hawrylak
rafal-hawrylak force-pushed the feat/property-graph-semantic-ontology branch from 78623ab to bcee35f Compare November 4, 2026 09:25
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
@Ford544

Ford544 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Not this PR's fault, but we appear to be missing Property Graphs from the docs (https://dataform-co.github.io/dataform/docs/reference/configs). Might be a good opportunity to introduce them.

Comment thread core/actions/property_graph_utils.ts Resolved
@rafal-hawrylak
rafal-hawrylak force-pushed the feat/property-graph-semantic-ontology branch from bcee35f to a2b6564 Compare November 4, 2026 12:05
@rafal-hawrylak
rafal-hawrylak force-pushed the feat/property-graph-semantic-ontology branch from a2b6564 to 4dbd752 Compare November 4, 2026 12:22
@rafal-hawrylak

Copy link
Copy Markdown
Collaborator Author

Not this PR's fault, but we appear to be missing Property Graphs from the docs (https://dataform-co.github.io/dataform/docs/reference/configs). Might be a good opportunity to introduce them.

That's correct, I planned to add it soon. This PR is big already so I'm going to have a follow up PR.

Comment thread protos/property_graph_config.proto Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph.ts Resolved
Comment thread core/actions/property_graph.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
Comment thread core/actions/property_graph_utils.ts Resolved
@rafal-hawrylak
rafal-hawrylak merged commit f1a6ef0 into main Nov 6, 2026
15 checks passed
@rafal-hawrylak
rafal-hawrylak deleted the feat/property-graph-semantic-ontology branch November 6, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants