Terraform for the Google Cloud infrastructure of the Development Finance Observatory (DFO). It defines the Cloud Run services, databases, storage, networking and access rules behind the platform. Each environment and stack is its own Terraform root with its own state.
The DFO is an initiative of ONE Data, built on Google Data Commons. This repository is where the other DFO repositories get somewhere to run.
This repository is under active development. Interfaces, schemas, and data may change without notice.
infra/
├── modules/ # reusable modules shared by the stacks
├── stacks/ # Terraform source for each stack, shared by both environments
├── dfo-dev/ # dev environment: one Terraform root (own state) per stack
│ ├── dcp/
│ ├── foundation/
│ ├── ai/
│ ├── apis/
│ ├── pipelines/
│ ├── platform/
│ └── website/
└── dfo-prod/ # prod environment: the same stacks except pipelines, plus the public edge and telemetry
├── edge/
└── telemetry/
scripts/ # helpers for CI, GitHub setup and the dcp stack
justfile # task runner recipes (run `just` to list them)
A change to one stack appears only in that stack's plan. The stacks are:
- dcp: the Data Commons Platform. The upstream stack module (services, Spanner, Redis, ingestion) plus DFO additions in
dfo.tf. Thedcprepository's CI builds images and opens an auto-merged pull request that pins them here. - edge: prod only. The load balancer and certificate for developmentfinance.org, the Cloud Armor policy that admits only Cloudflare, and the Cloudflare zone's DNS and rules.
- foundation: shared plumbing. Artifact Registry, Workload Identity Federation, and the deployer service accounts.
- ai: the AI service (agent harness and MCP servers). The
airepository's CI deploys images onto it. - apis: the Search and Datasets APIs on one Cloud Run service, with the storage, database and private-network access the Datasets API needs. The
apisrepository's CI deploys images onto it. - pipelines: storage for the data pipelines, which the
pipelinesrepository reads and writes. - platform: the shared Cloud SQL Postgres instance, reached over Private Service Connect. Each tenant's database and users live in the tenant's own stack.
- telemetry: prod only. Cloud Run logs routed to a Log Analytics bucket, the daily
logs_dailyaggregate, and the BigQuery datasets the team dashboard reads (dfo_reporting,searchconsole). - website: the DFO website's Cloud Run service, plus its private network path to the shared Postgres instance. The
websiterepository's CI deploys images onto it.
Each stack's README says what belongs in it.
Planning and applying need access to ONE's Google Cloud projects (dev-fin-observatory and prod-fin-observatory), which only the ONE Data team holds. Without it you can read the code and run the checks that need no credentials. You need just, Terraform 1.7 or later, and uv.
just install # install the pre-commit hooks
just fmt # terraform fmt across infra/
just check # run every pre-commit hook
cd infra/dfo-dev/dcp && terraform init -backend=false && terraform validateWith access, just auth signs in to gcloud, just bootstrap runs terraform init for the dcp stack, and just plan previews its changes. Select another stack or environment with STACK= and ENV=, for example just STACK=foundation plan.
Pull requests target dev. CI posts a terraform plan for each changed stack as a pull request comment, and merging to dev applies the change to the dev environment. A release is a pull request from dev into main, which deploys prod after a maintainer approves it. The full flow, the destroy guard and the break-glass path are in docs/development.md.
- docs/development.md: CI, releases, state, wiring a service repository, and operating the dcp stack.
- infra/dfo-dev/dcp/dfo.README.md: which dcp files the upstream CLI generates and which belong to DFO.
- infra/modules: the shared modules.
Issues and pull requests are welcome. See CONTRIBUTING.md.
MIT. See LICENSE.