Skip to content

A Climate TRACE site can grow its waste with the city near it - #108

Merged
HughRunyan merged 4 commits into
mainfrom
site-waste-grows-with-its-nearest-city
Oct 29, 2026
Merged

HughRunyan merged 4 commits into
mainfrom
site-waste-grows-with-its-nearest-city

Conversation

@HughRunyan

@HughRunyan HughRunyan commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

A Climate TRACE site row can now carry city_population_by_year: the population, by year, of the UN-listed city near the site (UN World Urbanization Prospects 2025). When it does, SWEET grows the site's waste with that city's population instead of the country's.

The UN lists a city only once it passes 50,000 people, so many cities start after 1975, and a quarter of those near our sites start after 2000. SWEET fills the missing years like this:

  • A year the city has a number: that number.
  • Between two listed years: constant growth from one to the next. The supplement now lists every year a city was listed (RMI/waste_data_ingestion_pipeline#299), so this only bridges a gap if one appears.
  • Before the first listed year, or after the last: the city's nearest number, moved by the country's population. For example, a city first listed in 2005 at 50,000 is 50,000 × country(1990) / country(2005) in 1990.

Why the country's curve and not a fitted line: a straight line back from a fast-growing city reaches zero within a few decades, and its growth rate carried back shrinks it to almost nothing. The country's curve joins smoothly, never goes negative, and is what a site with no city gets anyway. A city usually grows faster than its country, so this slightly overstates a city's population before its first year.

A row without the column grows with the country, exactly as before. No WasteMAP path passes one.

  • population.parse_city_population: reads the mapping or its JSON text.
  • population.city_population_series: builds the yearly series.
  • city_params.import_basics_site (Climate TRACE path): uses it when the row has a city.

Climate TRACE decides which city a site gets (same country, within 50 km): RMI/RMI_Climate_TRACE_Waste_Methane#246.

Does this change a modelling decision?

  • No
  • Yes, and its entry is updated in this PR (Paths included). population-growth gains a un-city-series option, used on the ct-site path.

Do model numbers change?

  • No
  • Yes. The PR is labelled model-output-change, and the changelog entry says what moves. Only Climate TRACE sites whose row carries a city move. On the dev input that is 8,518 of 14,054 modelled sites. Against the country curve, a site's 1970 waste relative to 2023 moves by a median 0.97x (10th–90th percentile 0.50x–1.28x).

Acceptance criteria

  • A site row with city_population_by_year grows with that city; a row without it grows with its country, unchanged.
  • The city's own numbers are kept exactly; gaps before, between and after them are filled as described.
  • The series never goes to zero or below.
  • SWEET_python/MODEL_DECISIONS.md and changelog/2026-10.md are updated.
  • Tests: tests/test_site_city_population.py (22). Full suite: 667 passed, 1 skipped.

Definition of done

  • Acceptance criteria met
  • Tests pass
  • Changelog updated
  • Reviewed and merged

Merged with main (2026-10-29)

The only conflict was changelog/2026-10.md. The merge brings in #111–#113, which are unreleased, so the release that ships this also ships them. #113's new _site_intake_series grows a site's averaged intake with population_series, which is the city's series when the row carries one, so the two work together. 731 fast tests pass.

Nothing changed here for the new upstream format: the supplement's city_population is a mapping of year text to people, which parse_city_population already reads.

🤖 Generated with Claude Code

A site row may carry city_population: the nearby city's UN World Urbanization Prospects
2025 population by year. SWEET grows the site's waste with that series instead of the
country's. Before the city's first year and after its last, the city's nearest number
moves with the country's population; between two reported years growth is constant.
A row without the column grows with the country, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HughRunyan HughRunyan added enhancement New feature or request model-output-change Changes model OUTPUT values (expected progress, not breaking); results differ from prior runs labels Oct 26, 2026
Copilot AI lite review requested due to automatic review settings October 26, 2026 20:35
@HughRunyan HughRunyan added the model-output-change Changes model OUTPUT values (expected progress, not breaking); results differ from prior runs label Oct 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes modelled waste for about 8,500 Climate TRACE sites, and three moderate issues (a pd.NA crash, a silent flat-line on gappy country data, and a reused city_population key) still need fixing.

3 open issues
What changed in this PR

This PR lets a Climate TRACE site grow its waste with the UN-listed city near it, instead of its whole country. A site row can carry city_population, a mapping (or JSON text) of the city's population by year. Where the city has a number, SWEET uses it. In gaps between reported years, it grows at a constant rate. Before the first or after the last reported year, it scales the city's nearest number by the country's population curve. Sites without a city keep the country curve, so WasteMAP paths are unchanged.

Changes:

  • Adds parse_city_population and city_population_series to population.py, which read the city's data and fill its yearly gaps.
  • Uses the city series in import_basics_site when a Climate TRACE row has a city.
  • Updates the population-growth decision entry, the changelog, and adds 16 tests.
File Description
tests/​test_site_city_population.py New tests for parsing, gap filling, and a site growing with its city or country.
SWEET_python/​population.py Adds the city parser and the yearly series builder.
SWEET_python/​MODEL_DECISIONS.md Updates the population-growth entry with the un-city-series option and the ct-site path.
SWEET_python/​city_params.py Uses the city series in import_basics_site when a Climate TRACE row has a city.
changelog/​2026-10.md Records the model-output change for Climate TRACE sites with a city.

Review effort: Lite

Comment thread SWEET_python/city_params.py Resolved
Comment thread SWEET_python/population.py Resolved
Comment thread SWEET_python/population.py Resolved
HughRunyan and others added 2 commits October 26, 2026 16:57
…se a gappy country

- The site-row key is now city_population_by_year. city_population already names the
  Brazil city-site path's one number per city, on a different frame.
- parse_city_population treats any scalar missing value (pd.NA, NaT) as no city.
- city_population_series returns None when the country series has a gap or a
  non-positive year, so the site falls back to the country instead of a NaN end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-its-nearest-city

# Conflicts:
#	changelog/2026-10.md
Copilot AI lite review requested due to automatic review settings October 29, 2026 20:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical active defect leaves single-row sites using the frozen CAGR for landfill waste because the selected population series is not passed into the waste-generation frame.

1 open issue
3 resolved since last review

Review effort: Lite

Comment thread SWEET_python/city_params.py Resolved
…ear data

Ingestion #299 lists every year from a city's first to 2050, not every fifth
year, so log-linear growth now only bridges a gap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 29, 2026 22:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain, and the changes are supported by focused tests and documentation covering city-population parsing, growth application, fallback behavior, and Climate TRACE integration.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@HughRunyan
HughRunyan merged commit 9dbebe9 into main Oct 29, 2026
3 checks passed
@HughRunyan
HughRunyan deleted the site-waste-grows-with-its-nearest-city branch October 29, 2026 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request model-output-change Changes model OUTPUT values (expected progress, not breaking); results differ from prior runs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants