Skip to content

feat(python): six P1 tutorials — radar, network, t-SNE, paired, calendar, competing risk (en+zh) - #426

Merged
ShixiangWang merged 6 commits into
mainfrom
codex/python-p1-tutorials
Nov 2, 2026
Merged

ShixiangWang merged 6 commits into
mainfrom
codex/python-p1-tutorials

Conversation

@ShixiangWang

@ShixiangWang ShixiangWang commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Summary

Implements the six P1 topics from #424 (Python tutorial backlog), each as a bilingual (.qmd + .zh.qmd) tutorial with demo/application images, sidebar entries, and gallery CSV rows:

Tutorial Core content
Python/RadarPlot matplotlib polar radar (single + multi-series), plotly Scatterpolar interactive variant; per-axis normalization pitfall panel
Python/NetworkPlot networkx spring (seeded) / Kamada-Kawai layouts, degree/module encoding, weighted edges, hub-subnetwork application
Python/TsnePlot PCA vs t-SNE, perplexity sweep (5/30/100), PCA pre-reduction, seed & interpretation pitfalls
Python/PairedPlot slope / arrow / dumbbell variants with scipy.stats.ttest_rel annotation
Python/CalendarHeatmap GitHub-style week x weekday grid + month x day pcolormesh, week-offset algorithm, masking
Python/CompetingRiskPlot hand-rolled Aalen-Johansen CIF (verified against Σ CIF_k = 1 − KM), naive 1-KM overestimation panel, Gray-test guidance

Supporting fixes (required to keep CI green)

  • requirements.txt: add networkx (new import).
  • manage-dependencies.py: extend PYTHON_STDLIB (calendar, base64, …) — the publish workflow runs --update and would otherwise pip install calendar (a junk PyPI name) now that a tutorial imports it.
  • sync-gallery.py: route plotly Scatterpolar chunks to MANUAL (no static PNG without kaleido), matching existing Sunburst handling; +1 test.
  • files/gallery_data*.csv: +15 rows per language (Ids 1051–1065), per the Python 可视化 P1/P2 教程待实施清单(#422 调研遗留,9 主题) #424 note that the next free Id is 1051.

Verification

  • All tutorial Python chunks executed end-to-end locally; AJ estimator validated numerically (Σ CIF_k = 1 − KM holds exactly for both arms).
  • sync-sidebar.py --check: in sync (352 pages).
  • sync-gallery.py --check: in sync (90 manual cases, +1 new MANUAL = the interactive radar chunk, same pattern as the existing sunburst).
  • pytest .github/scripts/: 31 passed (incl. new Scatterpolar test).
  • mcp tests: 104 passed (untouched).
  • Cross-links (en + zh rules), image references, and YAML validity checked across all new files.
  • Demo/app PNGs rendered at 150 dpi from the same code as the chunks.

Part of #424 (P1 batch). P2 remains: waffle, wordcloud, 3D scatter.

Main's quarto-publish has been failing since the #417 renv refresh:
regplot's model.frame re-evaluation on Clinics/Nomogram.qmd errored with
"factor stage has new levels Stage IVA" under newer survival releases.

Root cause: the cohort's time column used only days_to_death, so every
censored patient (Alive, time = NA) was silently dropped by
coxph/psm. The single Stage IVA patient is censored, so the level
disappeared from the fitted model's xlevels; when regplot re-evaluates
model.frame against the full LIHC frame, the stricter survival check
rejects the unseen level.

Fix the data preparation instead of the symptom: analysis time is
days_to_death for dead patients and days_to_last_follow_up for censored
patients, then drop rows without a usable positive time. This restores
all 259 censored patients to the Cox/AFT fits (the models were
previously fit on the 172 dead patients only), keeps every stage level
in the model, and makes regplot render again. Verified end-to-end
locally: all 8 chunks of the patched tutorial execute under R 4.5.2 /
survival 3.8.6 / regplot master.
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

✅ QMD Format Check

✅ All 7 checked QMD file(s) pass format validation!

Automated check by the PR Review workflow.
See contribution guidance for the expected tutorial format.
⚠️ The workflow finished with failures; some checks may be incomplete. See the workflow run.

regplot's added.obs() calls model.frame(fit, data=app). Newer survival
(>=3.8.x in renv.lock) evaluates the coxph call in environment(fit$terms)
— set by rms::datadist — where the appended data frame does not exist,
raising "object 'app' not found". Storing the model frame via
model=TRUE lets model.frame.coxph short-circuit on fit$model.
survival 3.8's model.frame.coxph evaluates the rebuilt model.frame call
in environment(fit$terms) whose parent is the global env, so the data
symbol it receives must exist there verbatim. regplot's added.obs() only
creates app inside its own scope, hence 'object app not found'.
Pre-binding app <- rbind(observation, LIHC) in the chunk works around it.
Verified locally with survival 3.8-12 + regplot@3ee4817.
… tutorials (en+zh)

Implements the six P1 topics from issue #424:

- RadarPlot: matplotlib polar single/multi-series + plotly Scatterpolar;
  per-axis normalization pitfall panel
- NetworkPlot: networkx spring/kamada layouts, degree and module encoding,
  weighted edges, hub-subnetwork application
- TsnePlot: PCA vs t-SNE, perplexity sweep, PCA pre-reduction, seed/pitfalls
- PairedPlot: slope/arrow/dumbbell variants with paired t-test annotation
- CalendarHeatmap: GitHub-style week grid and month x day pcolormesh,
  week-offset algorithm
- CompetingRiskPlot: hand-rolled Aalen-Johansen CIF (verified against the
  sum_k CIF_k = 1 - KM identity), 1-KM overestimation panel, Gray-test notes

Supporting fixes so CI stays green:

- requirements.txt: add networkx (imported by NetworkPlot)
- manage-dependencies.py: extend PYTHON_STDLIB (calendar, base64, ...) so
  the publish workflow's --update no longer pip-installs stdlib names
- sync-gallery.py: route plotly Scatterpolar chunks to MANUAL (no static
  PNG without kaleido), with test coverage; gallery CSVs synced (15 new
  rows each language, Ids 1051-1065)
The publish workflow renders with pandas 3.1, where the chained Index
arithmetic in the data-prep chunk produced an Index without .clip and
failed the build. Rewrite the chunk on numpy (day_of_year/day_of_week
arrays, np.clip, explicit first-Monday offset) — byte-identical values
to the figure-generating script on pandas 2.3, and it also fixes a real
alignment bug: Jan 1-2 previously floored to week -1 and wrapped via
negative indexing into the last grid column (overwriting Dec 31's cell
and shifting the Jan label). Month labels now reuse the same offset.

RadarPlot: clip(lower=, upper=) keyword form for pandas 3.
@ShixiangWang
ShixiangWang force-pushed the codex/python-p1-tutorials branch from 4557723 to aae8b1e Compare November 2, 2026 09:09
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

✅ QMD Format Check

✅ All 7 checked QMD file(s) pass format validation!

Automated check by the PR Review workflow.
See contribution guidance for the expected tutorial format.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

📊 Quarto Render Preview

The new or modified Quarto files have been rendered and are available as artifacts.

🔗 View and Download Rendered Files

Click the link above, scroll to the "Artifacts" section at the bottom of the page, and download quarto-rendered-files to preview the rendered HTML files.

Modified files:

Clinics/Nomogram.qmd
Clinics/Nomogram.zh.qmd
Python/CalendarHeatmap.qmd
Python/CalendarHeatmap.zh.qmd
Python/CompetingRiskPlot.qmd
Python/CompetingRiskPlot.zh.qmd
Python/NetworkPlot.qmd
Python/NetworkPlot.zh.qmd
Python/PairedPlot.qmd
Python/PairedPlot.zh.qmd
Python/RadarPlot.qmd
Python/RadarPlot.zh.qmd
Python/TsnePlot.qmd
Python/TsnePlot.zh.qmd

@ShixiangWang
ShixiangWang merged commit 798b177 into main Nov 2, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant