Skip to content

Add statistical comparison tutorial - #102

Open
maltekuehl wants to merge 8 commits into
mainfrom
feat/statistics-tutorial
Open

maltekuehl wants to merge 8 commits into
mainfrom
feat/statistics-tutorial

Conversation

@maltekuehl

Copy link
Copy Markdown
Collaborator

Currently, pytometry covers file reading, transforms and clustering, but no guidance exists on what users can eventually do with that data to answer their biological questions.

This PR adds a small example notebook based on pertpy and simulated data. It contains a good chunk of data wrangling to bridge the gap between pytometry and pertpy, this could also be integrated into pytometry directly as pertpy wrappers but would require adding another dependency and broadening its surface. Wdyt?

Happy to hear thoughts about having something like this in pytometry.

@maltekuehl
maltekuehl added this pull request to stack #103 October 27, 2026 12:49
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@maltekuehl maltekuehl added the enhancement New feature or request label Oct 27, 2026
@Zethson Zethson assigned mbuttner and unassigned Zethson Oct 27, 2026
@mbuttner

Copy link
Copy Markdown
Collaborator

Hi Malte,
thanks for creating the tutorial - I think it's amazing and demonstrates very nicely how to address differential abundance questions in (flow) data. Happy to have it as part of pytometry - I'm a bit concerned to add pertpy as dependency to pytometry though (counters the idea of having a lightweight package for cytometry data wrangling), but I like the idea of broadening the scope.
Please correct me if I misunderstood the part on adding pertpy as dependency.

Base automatically changed from feat/fastersom to main October 30, 2026 11:16
maltekuehl and others added 7 commits October 30, 2026 12:16
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Uses the BCR-XL stimulation data of the Bioconductor CyTOF workflow:
FlowSOM on the type markers, per-sample counts and medians with pandas
and scanpy, differential abundance with pertpy's scCODA and
differential state with a paired linear model through pertpy's
Statsmodels wrapper.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- install line pertpy[jax,de] plus session-info2, checked in a fresh venv
- download with pooch and checksums; data cached outside the repository,
  so the gitignore rules are no longer needed
- fixed number of meta-clusters (20) and populations named by rules on
  the cluster medians, which stop on ambiguous or missing matches;
  checked with seeds 0, 1 and 7; debris, doublets and unclear clusters
  are excluded
- show that type markers shift with the stimulation, and that the
  monocyte decrease is largely a shift to the DC population
- scCODA with CD4 T cells as explicit reference, with the dependence on
  the reference and the comparison with cytofWorkflow stated
- state results filtered by effect size as well as BH, broad changes of
  pAkt, pNFkB, pp38, pBtk and pSlp76 described with the cytofWorkflow
  result

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tutorial now downloads one archive of the QBC-CyTOF cross-species
dataset from Zenodo (10.5281/zenodo.23406825, CC-BY 4.0) and compares
14 human donors before and after CD40L stimulation, instead of the
BCR-XL data from an Internet Archive copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- cite the Zenodo dataset (creators, title, version, DOI, licence) and
  the authors' code repository, without the unpublished manuscript
- report all BH-significant state changes sorted by size instead of a
  cut-off chosen after seeing the data, and say plainly how small they are
- show proportions and per-donor B cell changes as violin plots with
  one dot per sample instead of boxplots

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tutorial now uses the Levine et al. (2015) CyTOF data from figshare
(10.6084/m9.figshare.19867021, CC BY 4.0): the basal samples of 5
healthy donors and 8 AML patients, compared unpaired, with scCODA
against T cells for abundance and Welch t-tests on per-sample medians
for state markers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Simulated data with planted effects show differential abundance and differential state in an unpaired and a paired design, with known ground truth. Both questions are tested with pertpy's linear models: CLR-transformed cell counts for abundance, per-sample medians for state.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Zethson
Zethson force-pushed the feat/statistics-tutorial branch from 9e7bce9 to e1771db Compare October 30, 2026 11:16
@Zethson

Zethson commented Oct 30, 2026

Copy link
Copy Markdown
Member

This PR isn't adding pertpy as a dependency of pytometry. It just shows that you can use both of them together for a specific purpose. I think that's perfectly fine.

@Zethson
Zethson marked this pull request as ready for review October 30, 2026 11:21
@maltekuehl

Copy link
Copy Markdown
Collaborator Author

Thank you both for the comments! @Zethson is right that this PR in particular does not add pertpy (yet), but I originally created this as a draft PR to discuss the possibility of adding some direct bindings and utils as part of pytometry, too. I will have a look next to explore what common utils might make sense to have as part of this lightweight package without adding pertpy as a dependency, only keeping it in the online tutorial. Thanks for the feedback!

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.69%. Comparing base (1dba141) to head (f6de253).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #102   +/-   ##
=======================================
  Coverage   76.69%   76.69%           
=======================================
  Files          15       15           
  Lines         618      618           
=======================================
  Hits          474      474           
  Misses        144      144           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants