feat: expose opt-in EP updater through declarative optimise

[Jammy2211]

Overview

The declarative EP API currently cannot pass an updater into EPOptimiser, even though the inner optimiser already supports fixed, per-factor, and dynamic update policies. This forces scientific workflows to use private glue when they want to experiment with damping. Expose the existing updater abstraction additively, while keeping updater=None as the public default so every existing fit remains undamped unless the caller explicitly opts in.

Plan

• Thread an optional updater through the declarative optimise() lowering path.
• Keep the default None, preserving current behaviour and enabling no damping by default.
• Export the existing updater classes through the public API.
• Add seam tests covering explicit updater propagation and unchanged default behaviour.
• Document damping as an opt-in, problem-dependent tool rather than a generally safe sigma-collapse fix.
• Show the option in the workspace tutorial without enabling it in the tutorial's default fit.
• Leave diagnostic variable-name improvements for a separate task.

Detailed implementation plan

Affected Repositories

• PyAutoFit (primary)
• autofit_workspace

Work Classification

Both — library first, followed by its workspace example.

Worktree root

~/Code/PyAutoLabs-wt/ep-optimise-updater/

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoFit	main	clean
./autofit_workspace	main	clean

No active PyAutoMind worktree claim currently overlaps either repository.

Suggested branch: feature/ep-optimise-updater

Implementation Steps

1. In PyAutoFit/autofit/graphical/declarative/abstract.py, add updater: Optional[ApproxUpdater] = None to AbstractDeclarativeFactor.optimise() and _make_ep_optimiser(), and pass it unchanged into EPOptimiser. Do not construct or select an updater in the declarative layer when the argument is omitted.
2. Re-export ApproxUpdater, SimplerUpdater, FactorUpdater, and DynamicUpdater through autofit.graphical; expose the normal fixed-damping entry point through the top-level autofit API so users can write af.SimplerUpdater(delta=...).
3. Add declarative seam tests proving that an explicitly supplied updater reaches EPOptimiser unchanged and that omitting it retains the current undamped SimplerUpdater(delta=1.0) behaviour inside EPOptimiser.
4. Update the lowering-contract documentation in autofit/graphical/README.md to record that updater policy now survives declarative lowering.
5. Revise the sigma-collapse diagnostic guidance cautiously: point to the opt-in updater API, but state that damping is problem-dependent and has worsened hierarchical scale collapse in prior repeated-run diagnostics.
6. In autofit_workspace/scripts/features/expectation_propagation.py, explain and show the optional keyword without enabling it in the default fit. Regenerate the corresponding notebook after the script is validated.
7. Run focused PyAutoFit graphical tests, then the full PyAutoFit suite. Run the autofit_workspace smoke test, regenerate the notebook, and verify that only the intended script/notebook pair changes.

Key Files

• PyAutoFit/autofit/graphical/declarative/abstract.py — declarative-to-inner optimiser lowering.
• PyAutoFit/autofit/graphical/expectation_propagation/optimiser.py — existing updater types and default behaviour.
• PyAutoFit/autofit/graphical/__init__.py and PyAutoFit/autofit/__init__.py — public exports.
• PyAutoFit/test_autofit/graphical/gaussian/test_declarative.py — seam and default tests.
• PyAutoFit/autofit/graphical/expectation_propagation/diagnostics.py — actionable but cautious warning text.
• PyAutoFit/autofit/graphical/README.md — lowering contract.
• autofit_workspace/scripts/features/expectation_propagation.py — opt-in tutorial guidance.

Explicitly Out of Scope

Propagating human-readable model paths into EPDiagnostics and mean_field_summary. That is a distinct diagnostics task.

Original Prompt

Click to expand starting prompt

EP declarative optimise() cannot apply the damping its own diagnostics recommend

Type: feature
Target: autofit
Repos:

• PyAutoFit
  Difficulty: small
  Autonomy: supervised
  Priority: high
  Status: formalised

Problem

ep_diagnostics.results (the #1335 sigma-collapse guard) tells users to "consider
damping, e.g. delta < 1" — but the public declarative API cannot do it:
AbstractDeclarativeFactor.optimise() → _make_ep_optimiser() hardwires the default
SimplerUpdater(delta=1.0); neither optimise() nor EPOptimiser.run() accepts an
updater/delta, even though EPOptimiser.__init__ supports updater= and the
SimplerUpdater/FactorUpdater/DynamicUpdater hierarchy exists.

Found on slope_hierarchy (Jammy2211/slope_hierarchy#1): the undamped EP fit of 5
power-law lenses sigma-collapsed (every drawn slope pinned at the prior edge, std
~1e-19, parent sigma → 0; RAL job 330495). The diagnostics flagged it perfectly and
recommended the one knob the API doesn't expose. The project works around it by
replicating the optimise() glue with _make_ep_optimiser + opt.updater = SimplerUpdater(delta)
(slope_hierarchy/scripts/ep.py) — private-API use that should not be needed.

Ask

Thread an updater: Optional[ApproxUpdater] = None (or a plain delta: float = 1.0)
kwarg through AbstractDeclarativeFactor.optimise() → _make_ep_optimiser() →
EPOptimiser. Additive, default-preserving. Update the EP feature docs
(autofit_workspace/scripts/features/expectation_propagation.py) and the
sigma-collapse warning text to name the now-reachable kwarg.

Secondary (same wave, may split)

Diagnostics label variables uniformprior_19-style; on a realistic 65-parameter
lensing graph the sigma-collapse warnings and mean_field_summary are unreadable —
consider propagating model path names into EPDiagnostics/mean_field_summary.

[Jammy2211]

Library PR Created — 2026-08-08

PR

• feat: expose opt-in EP updater through declarative optimise #1457

API Changes

The declarative FactorGraphModel.optimise() API now accepts an optional updater, and the updater hierarchy is publicly exported. The default remains updater=None, which preserves undamped SimplerUpdater(delta=1.0) behaviour; damping is opt-in only.

Validation

• PyAutoFit: 1667 passed, 6 skipped
• Focused declarative/diagnostic tests: 17 passed
• Changed workspace tutorial script: passed
• Generated notebook: exact canonical PyAutoHands output

Next

The workspace changes are ready locally but /ship_workspace is parked at its smoke gate. This sandbox blocks all Jupyter kernels with ZMQError: Operation not permitted, Heart reports PyAutoFit: on branch feature/ep-optimise-updater (not main) for the web working-clone layout, and the broad AutoLens integration rerun ended in fatal library error, lookup self after its JAX/NUFFT checks. Resume shipping after those exact gate limitations are acknowledged or rerun in an unrestricted environment.

[Jammy2211]

Workspace PR created: PyAutoLabs/autofit_workspace#136

Published under the acknowledged workspace-gate exception for sandbox-only limitations (branch-sensitive Heart check, blocked notebook IPC kernels, and the AutoLens integration runtime's fatal library error, lookup self). The workspace PR remains draft and must not merge before PyAutoFit PR #1457.