exorcist user-facing docs by atravitz · Pull Request #2076 · OpenFreeEnergy/openfe

atravitz · 2026-07-14T20:39:38Z

resolves #2075

LLM / AI generated code disclosure

LLMs or other AI-powered tools (beyond simple IDE use cases) were used in this contribution: no

Checklist

All new code is appropriately documented (user-facing code must have complete docstrings).
Added a news entry, or the changes are not user-facing.
Ran pre-commit: you can run pre-commit locally or comment on this PR with pre-commit.ci autofix.
Filled in the AI generated code disclosure.

Manual Tests: these are slow so don't need to be run every commit, only before merging and when relevant changes are made (generally at reviewer-discretion).

GPU integration tests
example notebook testing
packaging tests: run this for any large feature PRs or PRs that add test data.

Developers certificate of origin

I certify that this contribution is covered by the MIT License here and the Developer Certificate of Origin at https://developercertificate.org/.

github-actions · 2026-07-14T20:40:16Z

🚨 API breaking changes detected! 🚨

View workflow run

Griffe output

$ griffe check "openfe" -s src --no-inspection --no-color --verbose -a origin/main
src/openfe/storage/metadatastore.py:0: <module>:
Public object was removed

src/openfe/storage/resultserver.py:0: <module>:
Public object was removed

src/openfe/storage/resultclient.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_metadatastore.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_resultclient.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_resultserver.py:0: <module>:
Public object was removed


$ griffe check "openfecli" -s src --no-inspection --no-color --verbose -a origin/main
src/openfecli/commands/plan_rbfe_network.py:121: plan_rbfe_network(warehouse):
Parameter was added as required

codecov · 2026-07-23T19:00:34Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.48%. Comparing base (2f467d3) to head (27c9775).

Additional details and impacted files

@@                     Coverage Diff                      @@
##           epic/execution_improvements    #2076   +/-   ##
============================================================
  Coverage                        90.48%   90.48%           
============================================================
  Files                              217      217           
  Lines                            21239    21240    +1     
============================================================
+ Hits                             19218    19219    +1     
  Misses                            2021     2021

Flag	Coverage Δ
fast-tests	`90.48% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ethanholz

Just a few comments but nothing blocking merge.

ethanholz · 2026-07-23T20:34:27Z

+
+The following is an example script that runs up to 4 workers at a time, with each automatically picking up the next valid unit to be executed.
+
+.. code:: bash


I wonder if it might be worth adapting this to Python because it is not pleasant to read (even though it is my code lol)

I think that makes sense given our target audience!

ethanholz · 2026-07-23T20:35:46Z

+
+        db.mark_task_completed(taskid, success=result.ok())
+        # 4. output result to warehouse
+        # TODO: we may need to end up handling namespacing on the warehouse side for tokenizables


Are there any questions on this piece?

yes, but we should probably move that to the warehouse PR

IAlibay · 2026-07-28T12:55:13Z

+
+.. code:: bash
+
+    openfe plan-rbfe-network ... --warehouse


As a user, it's not immediately clear to me what --warehouse would do, would I not want to call plan-rbfe-network to get the AlchemicalNetwork and then use that as the input to my execution?

I.e. this section needs a "what arre we doing in each of these calls" explanation.

IAlibay · 2026-07-28T13:05:09Z

+import click
+from plugcli.params import Option
+
+WAREHOUSE = Option("--warehouse", type=click.BOOL, help="Use a warehouse", default=False)


This needs more detail in the help field.

IAlibay · 2026-07-28T13:06:05Z

@@ -0,0 +1,105 @@
+.. userguide_exorcist:
+
+Execution with Exorcist Workers


This is missing user centric information on the why of Exorcist. I.e. what is the advantage of using this over the old quickrun, etc...

IAlibay · 2026-07-28T13:06:32Z

+You can execute the network of simulation units defined by an ``AlchemicalNetwork`` (see `create_alchemical_network`) using ``openfe.orchestration``:
+
+
+First, we build a graph of tasks to be executed from the ``AlchemicalNetwork``:


Might be good to define tasks here - is that each individual unit that needs to be executed?

IAlibay · 2026-07-28T13:14:34Z

+
+    openfe worker warehouse/
+
+To run a single task to completion.


What if it doesn't complete? What should users be doing in that instance?

* refactor: remove result_server references in favor of gufe ExternalStorage (#1632) * feat: remove result_server references in favor of gufe external_storage Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * fix: remove extra external_store * test: change from result_server to external_storage * test: remove external_store reference * test: change extension to item per class changes * refactor: remove result server import * chore: remove metadata store from resultclient Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: remove result server * refactor: remove metadata store --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: rename to WarehouseBaseClass (#1660) * refactor(warehouse): rename to warehouse * refactor: rename _ResultContainerTest to _ContainerTest * refactor: rename _Container instances to _DataContainer * feat/refactor: add setup store (#1671) * refactor: remove extra implementations We may end up adding these back later but for the time being, we are going to remove them to keep life easy. * refactor: remove load_* and store_* functions Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * refactor: keys are being stored really strangely * refactor(warehouse): huge refactor to simplify working with Warehouse * refactor(warehouse): remove dead code * fix: delete failed due to incorrect object * test: add a test for the FileSystemWarehouse * refactor: remove dead code * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs: update warehouse docstrings * feat(warehouse): leverage keyed_chain for object dedup * Update openfe/storage/warehouse.py * import Literal * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs(warehouse): change from return, to raises * refactor(warehouse): move _key_exists to exists * chore(warehouse): remove extra todo * chore(warehouse): remove _load_stream * fix(warehouse): deduplicate objects on the filesystem * chore(warehouse): add typehint to exists --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * Temporarily build pooch from main w/ hotfix (#1806) * build with pooch@main to see if hotfix works * add link * Add support for result tokenizables to warehouse (#1763) * feat(warehouse): add result tokenizable store * docs(warehouse): add docs on the result store * chore: add property for result store * fix: use the correct function signature for handling setup --------- Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * test: cleanup warehouse test to be more modular (#1809) * fix dict type * add news item * run mypy CI on PRs into all branches (not just main) (#2112) * ignore a couple type check violations for now --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Ethan Holz <ethan.holz@omsf.io>

* refactor: remove result_server references in favor of gufe ExternalStorage (#1632) * feat: remove result_server references in favor of gufe external_storage Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * fix: remove extra external_store * test: change from result_server to external_storage * test: remove external_store reference * test: change extension to item per class changes * refactor: remove result server import * chore: remove metadata store from resultclient Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: remove result server * refactor: remove metadata store --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * refactor: rename to WarehouseBaseClass (#1660) * refactor(warehouse): rename to warehouse * refactor: rename _ResultContainerTest to _ContainerTest * refactor: rename _Container instances to _DataContainer * feat/refactor: add setup store (#1671) * refactor: remove extra implementations We may end up adding these back later but for the time being, we are going to remove them to keep life easy. * refactor: remove load_* and store_* functions Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * refactor: keys are being stored really strangely * refactor(warehouse): huge refactor to simplify working with Warehouse * refactor(warehouse): remove dead code * fix: delete failed due to incorrect object * test: add a test for the FileSystemWarehouse * refactor: remove dead code * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs: update warehouse docstrings * feat(warehouse): leverage keyed_chain for object dedup * Update openfe/storage/warehouse.py * import Literal * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * Update openfe/storage/warehouse.py Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> * docs(warehouse): change from return, to raises * refactor(warehouse): move _key_exists to exists * chore(warehouse): remove extra todo * chore(warehouse): remove _load_stream * fix(warehouse): deduplicate objects on the filesystem * chore(warehouse): add typehint to exists --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * Temporarily build pooch from main w/ hotfix (#1806) * build with pooch@main to see if hotfix works * add link * Add support for result tokenizables to warehouse (#1763) * feat(warehouse): add result tokenizable store * docs(warehouse): add docs on the result store * chore: add property for result store * fix: use the correct function signature for handling setup --------- Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io> * test: cleanup warehouse test to be more modular (#1809) * feat: add warehouse primitives for handling protocol units * feat: inital worker for exorcist * test: add tests for warehouse * fix: can now return protocol unit * refactor: make things more consistent * test: initial test setup for orchestration subpackage * test: initial exorcist utility testing * refactor: provide a root path to the exorcist DB * test: inital worker testing * feat: add shared_store * feat: add better handling for CLI application Signed-off-by: Ethan Holz <ethan.holz@omsf.io> * test: add new worker tests * feat: add exorcist worker to CLI * test: add for worker CLI command * docs: add numpy docstrings * feat: add support for planning an RBFE to a Warehouse * fix: correct edge direction for task graph * refactor: remove extra debugging from warehouse * refactor: cleanup handling of tasks for worker * refactor: fix issues in the CLI for running the worker * Update src/openfe/orchestration/__init__.py * fix dict type * add news item * ignore a couple type check violations for now * fix type checking * add exorcist to docs build --------- Signed-off-by: Ethan Holz <ethan.holz@omsf.io> Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com> Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

atravitz changed the base branch from main to feat/warehouse July 14, 2026 20:54

atravitz requested a review from ethanholz July 23, 2026 19:30

ethanholz reviewed Jul 23, 2026

View reviewed changes

atravitz self-assigned this Jul 27, 2026

atravitz changed the title ~~[WIP] exorcist docs~~ exorcist user-facing docs Jul 27, 2026

atravitz linked an issue Jul 27, 2026 that may be closed by this pull request

exorcist user-facing docs #2075

Open

IAlibay reviewed Jul 28, 2026

View reviewed changes

Base automatically changed from feat/warehouse to epic/execution_improvements July 30, 2026 21:58

atravitz mentioned this pull request Jul 30, 2026

feat: filesystem exorcist worker #1833

Merged

7 tasks

ethanholz and others added 10 commits July 30, 2026 16:13

re-org index to include exorcist execution

1d940df

copy/paste in example exorcist execution code

881eefd

brain dump for exorcist example

6d9e5dd

clean up example

3725453

fix docs build

a6133ab

Apply suggestions from code review

fbcd294

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

Apply suggestions from code review

dd82dfb

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

add type hints for me

74f6b1d

update docstring

27c9775

atravitz force-pushed the exorcist-docs branch from ab6a7d3 to 27c9775 Compare July 30, 2026 23:33

atravitz force-pushed the epic/execution_improvements branch 2 times, most recently from 6fc30c6 to a07f1c8 Compare August 7, 2026 19:51

atravitz force-pushed the epic/execution_improvements branch from a07f1c8 to a85734f Compare August 12, 2026 19:39


		The following is an example script that runs up to 4 workers at a time, with each automatically picking up the next valid unit to be executed.

		.. code:: bash

		@@ -0,0 +1,105 @@
		.. userguide_exorcist:

		Execution with Exorcist Workers

		You can execute the network of simulation units defined by an ``AlchemicalNetwork`` (see `create_alchemical_network`) using ``openfe.orchestration``:


		First, we build a graph of tasks to be executed from the ``AlchemicalNetwork``:

Uh oh!

Conversation

atravitz commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LLM / AI generated code disclosure

Developers certificate of origin

Uh oh!

github-actions Bot commented Jul 14, 2026

Uh oh!

codecov Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ethanholz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

atravitz commented Jul 14, 2026 •

edited

Loading

codecov Bot commented Jul 23, 2026 •

edited

Loading