[minatoaquaMK2]

Tenants create node-local data disks with PUT /v1/volumes/{name}, attach one to a claim (warm VMs included), and keep it across release, restart, template changes and hibernate. One claim at a time holds a writable volume, and the hold is derived from claims.json and the #349 stale-VM inventory. A claim for a volume on another mesh node redirects there. This implements #300. Online growth of an attached volume, idle hibernation and archive export stay open.

What changed

• PUT, GET and DELETE /v1/volumes/{name}, and POST /v1/volumes/{name}/grow, manage raw sparse files under managed_volume_dir. registry.json records each volume's owner, size and backing file, and nothing else.
• A claim attaches a managed volume the same way as a catalog volume. Holds survive a restart with no second journal.
• With managed_volume_hibernate set, hibernate of a claim with one mounted writable volume runs a guest hook, unmounts, detaches and captures. Wake reverses the steps. The claim row records the phase as volume_sleep.
• Each mesh node gossips a tenant-scoped key per volume, so peers can redirect a claim without learning names.

Scope

• Growth is offline only. A held volume answers 409 naming the holder, because cocoon cannot resize an attached disk yet.
• Archive refuses a claim that holds a volume. Idle hibernation and sleep of attach-only or multi-volume claims are not supported.

Points we'd like to talk through

We made a call on each of these so the branch works end to end. Any of them can change if you see it differently.

• Sleep phase in claims.json. It sits on the claim row as volume_sleep. An older sandboxd would wake such a claim without its disk, so the docs ask for a drain before a rollback.
• Error replies. The managed routes return {"error","code"} with statuses picked in pool (VolumeError). We can move them to sentinels in poolErrHTTP if you prefer that style.
• Hook contract. The hook runs as a host-built flock and timeout exec of hook_argv in the guest, set per node. A silkd-native hook would work too.
• Growth. Growth is offline only for now. Online growth needs cocoon to resize an attached disk (Cloud Hypervisor /vm.resize-disk).
• Placement. Root claims are not redirected to managed volumes, and name uniqueness across the mesh holds once gossip catches up.
• Stacked mounts on main. Release unmounts the topmost mount at a volume path, so a mount stacked on a writable volume is popped and its dirty marker cleared. This predates the PR, and we can open a separate issue for it.

Blast radius

Nodes without managed_volume_dir keep their behavior, with two shared changes. A guest mount conflict or a too-old cocoon on hibernate or wake answers 409 or 503 instead of 500. The Python SDK rejects a bare string or dict as a volume list. Nodes that ran earlier builds of this branch must be drained before upgrade, because the formats have no migration.

Verification

• go test -race passes for sandboxd and e2e, except two tests that also fail on main when run as root. golangci-lint passes on linux and darwin, and ruff, pytest and mypy pass.
• TestManagedVolumeIssue300Acceptance walks the acceptance line over HTTP. TestManagedSleepRoundTripAndExclusiveHold covers hibernate and wake, and two e2e tests cover placement.
• Not yet run on real Cloud Hypervisor.

The first comment explains how we built this and the decisions behind it.

[minatoaquaMK2]

How we built it and what we decided

This replaces #368 and #369. We re-checked the branch against every edge case #339 settled, then reworked it.

Process. We listed 32 cases from the #339 review (busy and stacked mounts, a release racing a change, restarts mid-change, error codes, Retry-After, journal rules) and checked each one against every managed path. Five parallel reviews found 12 must-fix bugs, each proven by a failing test. We compared two designs before any rework. Every fix landed as a failing test, then the fix. Two independent reviews followed, and the second approved.

Decisions.

Decision	Why
One journal: holds come from claims.json and the #349 stale-VM inventory. registry.json keeps only volume records.	Two journals had to stay in step, and most must-fix bugs came from the gaps.
No node-wide fence. A leftover VM that cannot be removed is queued like any stale VM. If cocoon cannot inspect it, it holds every managed name until a later pass.	One failed vm rm used to make every managed volume on the node fail until a restart. This answers the stale-VM question from #369.
The sleep phase lives on the claim row. A failed final wake write keeps memory and retries.	Matches main's claim store. A crash before the write replays the resume hook once, and the hook is required to be idempotent.
Release and expiry use main's paths and do not wait behind an in-flight hibernate or wake.	Waiting behind the hook could stall maintenance on the whole node.
Offline growth replaces the cocoon vm disk grow path.	No cocoon release has that command.
A tenant claim naming an unknown name, a forbidden catalog volume, or another tenant's managed volume gets the same 400. /v1/volumes/{name} answers 404.	Privacy needs identical answers, not a particular status, and the claim contract stays as on main.
An interrupted create stays creating. The same PUT finishes it, and DELETE removes it.	The file was never handed to a VM, so it is safe to replace.
volume_hook_failed (409) only for a guest hook exit. Transport errors stay a generic 500.	Tenants must not see host paths.
UnmountVolume stays as on main.	The device-aware version added guest round trips to every catalog release.

Deferred. Pool-side error statuses (question 2), online growth, idle and attach-only sleep, re-arming the egress proxy on wake, an e2e hibernate test, and archive export.

[CMGS]

thanks a lot for this, and for checking every case #339 settled. it's careful work.

i want to talk through the scope before we go further. my view is that a volume's lifecycle should not be controlled by the sandbox control plane. sandboxd controls the sandbox lifecycle and nothing else. to sandboxd, a disk is only an interface: it attaches it, detaches it, holds it for one writer, and checks the tenant may use it. creating, growing and deleting disks, ownership, placement, and the steps around a hibernate belong to whatever control plane needs them. that can live anywhere, it's up to whoever needs it.

so i updated the scope of #300 to say this. it also reverses my earlier comment there that sandboxd owns the detach/capture/reattach sequence, sorry for the back-and-forth.

with that split, i think one small PR on the sandboxd side closes the gap. two things are missing today:

• live attach and detach of writable and attach-only volumes on a running claim. feat(sandboxd): reconcile live read-only volume attachments #339 only takes mounted read-only ones now.
• registering and removing catalog entries at runtime, root only, so a control plane's disks can be attached without a config edit and a restart.

with those, a control plane can do the whole flow from outside: stop writers and unmount in the guest, detach, hibernate, and later wake, attach, mount. idle hibernate already skips claims with volumes, so sandboxd never captures one on its own.

what do you think? if your use case needs something this split can't do, tell me and we'll look at it.

[minatoaquaMK2]

Thanks for clarifying the boundary. This fits our original use case of changing datasets on a running sandbox without replacing the VM. The earlier version followed the previous #300 hibernate contract, and I can rework the contribution around the new scope.

I propose narrowing the sandboxd work to two pieces:

• Extend the existing live-volume API to writable and attach-only entries, preserving tenant checks, reader/writer exclusion, and feat(sandboxd): reconcile live read-only volume attachments #339's partial-progress and retry behavior. For attach-only disks, the caller would quiesce and unmount its workload before detach.
• Add root-only registration and removal of catalog entries for existing backing files. Removing an entry would never delete the file. Registration would be idempotent, and removal would refuse entries still referenced by a claim, an in-flight operation, or unresolved VM cleanup.

Disk creation, growth, ownership, placement, and the hibernate sequence would stay in the external controller. Once detach and its recovery bookkeeping complete, sandboxd's attachment hold ends. A controller that needs a reservation across sleep must keep it separately and gate workload access until wake, reattach, and mount complete, including requests that can wake the claim directly. I would document these as controller requirements.

For runtime catalog entries, I recommend persisting only the registration metadata locally and loading it before claim recovery. That keeps registration identity stable through a daemon restart without bringing disk lifecycle management back into sandboxd. Would local persistence fit your intended scope, or do you want an explicitly ephemeral catalog that the controller must replay?

The restart tests would cover physical detach succeeding while the claim write fails. That must retain the old registration binding and block unregister or name reuse until the reference retires. I also plan to validate the composed flow on a real VM before claiming end-to-end coverage.

[CMGS]

sounds good, both pieces are the right size.

on your question: persist the registration locally. keep only name → backing file (plus whatever you need to check it on load), and load it before claim recovery. no lifecycle state goes in it.

the reason is restart. a claim that holds a runtime-registered volume comes back during recovery, before any controller has reconnected. with an ephemeral catalog that claim points at a name sandboxd doesn't know, and the "detach done, claim write failed" case you describe can't be kept safe. a small local record closes that window without bringing disk lifecycle back into sandboxd.

a few things i'd like the PR to keep:

• registration is idempotent for the same name and file, and a different file under a live name is refused, not replaced.
• unregister never touches the file, and refuses while a claim, an in-flight op or pending vm cleanup still references the name, as you said.
• the record is root-owned, and the file path is validated the same way config entries are today.
• the controller requirements (quiesce and unmount before detach, its own reservation across sleep, gating access until wake/attach/mount) go in the docs next to the API.

looking forward to the smaller PR. thanks again.

[minatoaquaMK2]

Thanks, the scope and persistence requirements are clear now. I've finished the implementation plan and will start on one focused PR, organized into separate commits for runtime catalog registration, live writable/attach-only support, and integration coverage and docs. I'll share it for review once it's ready.