Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ has a matching GitHub release whose body comes from its section below.

## [Unreleased]

## [v0.4.0] - 2026-10-29

One berthd now governs every card on a machine. `berth run --gpu gpu1` puts an engine on one card,
`--gpu gpu0,gpu1` holds one lease on several, and `berth run --pinned` keeps a training run's cards. The gateway,
`berth status`, `berth notify`, `/metrics` and the harbor name the card, and one card can go to a VM while the
others serve. Engines with no offload call of their own, llama.cpp and NInfer included, park in RAM through
`cuda-checkpoint`. `whole_machine = true` is for an engine such as Strata that needs the card, the RAM and the CPU,
so nothing parks beside it. A Docker Compose project runs as one engine
([#172](https://github.com/T-Crypt/berth/issues/172)). An engine that can shrink a cache can lend part of its card
and keep serving (experimental, [#189](https://github.com/T-Crypt/berth/issues/189)). berthd uses less CPU, and the
AMD probe no longer stops berthd on a desktop session ([#196](https://github.com/T-Crypt/berth/issues/196)).

### Added

- The harbor names the model each lease serves on its card's panel: the model its engine reports loaded or ready, or
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ claim. When another engine holds it, Berth:
The numbered markers on the timeline are those four steps. A process that holds VRAM without a lease
shows up as a stowaway: Berth reports its PID and memory and leaves it running.

> **Status: alpha, v0.3.6.** Runs daily on an RTX 4090 workstation beside llama-swap, NInfer,
> **Status: alpha, v0.4.0.** Runs daily on an RTX 4090 workstation beside llama-swap, NInfer,
> Strata, ComfyUI and games, and testers run it on Windows. The [docs](docs/README.md) cover what works and has tests; the
> [roadmap](docs/roadmap.md) lists the phases and what is still missing.

Expand Down
4 changes: 2 additions & 2 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ desktop notifications on Linux, `berth skill install` adds a ComfyUI skill, and
the card is free ([#172](https://github.com/T-Crypt/berth/issues/172)). [Swap numbers](benchmarks.md#swap-numbers)
now has page cache, park and `cuda-checkpoint` timings from the RTX 4090.

## Shipped: v0.4, many cards and parks for any engine
## Shipped: v0.4, many cards and parks for any engine (2026-10-29)

v0.4 puts a whole machine under one berthd, first as pre-releases (`v0.4.0-rc.1` to `rc.7`, see
v0.4.0 puts a whole machine under one berthd. Testers ran it first as pre-releases (`v0.4.0-rc.1` to `rc.8`, see
[Testing a pre-release](prerelease.md)).

- **Many cards.** berthd and `berth service install` govern every card by default (`--gpu all`).
Expand Down
8 changes: 5 additions & 3 deletions internal/engines/strata_requests_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ func strataServer(t *testing.T, model string, at time.Time, finish string) *http
// under the model their /health names, whatever port they run on, and a
// second poll of the same request list counts nothing twice.
func TestStrataRequestsReachTheLeaderboard(t *testing.T) {
at := time.Date(2026, 10, 28, 21, 35, 21, 155911000, time.FixedZone("MDT", -6*60*60))
// An hour ago, inside the watcher's 24h request window.
sec := time.Now().Add(-time.Hour).Unix()
at := time.Unix(sec, 155911000).In(time.FixedZone("MDT", -6*60*60))
iq4 := strataServer(t, "qwen3.8-flash-next-ud-iq4_xs", at, "stop")
sc117 := strataServer(t, "qwen3.8-flash-next-sc117-iq3_s", at.Add(time.Second), "cancel")
w := &Watcher{Engines: []config.Engine{
Expand All @@ -54,10 +56,10 @@ func TestStrataRequestsReachTheLeaderboard(t *testing.T) {

got := w.Requests(time.Time{})
want := []proto.Request{
{ID: "strata/1793244921.155911", Engine: "strata", Model: "qwen3.8-flash-next-ud-iq4_xs", T: at, Status: 200,
{ID: fmt.Sprintf("strata/%d.155911", sec), Engine: "strata", Model: "qwen3.8-flash-next-ud-iq4_xs", T: at, Status: 200,
Prompt: 32000, Cache: 31552, Output: 388, Draft: 277, DraftAcc: 214,
PrefillTPS: 448 / 0.6211, DecodeTPS: 95.5, DurationMS: 4700, PrefillMS: 621.1},
{ID: "strata-sc117/1793244922.155911", Engine: "strata-sc117", Model: "qwen3.8-flash-next-sc117-iq3_s", T: at.Add(time.Second), Status: 499,
{ID: fmt.Sprintf("strata-sc117/%d.155911", sec+1), Engine: "strata-sc117", Model: "qwen3.8-flash-next-sc117-iq3_s", T: at.Add(time.Second), Status: 499,
Prompt: 32000, Cache: 31552, Output: 388, Draft: 277, DraftAcc: 214,
PrefillTPS: 448 / 0.6211, DecodeTPS: 95.5, DurationMS: 4700, PrefillMS: 621.1},
}
Expand Down
24 changes: 24 additions & 0 deletions internal/runner/park_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import (
"github.com/T-Crypt/berth/internal/park"
"github.com/T-Crypt/berth/internal/procs"
"github.com/T-Crypt/berth/internal/proto"
"github.com/T-Crypt/berth/internal/runs"
"github.com/T-Crypt/berth/internal/server"
)

Expand Down Expand Up @@ -601,6 +602,29 @@ func TestDrainOfAParkedEngineSaysWhy(t *testing.T) {
// A parked engine whose daemon restarts comes back parked: the new daemon
// grants a whole-card waiter beside its residual, and the engine is never
// drained, where an active lease would be drained and stopped.
// An engine that exits while its daemon is gone leaves no unprotected-run
// record behind once Run returns.
func TestRunReturnsWithItsLostRecordGone(t *testing.T) {
card := &parkCard{total: 24 << 30}
sock := filepath.Join(t.TempDir(), "berth.sock")
b, stop := parkDaemonAt(t, sock, card, 16<<30)
exited, _ := runPark(t, sock, "vllm", "http://127.0.0.1:1", false)
pid := attachAndMeasure(t, b, card)
stopAtEnd(t, pid, exited)

stop()
dir := runs.Dir(sock)
waitFor(t, "the lost record", func() bool {
left, _ := os.ReadDir(dir)
return len(left) > 0
})
_ = syscall.Kill(-pid, syscall.SIGKILL)
<-exited
if left, _ := os.ReadDir(dir); len(left) != 0 {
t.Fatalf("records left after Run returned: %v", left)
}
}

func TestParkedEngineReattachesParked(t *testing.T) {
card := &parkCard{total: 24 << 30}
sock := filepath.Join(t.TempDir(), "berth.sock")
Expand Down
6 changes: 6 additions & 0 deletions internal/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,12 @@ func Run(o Options) int {
}()
}

// Run returns only after the listener does, so an unprotected-run record
// it wrote is gone by then.
listening := make(chan struct{})
go func(c net.Conn) { <-done; c.Close() }(raw)
go func() {
defer close(listening)
for {
err := listen(conn, func(m proto.Msg) {
switch {
Expand Down Expand Up @@ -388,6 +393,7 @@ func Run(o Options) int {
_ = line.send(proto.Msg{Op: proto.OpFinished})
}
close(done)
<-listening
return code
}

Expand Down
Loading