client/transport: in-process dispatch fast-path (skip codec + network hop)

[asim]

Context (from the gap audit + Mu benchmarks — architectural, human-led)

For "go-micro is the spine an agent dispatches through" to be true and fast, in-process calls can't pay codec + transport tax. Mu measured a service.Call at ~398µs over HTTP loopback, ~64µs / 187 allocs over the memory transport — still too heavy for an agent making many tool calls per turn, and the reason Mu built capabilities three ways (HTTP / go-micro / package func).

Audit findings:

• client/rpc_client.go:148 — no in-process fast-path: every Call does pool.Get(address) → Dial → transport, simulating a network hop even when caller and callee share a process.
• transport/memory.go:82 — the memory transport double-serializes (gob over an io.Pipe on top of the RPC codec) with ~4–5 goroutine handoffs per call — the dominant latency source.

Good news

Low-risk: the server already keeps a process-local handler table (server router serviceMap), and the RPC codec already passes *codec/bytes.Frame bodies through without serialization. So a local transport / a client shortcut that dispatches directly to router.ServeRequest would eliminate the gob+pipe double-serialization and the goroutine handoffs — plausibly low-single-digit µs and a handful of allocs.

Why human-led

Touches the client/transport/registry seam and dispatch semantics (metadata, context, streaming) — architectural. Design 1:1.

Acceptance (once designed)

• An in-process Call (client and server in the same process) skips codec+transport and dispatches directly; a benchmark shows a large drop in latency/allocs vs. the memory transport.
• No behavior change for cross-process calls; existing client/server/transport tests pass.

[asim]

Design proposal (for discussion — no code yet)

Grounded in the current paths.

Where the tax is

• client/rpc_client.go:148 — every Call does r.pool.Get(address) → Dial → newRPCCodec(socket) → stream, i.e. a full network round-trip even when the callee is a server in this process.
• transport/memory.go:82 — the memory transport Decodes a gob message off an io.Pipe socket on top of the RPC codec's own marshal, with ~4–5 goroutine handoffs per call. That double-serialize + handoff is the dominant cost.
• server/rpc_router.go:500 — router.ServeRequest(ctx, Request, Response) → service.call(...) is the process-local dispatch, and it already runs the server-side handler wrappers. That's the entrypoint a fast-path should reuse (so middleware still runs).

Proposed mechanism

1. A process-local router registry. On Server.Start, register the server's router (its serviceMap handler table) in a neutral registry keyed by service name; deregister on Server.Stop. The client consults it in Call before pool.Get.
2. Shortcut when local. If req.Service() is served in-process, the call is unary (not streaming), and no node was explicitly pinned, dispatch directly to ServeRequest; otherwise fall through to today's transport path unchanged.
3. Two possible direct paths — I'd ship (b) first:
   • (b) codec-once, no socket/pipe (recommended first): build a Request/Response over an in-memory buffer codec and call router.ServeRequest directly. Eliminates the transport pipe, the gob layer, and the goroutine handoffs — one marshal/unmarshal instead of two, no sockets. Big win, preserves content-type/codec semantics, makes no type-identity assumptions.
   • (a) zero-copy passthrough (later): for the *codec/bytes.Frame path the RPC codec already passes bodies through unserialized; a passthrough codec could hand argv→replyv with no marshal at all. Fastest, but assumes shared concrete types in-process — worth it later for the Frame-heavy MCP/agent dispatch, not first.

Semantics to preserve (checklist)

• Server wrappers run — via ServeRequest → service.call; client wrappers already run before Call. ✔
• Metadata / deadline / cancellation — flow through ctx directly (strictly better than the transport path, which serializes metadata into headers).
• Error mapping — handler errors → merrors exactly as the transport path returns them.
• No behavior change for cross-process, streaming, or explicitly-addressed calls — those take the existing path.
• Lifecycle — deregister on stop so we never dispatch into a stopped server; key by service name and handle multiple servers in one process.

The one real structural decision (needs your call)

Import cycle. client cannot import server (server imports client). So the local-router registry needs a neutral home — options:

1. A tiny internal/localdispatch package exposing Register(service string, d Dispatcher) / Lookup(service) (Dispatcher, bool) where Dispatcher is a minimal interface (ServeRequest-shaped); server registers into it, client reads from it. My recommendation — smallest surface, no cycle.
2. A client.Client option carrying an explicit local dispatcher (opt-in, no globals) — safer but requires wiring at construction and won't "just work" for service.Call.

Acceptance / validation

• A benchmark comparing memory-transport Call vs the in-process fast-path showing a large latency + allocs drop (the issue cites ~64µs / 187 allocs on memory transport as the bar to beat — target low-single-digit µs and a handful of allocs).
• Existing client / server / transport tests unchanged; a focused test proving a local call and a cross-process call produce identical results and both run wrappers.

Questions for you

1. Registry home: neutral internal/localdispatch global (auto, "just works") vs. an explicit opt-in client option? I lean global-but-internal.
2. Auto vs opt-in shortcut: shortcut automatically whenever the service is local, or gate behind a client.WithLocalFastPath() for a conservative rollout?
3. Start with (b) codec-once, treat (a) zero-copy Frame as a follow-up once the seam is proven?

If the shape's right, I'll prototype (b) behind the internal/localdispatch seam with the benchmark, so we can see the real number before committing to auto-on.

---

Generated by Claude Code