From c37a71842fa379b49974ba560255f23c9381624d Mon Sep 17 00:00:00 2001 From: Leo Date: Sun, 1 Nov 2026 01:24:12 +0800 Subject: [PATCH] feat(test): gate the language pairs, and lift the live-run rules out of the e2e MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two more layers of the testing system, and both found something. **A language-pair gate (`test/docs-pairing.test.ts`).** The README and `docs/quota-meter` are published twice, and every count in them is already held to the CODE — but nothing held the two LANGUAGES to each other, which is the drift that actually happens: a row added to one and not the other. It found two, immediately: - the Chinese README's settings table carried a control that no longer exists — the table listed TEN while its own prose said nine; - the Chinese README's configuration table was missing `credentialOrg`. The check compares structure: heading levels, fenced blocks, per-table body rows, and the list-item count. Prose is the translator's; a section, a code block or a table row is a fact, and a fact in one language only is invisible to whoever reads the other. The pair also disagreed about FORM in two places — one language stated the "three no-number states" as a table and the other as a list, and the "two design rules" the other way round. Both now say it the same way, which is what makes the row check fair rather than a demand for literal translation. **The live-run verdicts are now unit-tested (`test/e2e/verdict.ts`).** The rules that decide whether a live case SKIPS or FAILS used to live inside the e2e file, where only a run with a key could exercise them — so a mistake in them showed up as a red or a silently-skipped case, at the vendor's schedule. They are the rules that decide what a red run MEANS, which makes them worth more than the routing cases they gate: two of the last four rotations turned a live case red for a vendor-side reason, and each time the fix was here. The extraction also puts the picker's rule in one place (`usableAsSubject`), so the case and its picker cannot disagree about what a usable subject is. **Two coverage gaps closed where they were behavioural**, not decorative: - `decoratePlaneAdapters` on a model the catalog does not describe: the answer passes through WHOLE, with no `api`/`baseUrl` invented for a model the plugin knows nothing about (inventing one would send it to a guessed endpoint); - the hook's "cannot be served" diagnostic: a mistral model is filtered out of the picker deliberately, and the warning names the SDK that decided it — once per route/model pair, because the hook runs on every request. --- README.zh-CN.md | 2 +- docs/quota-meter.md | 5 +- docs/quota-meter.zh-CN.md | 8 +- test/docs-pairing.test.ts | 137 +++++++++++++++++++++++++++++++ test/e2e/protocol-routing.e2e.ts | 74 +++-------------- test/e2e/verdict.ts | 103 +++++++++++++++++++++++ test/models-discovery.test.ts | 31 +++++++ test/probe-verdict.test.ts | 111 +++++++++++++++++++++++++ test/stream-hook.test.ts | 29 +++++++ 9 files changed, 430 insertions(+), 70 deletions(-) create mode 100644 test/docs-pairing.test.ts create mode 100644 test/e2e/verdict.ts create mode 100644 test/probe-verdict.test.ts diff --git a/README.zh-CN.md b/README.zh-CN.md index 0c64a9a6..8d824557 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -247,7 +247,6 @@ opencode-responses: | **模型与免费额度** | 用 OpenCode 官方目录补全模型信息（默认开启） | `on` | 把目录的上下文上限、模态与思考档位补进模型信息，并让网关 `/models` 列表带上完整目录。**不会**往选择器里添加模型——选择器只显示已声明的；要添加请用卡片的**检查目录** | | **模型与免费额度** | 自动补全核心工具（默认开启） | `on` | 补上免费层 `/responses` 请求体所需的 `read` + `bash` schema | | **配额计量** | 开启输入区用量计量（默认开启） | `on` | 在输入框停靠栏挂载实时计量表——Go 额度环，或 Zen 会话消耗 | -| **配额计量** | 显示会话消耗与模型费率（默认开启） | `on` | 追加显示本会话累计花费与当前模型每百万 Token 费率 | | **账户与端点** | API 端点 | `auto` | 请求发往哪个端点家族：**自动**（跟随目录） · **旧端点**（`zen/v1`） · **新端点**（`inference/…`） | | **账户与端点** | 凭据来源 | `auto` | 多个密钥同时可用时以哪个为准：**自动** · **优先实时请求** · **优先已声明密钥** | | **账户与端点** | 读取 OpenCode 安装版的密钥（默认关闭） | `off` | 允许本插件读取 OpenCode 自己保存的凭据，且仅作最后手段——默认关闭 | @@ -362,6 +361,7 @@ Zen 路线上不出现任何 Go 的数字：根本不画圆环（没有可量的 | `sessionIdEnv` | `OPENCODE_SESSION_ID` | 指定一个仅当调用方提供会话 id 时才使用的环境变量名 | | `freeModelMarker` | `free` | 模型 id 子串；`*` 强制启用回退，`''` 关闭回退 | | `usageBaseURL` | `https://opencode.ai/zen/go/v1` | 端点覆盖；从组合中自动发现 | +| `credentialOrg` | — | 账户凭据所属的组织 id，在部署从带外提供一个时使用 | | `debug` / `debugFile` | `false` / — | 诊断用 JSONL 日志，不是谁会在 UI 里调节的行为 | ```yaml diff --git a/docs/quota-meter.md b/docs/quota-meter.md index 7818980e..f66bcb61 100644 --- a/docs/quota-meter.md +++ b/docs/quota-meter.md @@ -136,6 +136,5 @@ The token is generated at runtime by `dsh web` and never written to disk, so it ## 8. Two design rules -A number that cannot tell the truth is not rendered; the explanation of why it is absent takes its place. An estimate wearing a precise outfit is worse than a sentence, because it reads as a measurement. - -An explanation can hide behind a hover; an instruction cannot. +- **A number that cannot tell the truth is not rendered; the explanation of why it is absent takes its place.** An estimate wearing a precise outfit is worse than a sentence, because it reads as a measurement. +- **An explanation can hide behind a hover; an instruction cannot.** diff --git a/docs/quota-meter.zh-CN.md b/docs/quota-meter.zh-CN.md index 6517a7ff..d72ffe1d 100644 --- a/docs/quota-meter.zh-CN.md +++ b/docs/quota-meter.zh-CN.md @@ -98,9 +98,11 @@ composer 输入框右侧的小控件。它只回答一个问题：**你当前所 **三种"没有数字"的情况，它们不一样：** -1. **本会话暂无记录** — 累加器是 Host 进程内存里的，**重启 `dsh web` 会清空**；而且它只统计走 OpenCode 路由的轮次。发一轮就有了。 -2. **读不到配额** — 凭据缺失或请求失败。此时面板**不画**那三个窗口，而不是画三个 `0%`：编造的零比空白更糟。 -3. **额度用尽** — Go 窗口到 100%，面板转为显示 Zen 承接状态。 +| 状态 | 含义 | +| :-- | :-- | +| 本会话暂无记录 | 累加器是 Host 进程内存里的，**重启 `dsh web` 会清空**；而且它只统计走 OpenCode 路由的轮次。发一轮就有了。 | +| 读不到配额 | 凭据缺失或请求失败。此时面板**不画**那三个窗口，而不是画三个 `0%`：编造的零比空白更糟。 | +| 额度用尽 | Go 窗口到 100%，面板转为显示 Zen 承接状态。 | ## 六、设置页上的用量摘要 diff --git a/test/docs-pairing.test.ts b/test/docs-pairing.test.ts new file mode 100644 index 00000000..a1c9a3f9 --- /dev/null +++ b/test/docs-pairing.test.ts @@ -0,0 +1,137 @@ +/** + * The language pairs: one document each, two languages. + * + * `README` and `docs/quota-meter` are the documents this repository publishes + * twice. Every count and every table in them is already held to the code — but + * nothing held the two LANGUAGES to each other, and that is the drift that + * actually happens: a row added to one language and not the other. + * + * It had, when this check was first run, in both directions: + * + * - the Chinese README's settings table carried a control that no longer exists + * (its own prose said nine while the table listed ten); + * - the Chinese README's configuration table was missing `credentialOrg`. + * + * The check is deliberately about STRUCTURE, not prose. A translator may + * rephrase, and the two languages are allowed their own idiom — but a section, a + * code block or a table row is a FACT, and a fact present in one language and + * absent from the other is invisible to whoever reads only one of them. + * + * @module test/docs-pairing.test + */ + +import { readFileSync } from "node:fs"; +import { join } from "node:path"; + +import { describe, expect, it } from "vitest"; + +/** Every document published in two languages, by its base name. */ +const PAIRS = ["README", "docs/quota-meter"] as const; + +/** Read one document of the repository. */ +const read = (base: string, suffix: string): string => + readFileSync(join(process.cwd(), `${base}${suffix}.md`), "utf8"); + +/** The heading levels, in order — the document's skeleton. */ +const headings = (text: string): number[] => + [...text.matchAll(/^(#{1,6}) /gmu)].map((match) => match[1]?.length ?? 0); + +/** How many fenced blocks a document carries. */ +const fences = (text: string): number => + (text.match(/^```/gmu) ?? []).length / 2; + +/** + * The row count of each table, in order. + * + * The header and the separator are the table's SHAPE; the body rows are its + * facts. A fact is one row in one language and one row in the other — unlike a + * prose sentence, which may be one clause or three. + */ +const tableRows = (text: string): number[] => { + const counts: number[] = []; + let rows = 0; + /** Whether the previous line belonged to the same table. */ + let inTable = false; + for (const line of text.split("\n")) { + const isRow = /^\s*\|/u.test(line); + const isSeparator = /^\s*\|[\s:|-]+\|\s*$/u.test(line); + if (!isRow) { + if (inTable) { + counts.push(rows); + rows = 0; + inTable = false; + } + continue; + } + if (!inTable) { + // The first line of a table is its header: shape, not a fact. + inTable = true; + continue; + } + if (!isSeparator) { + rows += 1; + } + } + if (inTable) { + counts.push(rows); + } + return counts; +}; + +/** How many top-level list items a document carries. */ +const listItems = (text: string): number => + (text.match(/^(?:\d+\.|[-*]) /gmu) ?? []).length; + +describe("the documents published in two languages", () => { + for (const base of PAIRS) { + it(`${base}: the two languages carry the same sections`, () => { + // A section added to one language is the drift a reader cannot see: they + // read one language and never learn the other says more. + const en = headings(read(base, "")); + const zh = headings(read(base, ".zh-CN")); + expect(en.length).toBeGreaterThan(0); + expect(zh, `${base}.zh-CN`).toEqual(en); + }); + + it(`${base}: the two languages carry the same code blocks`, () => { + // A command that exists in one language only is worse than a typo: the + // reader following that language cannot run it. + const en = fences(read(base, "")); + const zh = fences(read(base, ".zh-CN")); + expect(en).toBeGreaterThan(0); + expect(zh, `${base}.zh-CN`).toBe(en); + }); + + it(`${base}: every table has the same rows in both languages`, () => { + // Row for row: a row may be rephrased, not removed. This is what found + // the stale settings row and the missing `credentialOrg` row. + const en = tableRows(read(base, "")); + const zh = tableRows(read(base, ".zh-CN")); + expect(en.length).toBeGreaterThan(0); + expect(zh, `${base}.zh-CN`).toEqual(en); + }); + + it(`${base}: the two languages carry the same list items`, () => { + // The other fact form. Counted rather than compared, because a list item + // is prose with a marker and the prose is the translator's. + const en = listItems(read(base, "")); + const zh = listItems(read(base, ".zh-CN")); + expect(zh, `${base}.zh-CN`).toBe(en); + }); + } + + it("finds the pairs it claims to check", () => { + // A loop over an empty list passes silently, which is how a gate stops being + // one. Each pair is read, and each read has to produce structure. + for (const base of PAIRS) { + for (const suffix of ["", ".zh-CN"]) { + const text = read(base, suffix); + expect(text.length, `${base}${suffix}.md`).toBeGreaterThan(1000); + expect(headings(text).length, `${base}${suffix}.md`).toBeGreaterThan(1); + expect(tableRows(text).length, `${base}${suffix}.md`).toBeGreaterThan( + 0 + ); + } + } + }); +}); diff --git a/test/e2e/protocol-routing.e2e.ts b/test/e2e/protocol-routing.e2e.ts index af1c9735..9d5b519f 100644 --- a/test/e2e/protocol-routing.e2e.ts +++ b/test/e2e/protocol-routing.e2e.ts @@ -64,6 +64,12 @@ import { internalRouteFor, isServableSdk, } from "../../src/index.ts"; +import { + accessDisabled, + freeTierClosed, + gatewayBusy, + usableAsSubject, +} from "./verdict.ts"; const LIVE = process.env.OPENCODE_E2E === "1"; const ZEN_KEY = process.env.OPENCODE_API_KEY; @@ -165,10 +171,7 @@ const PLUGIN_HEADERS: Readonly> = { "x-opencode-session": "ses_e2e0000000000abcdefghij", }; -interface Probe { - readonly status: number; - readonly summary: string; -} +import type { Probe } from "./verdict.ts"; /** Ask one endpoint about one model, optionally with a credential. */ const probe = async ( @@ -241,57 +244,6 @@ const otherProtocols = (expected: string) => * A skip is the honest answer in both: it says the routing was not verified, * which is not the same as saying it was verified and passed. */ -/** - * Whether the gateway is refusing the FREE TIER rather than the route. - * - * Measured, one request each, on `step-5-preview-free` — which was the canonical - * free-tier id when those runs were made and has since been retired, so the - * cases below now sample `beam-free`: the deployment's `oc_…` key answers - * `403 FreeTierError` on both `…/zen/v1` and the new `…/inference/openai/v1`, - * with and without `x-opencode-org-id`; the literal `public` answers the same; - * and the real client — whose captured request carries an `st_…` session token — - * is served. - * - * **Read that as a statement about the CLIENT, not the credential.** The - * gateway's own message is "OpenCode's free tier can only be used from within - * OpenCode", and presenting as that client is what this plugin does — so a free - * model is OFFERED now, and a refusal names its own reason rather than being - * pre-empted by an exclusion. What the older runs establish still holds: an - * org id and a key do not unlock it. - * - * `docs/opencode-1.19-api.md` has the endpoints, the captured headers and the - * login flow. - */ -const freeTierClosed = (result: Probe): boolean => - result.summary.includes("FreeTierError"); - -const accessDisabled = (result: Probe): boolean => - result.status === 402 || - result.summary.includes("Insufficient account funds") || - (result.status === 403 && - result.summary.includes("Model access is disabled")) || - result.status === 429 || - // No answer at all is the third shape of "cannot tell": the gateway is busy, - // so nothing about the route was observed either way. - result.status === 0; - -/** - * The gateway says this model is retired. - * - * **The registry does not always know.** The catalog's deprecation flag is the - * vendor's own bookkeeping, and the gateway's answer is the authority: a model - * can still be listed, still be priced, and still be in the catalog's active - * slice while the gateway answers `410 ModelDeprecated` for it. Measured: the - * registry listed `gemini-3-flash` as active while the gateway refused it. - * - * Such a model is not a subject to assert on — the case would fail on a - * retirement the plugin had nothing to do with — so the picker walks past it. - */ -const retiredAtTheGateway = (result: Probe): boolean => - result.status === 410 || result.summary.includes("ModelDeprecated"); - -/** The gateway is busy: it answered, but not about the route. */ -const gatewayBusy = (result: Probe): boolean => result.status === 429; /** * The protocol the model is SERVED ON, per the shipped catalog on the Zen plane. @@ -364,14 +316,10 @@ describe.skipIf(!LIVE)("live protocol routing", () => { ): Promise => { for (const model of candidates) { const answer = await probe(protocol, model, key, base); - // A model the gateway has retired is walked past too: it answers, so it is - // not an access refusal, but asserting on it would fail on a retirement - // the plugin had nothing to do with. - if ( - !accessDisabled(answer) && - answer.status !== 0 && - !retiredAtTheGateway(answer) - ) { + // The rule is in `verdict.ts`, where it is unit-tested: a refusal and a + // retirement are both vendor-side facts, and a case built on either fails + // for a reason that is not its own. + if (usableAsSubject(answer)) { return model; } } diff --git a/test/e2e/verdict.ts b/test/e2e/verdict.ts new file mode 100644 index 00000000..a3ac75a3 --- /dev/null +++ b/test/e2e/verdict.ts @@ -0,0 +1,103 @@ +/** + * How a live probe's answer is READ — the rules that decide skip, and the rules + * that decide failure. + * + * These live here, outside the e2e file, for the reason every other pure + * predicate in this repository was extracted: **a predicate that lives in a + * closure cannot be exercised alone**. The e2e suite only runs with a key + * against the live gateway, so a rule in there is tested by the vendor's + * schedule — and the rules are exactly what decides whether a red run means + * "the plugin is wrong" or "the vendor moved". + * + * The distinction is not academic. Two of the last four rotations turned a live + * case red for a vendor-side reason (a retired model answered `410`, a loaded + * gateway answered `429`), and each time the fix was to the RULE, not to the + * plugin. Those rules are now unit-tested. + * + * @module test/e2e/verdict + */ + +/** What one endpoint said about one model. */ +export interface Probe { + /** The HTTP status, or `0` when nothing answered at all. */ + readonly status: number; + /** The body, truncated — the gateway names its refusals in it. */ + readonly summary: string; +} + +/** + * Whether the gateway is refusing the FREE TIER rather than the route. + * + * Measured: the deployment's key answers `403 FreeTierError` on every endpoint, + * with and without an org id, and the literal `public` answers the same — while + * a real client, whose captured request carries a session token, is served. + * + * Read that as a statement about the CLIENT, not the credential: the gateway's + * message is "OpenCode's free tier can only be used from within OpenCode", and + * presenting as that client is what this plugin does. So a refusal here is not + * the plugin's failure to route — it is a gate this repository cannot open. + * + * @param result - the probe's answer. + * @returns whether the free tier closed the request. + */ +export const freeTierClosed = (result: Probe): boolean => + result.summary.includes("FreeTierError"); + +/** + * Whether the account cannot spend on this model — an entitlement answer, not a + * routing answer. + * + * Four shapes, and they are not the same refusal: `402` is "no funds", a `403` + * naming `Model access is disabled` is a per-model entitlement, `429` is the + * gateway's own load and says nothing about the route, and `0` is silence. + * + * @param result - the probe's answer. + * @returns whether the account (or the gateway) refused for a reason other than + * routing. + */ +export const accessDisabled = (result: Probe): boolean => + result.status === 402 || + result.summary.includes("Insufficient account funds") || + (result.status === 403 && + result.summary.includes("Model access is disabled")) || + result.status === 429 || + // No answer at all is the third shape of "cannot tell": the gateway is busy, + // so nothing about the route was observed either way. + result.status === 0; + +/** + * The gateway says this model is retired. + * + * **The registry does not always know.** The catalog's deprecation flag is the + * vendor's own bookkeeping and the gateway's answer is the authority: a model + * can still be listed, still be priced, and still sit in the catalog's active + * slice while the gateway answers `410 ModelDeprecated`. Measured: the registry + * listed `gemini-3-flash` as active while the gateway refused it. + * + * Such a model is not a subject to assert on — the case would fail on a + * retirement the plugin had nothing to do with. + * + * @param result - the probe's answer. + * @returns whether the answer is a retirement. + */ +export const retiredAtTheGateway = (result: Probe): boolean => + result.status === 410 || result.summary.includes("ModelDeprecated"); + +/** The gateway is busy: it answered, but not about the route. */ +export const gatewayBusy = (result: Probe): boolean => result.status === 429; + +/** + * Whether a candidate model may be USED as a case's subject. + * + * The rule the pickers apply: a candidate that answered is a subject, except + * when the answer was about the account (a refusal) or about the model's + * retirement. Both are vendor-side facts, and a case built on either fails for + * a reason that is not its own. + * + * @param result - the probe's answer for this candidate. + * @returns whether the case may proceed with this model. + */ +export const usableAsSubject = (result: Probe): boolean => + !accessDisabled(result) && + !retiredAtTheGateway(result) && + result.status !== 0; diff --git a/test/models-discovery.test.ts b/test/models-discovery.test.ts index 72b4ad80..9391972f 100644 --- a/test/models-discovery.test.ts +++ b/test/models-discovery.test.ts @@ -753,6 +753,37 @@ describe("models-discovery: the adapter behind a plane route", () => { stop?.(); }); + it("leaves a model the catalog does not describe exactly as the adapter answered", async () => { + // The deployment's own model — a relay's id, a private deployment — is not in + // the catalog, and the wrapper has nothing to add to it: no context window, + // no reasoning levels, and above all no ROUTING fields. A `baseUrl` invented + // for it would send it to an endpoint the catalog guessed at, which is worse + // than the adapter's own answer. + const own = { + context: { contextWindow: 4096 }, + defaultMaxTokens: 1024, + id: "acme-private-model", + inputModalities: ["text"], + name: "Acme Private", + provider: "opencode", + }; + const llm = llmWithAdapters(() => own); + const stop = decoratePlaneAdapters( + llm as never, + resolveConfig({ enrichModels: true }) + ); + const resolved = (await resolveOn(llm, "opencode")( + "opencode", + "acme-private-model" + )) as Record; + // Passed through whole: the field set is the adapter's, and the plugin added + // nothing to it. + expect(resolved).toEqual(own); + expect(resolved.api).toBeUndefined(); + expect(resolved.baseUrl).toBeUndefined(); + stop?.(); + }); + it("applies modelOverrides in decoratePlaneAdapters when declared", async () => { const raw = { context: { contextWindow: 1_000_000 }, diff --git a/test/probe-verdict.test.ts b/test/probe-verdict.test.ts new file mode 100644 index 00000000..f05c7ad6 --- /dev/null +++ b/test/probe-verdict.test.ts @@ -0,0 +1,111 @@ +/** + * The rules that decide whether a live case SKIPS or FAILS. + * + * These used to live inside `test/e2e/protocol-routing.e2e.ts`, where only a run + * with a key against the live gateway could exercise them — so a mistake in them + * showed up as a red or a silently-skipped case, at the vendor's schedule. + * + * They are the rules that decide what a red run MEANS, which makes them worth + * more than the routing cases they gate: two of the last four rotations turned a + * live case red for a vendor-side reason, and each time the fix was here. + * + * @module test/probe-verdict.test + */ + +import { describe, expect, it } from "vitest"; + +import { + accessDisabled, + freeTierClosed, + gatewayBusy, + retiredAtTheGateway, + usableAsSubject, +} from "./e2e/verdict.ts"; + +describe("the free tier's refusal", () => { + it("is read from the body, whatever status carries it", () => { + // Measured: `403 FreeTierError` on both endpoint families, with and without + // an org id — and the same body can arrive under another status when the + // gateway changes its mind about the code. The body is the stable part. + expect( + freeTierClosed({ status: 403, summary: '{"type":"FreeTierError"}' }) + ).toBe(true); + expect( + freeTierClosed({ status: 500, summary: '{"type":"FreeTierError"}' }) + ).toBe(true); + // A 403 that is NOT the free tier is a different fact, and the cases treat + // it differently: one is a gate this repository cannot open, the other is an + // account that cannot spend. + expect( + freeTierClosed({ status: 403, summary: '{"type":"AuthError"}' }) + ).toBe(false); + }); +}); + +describe("an account-level refusal", () => { + it("covers each shape the gateway uses, and only those", () => { + expect(accessDisabled({ status: 402, summary: "" })).toBe(true); + expect( + accessDisabled({ status: 500, summary: "Insufficient account funds" }) + ).toBe(true); + expect( + accessDisabled({ status: 403, summary: "Model access is disabled" }) + ).toBe(true); + expect(accessDisabled({ status: 429, summary: "" })).toBe(true); + expect(accessDisabled({ status: 0, summary: "" })).toBe(true); + }); + + it("does not read a 403 for another reason as an entitlement refusal", () => { + // The conjunction matters: a bare `403` is not "the account cannot spend", + // and treating every 403 as one would turn a routing regression into a skip. + expect(accessDisabled({ status: 403, summary: "Forbidden" })).toBe(false); + expect(accessDisabled({ status: 401, summary: "Unauthorized" })).toBe( + false + ); + expect(accessDisabled({ status: 404, summary: "no route" })).toBe(false); + }); +}); + +describe("a retirement", () => { + it("is a 410, or the vendor's word for it under another status", () => { + expect(retiredAtTheGateway({ status: 410, summary: "" })).toBe(true); + expect( + retiredAtTheGateway({ + status: 200, + summary: '{"type":"ModelDeprecated"}', + }) + ).toBe(true); + // The registry can still list a retired model as active — that is exactly + // the case this rule exists for (`gemini-3-flash` was listed and refused). + expect(retiredAtTheGateway({ status: 200, summary: "ok" })).toBe(false); + }); +}); + +describe("the busy gateway", () => { + it("is its own shape, and not a closure of the tier", () => { + // A loaded gateway is not a dead tier: it says nothing about the route or + // the account, so it must not be reported as either. + expect(gatewayBusy({ status: 429, summary: "" })).toBe(true); + expect(gatewayBusy({ status: 403, summary: "FreeTierError" })).toBe(false); + }); +}); + +describe("a candidate is a subject when the answer was about the ROUTE", () => { + it("accepts a completion, and a refusal the case can explain", () => { + expect(usableAsSubject({ status: 200, summary: "ok" })).toBe(true); + // A 403 the case turns into a skip is still an answer about the route: the + // endpoint exists, the account cannot use it. `accessDisabled` is what the + // case consults next. + expect(usableAsSubject({ status: 403, summary: "Forbidden" })).toBe(true); + }); + + it("walks past silence, a refusal, and a retirement", () => { + expect(usableAsSubject({ status: 0, summary: "" })).toBe(false); + expect(usableAsSubject({ status: 429, summary: "" })).toBe(false); + expect(usableAsSubject({ status: 402, summary: "" })).toBe(false); + expect(usableAsSubject({ status: 410, summary: "" })).toBe(false); + expect(usableAsSubject({ status: 200, summary: "ModelDeprecated" })).toBe( + false + ); + }); +}); diff --git a/test/stream-hook.test.ts b/test/stream-hook.test.ts index 727fce90..9134b22b 100644 --- a/test/stream-hook.test.ts +++ b/test/stream-hook.test.ts @@ -150,6 +150,35 @@ describe("stream-hook · the pass-through guards", () => { expect(next).toHaveBeenCalledTimes(1); }); + it("says WHY a model cannot be served, instead of leaving the picker empty", async () => { + // The diagnostic a reader gets when a model they declared is filtered out. + // The three reasons are checked in the order the filter applies them, so the + // sentence names the decision that actually fired — naming a protocol for a + // model filtered on its tier sends the reader to the routing table for an + // entitlement problem. + // + // `@ai-sdk/mistral` is the reachable case: the catalogue carries one such + // model and `isServableSdk` keeps it out of every picker, deliberately. + const hook = await hookWith(["opencode"]); + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + try { + const next = vi.fn(() => "next-result"); + expect( + hook({ model: "mistral-large-4", provider: "opencode" }, next) + ).toBe("next-result"); + expect(next).toHaveBeenCalledTimes(1); + const said = warn.mock.calls.map((call) => String(call[0])).join("\n"); + expect(said).toContain("opencode/mistral-large-4 cannot be served"); + expect(said).toContain("@ai-sdk/mistral"); + // And once per pair: the hook runs on every request, and the same missing + // route must not warn on each one. + hook({ model: "mistral-large-4", provider: "opencode" }, next); + expect(warn).toHaveBeenCalledTimes(1); + } finally { + warn.mockRestore(); + } + }); + it("accepts a numeric provider id, which the seam may carry", async () => { // `provider` is `unknown` on the seam; a number is stringified before the // allow-list is consulted, and the call still reaches `next`.