feat(migrate): record the source, stage, and target app and instance in telemetry

.changeset/migrate-telemetry.md

@@ -0,0 +1,5 @@
+---
+"clerk": patch
+---
+
+Record in usage telemetry which platform a `clerk migrate` run's users come from, how far the run got, and the Clerk app and instance it acted on, and say so in the first-run telemetry notice.

README.md

@@ -102,11 +102,15 @@ Commands:
 ## Telemetry
 
 The Clerk CLI collects usage telemetry: command name, flag names, duration, outcome,
-environment signals (OS, install method, terminal), a random machine identifier — and
-your workspace and app IDs when a project is linked. It never collects command
-arguments, option values, file paths, or personal data, except the text you choose to
-send with `clerk feedback`, which is scrubbed on a best-effort basis for common secret
-formats, home-directory paths and email addresses before it leaves your machine;
+environment signals (OS, install method, terminal), a random machine identifier, the
+platform a migration's users come from, and the workspace, app and instance IDs of the
+Clerk app a command acts on or the project is linked to. It doesn't collect raw command
+arguments, option values, file paths, or personal data, with two exceptions. `clerk migrate`
+reports where a migration's users come from as a built-in platform name or `custom`, and
+the app and instance IDs it acts on, which can come from `--app` and `--instance`.
+`clerk feedback` sends the text you choose to send, which is scrubbed on a best-effort
+basis for common secret formats, home-directory paths and email addresses before it
+leaves your machine;
 `clerk feedback --dry-run` shows exactly what would be sent. The first run only shows a
 disclosure notice and sends nothing (CI environments send from the first run), except
 `clerk feedback`, which shows the notice and then sends the report you asked it to send

packages/cli-core/src/commands/migrate/README.md

@@ -1510,6 +1510,36 @@ stamping every user with today's.
 | `skipLegalChecks`           | `boolean` | Skip legal acceptance checks                                                         |
 | `skipPasswordChecks`        | `boolean` | Skip password requirements on import                                                 |
 
+## What telemetry records
+
+Every `clerk migrate` event carries the fields every command sends, plus four that say what the run did. Each is null when the run never learned it.
+
+| Field            | What it holds                                                                                                                                                     |
+| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `migrate_source` | Where the users come from: a built-in source or export platform (`auth0`, `clerk`, …), or `custom` for a source you wrote. Never its path or its key.             |
+| `instance_id`    | The Clerk instance the command acted on (`ins_…`). For `export clerk`, the instance it read.                                                                      |
+| `app_id`         | Its app (`app_…`): from `--app`, or from the linked profile when that profile owns the instance. A key for another app's instance sends null, not the linked app. |
+| `stage`          | How far the run got. One funnel per subcommand, below.                                                                                                            |
+
+`workspace_id` is the linked profile's only when `app_id` is that profile's app. A `key_…` stand-in, the hash a run records when `GET /v1/instance` fails, is never sent: neither is an `--instance` alias or an app label.
+
+Stages, in the order each subcommand runs them. The last one set is the one sent, so a failure or a Ctrl-C reports where the run stopped:
+
+- `import`: `migrate_source`, `migrate_target_setup` (sign-in, link), `migrate_input`, `migrate_target`, `migrate_resume`, `migrate_load`, `migrate_checks`, `migrate_confirm`, `migrate_import`, `done`. Terminal branches: `migrate_already_imported`, `migrate_dry_run`, `migrate_nothing_to_import`.
+- `export`: `migrate_pick_platform` (bare `clerk migrate export` only), `migrate_export_read` (credentials and reading users), `migrate_export_write`, `done`.
+- `undo`: `migrate_run_lookup`, `migrate_target`, `migrate_inspect`, `migrate_confirm`, `migrate_delete`, `done`. Terminal branch: `migrate_dry_run`.
+
+`runs` and `sources` are one step each and send no stage.
+
+A run that finishes part-way exits nonzero without throwing, and carries a code so it is not an uncoded error:
+
+| Ending                                     | `error_code`               | Exit |
+| ------------------------------------------ | -------------------------- | ---- |
+| `import --dry-run` the checks would refuse | `migrate_checks_rejected`  | 2    |
+| `import` where some users failed           | `migrate_import_partial`   | 1    |
+| `undo` with deletes left                   | `migrate_undo_incomplete`  | 1    |
+| `export` the platform cut short            | `migrate_export_truncated` | 1    |
+
 ## API endpoints
 
 | Method   | Path                                                    | Used by                                                                                                                        |

packages/cli-core/src/commands/migrate/export/clerk.ts

@@ -28,6 +28,7 @@ import {
 } from "../lib/target.ts";
 import { resolveClerkSource } from "./clerk-source.ts";
 import { finishExport, startExportRun } from "./shared.ts";
+import { recordMigrateTarget } from "../telemetry.ts";
 
 /** BAPI's maximum page size for `GET /v1/users`. */
 const PAGE_SIZE = 500;
@@ -296,6 +297,13 @@ export async function exportClerk(options: ExportClerkOptions): Promise<void> {
       instanceId: identity.instanceId,
       ...(keySource ? { keySource } : {}),
     };
+    // The source instance: for this command, the one it acts on.
+    // `--app` named the app unless a key overrode it; a linked profile's app
+    // is filled in from its instances.
+    recordMigrateTarget({
+      appId: keyFrom ? undefined : options.app,
+      instanceId: identity.instanceId,
+    });
     if (!options.json) printTarget(target);
 
     const users = await withSpinner("Fetching users from Clerk...", async (spinner) =>

packages/cli-core/src/commands/migrate/export/index.ts

@@ -12,6 +12,8 @@ import { exportWorkOs } from "./workos.ts";
 import type { DbExportOptions } from "./db-options.ts";
 import { RUNS_DIR_DESCRIPTION, RUNS_DIR_FLAG } from "../lib/run-store.ts";
 import { exportPlatformKeys, exportPlatforms, getExportPlatform } from "./registry.ts";
+import { setTelemetryStage } from "../../../lib/telemetry.ts";
+import { recordMigrateSource } from "../telemetry.ts";
 
 /**
  * Bare `clerk migrate export` — pick a platform, then run its export.
@@ -33,6 +35,7 @@ export async function exportPicker(options: Record<string, unknown> = {}): Promi
     );
   }
 
+  setTelemetryStage("migrate_pick_platform");
   const platform = await select<string>({
     message: "Which platform are you exporting from?",
     choices: exportPlatforms.map((entry) => ({
@@ -46,6 +49,8 @@ export async function exportPicker(options: Record<string, unknown> = {}): Promi
   // Unreachable via the picker; a guard so a registry edit cannot silently
   // produce a choice with nothing behind it.
   if (!entry) throwUsageError(`Unknown export platform "${platform}".`);
+  recordMigrateSource(platform);
+  setTelemetryStage("migrate_export_read");
 
   await entry.run(options);
 }
@@ -118,6 +123,11 @@ export function registerMigrateExport(migrateCommand: Command<[], Record<string,
   // extension: an envelope written to users.ndjson could not be imported by
   // run ID. Checked before anything is read.
   exportCommand.hook("preAction", (_thisCommand, actionCommand) => {
+    // The bare picker records both once a platform is chosen.
+    if (actionCommand !== exportCommand) {
+      recordMigrateSource(actionCommand.name());
+      setTelemetryStage("migrate_export_read");
+    }
     const { output } = actionCommand.opts() as { output?: unknown };
     if (typeof output === "string" && !output.toLowerCase().endsWith(".json")) {
       throwUsageError(

packages/cli-core/src/commands/migrate/export/shared.test.ts

@@ -3,7 +3,7 @@ import fs from "node:fs";
 import os from "node:os";
 import path from "node:path";
 import { quoteArg } from "../../../lib/json-body.ts";
-import { useCaptureLog } from "../../../test/lib/stubs.ts";
+import { captureTelemetryPayload, useCaptureLog } from "../../../test/lib/stubs.ts";
 import { readEnvelope } from "../lib/export-file.ts";
 import { latestUserLines, readRun } from "../lib/run-store.ts";
 import { finishExport, formatImportCommand, startExportRun } from "./shared.ts";
@@ -196,6 +196,33 @@ describe("finishExport", () => {
     }
   });
 
+  test("a finished export ends at done", async () => {
+    const { payload } = await captureTelemetryPayload("migrate export auth0", async () => {
+      const run = await startExportRun({ runsDir }, { platform: "auth0" });
+      finishExport({ run, options: {}, users, coverage });
+    });
+    expect(payload).toMatchObject({ outcome: "success", stage: "done" });
+  });
+
+  test("a truncated export is coded", async () => {
+    const { payload } = await captureTelemetryPayload("migrate export auth0", async () => {
+      const run = await startExportRun({ runsDir }, { platform: "auth0" });
+      finishExport({ run, options: {}, users, coverage, truncated: true });
+    });
+    expect(payload).toMatchObject({
+      outcome: "error",
+      stage: "done",
+      error_code: "migrate_export_truncated",
+    });
+  });
+
+  test("starting the export run records migrate_export_write", async () => {
+    const { payload } = await captureTelemetryPayload("migrate export auth0", async () => {
+      await startExportRun({ runsDir }, { platform: "auth0" });
+    });
+    expect(payload.stage).toBe("migrate_export_write");
+  });
+
   test("prints the import command by run ID", async () => {
     const run = await startExportRun({ runsDir }, { platform: "clerk" });
 

packages/cli-core/src/commands/migrate/export/shared.ts

@@ -12,6 +12,7 @@ import fs from "node:fs";
 import path from "node:path";
 import { dim, green, yellow } from "../../../lib/color.ts";
 import { CliError } from "../../../lib/errors.ts";
+import { declareSoftExitOutcome, setTelemetryStage } from "../../../lib/telemetry.ts";
 import { log } from "../../../lib/log.ts";
 import { carryFlags, withFlags } from "../lib/carry-flags.ts";
 import { ENVELOPE_VERSION, type ExportEnvelope } from "../lib/export-file.ts";
@@ -45,6 +46,7 @@ export async function startExportRun(
   options: ExportCommonOptions,
   target: RunTarget,
 ): Promise<Run> {
+  setTelemetryStage("migrate_export_write");
   const runsDir = await resolveRunsDir(options.runsDir, { write: true });
   return startRun(runsDir, { kind: "export", target, source: target.platform });
 }
@@ -199,7 +201,11 @@ export function finishExport(input: FinishExportInput): FinishedExport {
   const skipped = record.counts.skipped ?? 0;
   // Partial, like an import with failures: the file holds what came back, and
   // the exit says it is not everyone, where `truncated` alone is easy to miss.
-  if (input.truncated) process.exitCode = 1;
+  setTelemetryStage("done");
+  if (input.truncated) {
+    process.exitCode = 1;
+    declareSoftExitOutcome("error", "migrate_export_truncated");
+  }
 
   if (options.json) {
     log.data(

packages/cli-core/src/commands/migrate/index.test.ts

@@ -6,6 +6,7 @@ import { getMode, setMode } from "../../mode.ts";
 import { createProgram } from "../../cli-program.ts";
 import { exportPlatformKeys } from "./export/registry.ts";
 import { isAssumeYes, setAssumeYes } from "./lib/assume-yes.ts";
+import { captureTelemetryPayload } from "../../test/lib/stubs.ts";
 
 let saved: string | undefined;
 beforeAll(() => {
@@ -302,3 +303,61 @@ describe("the migrate group's -y hook", () => {
     expect(await parse(["migrate", "import", missing, "--secret-key", "sk_test_x"])).toBe(false);
   });
 });
+
+describe("what the migrate hooks record for telemetry", () => {
+  let missing: string;
+  let missingDb: string;
+  beforeAll(() => {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), "clerk-migrate-telemetry-"));
+    missing = path.join(dir, "none.json");
+    missingDb = path.join(dir, "none.sqlite");
+  });
+
+  // The action is allowed to fail: only what the hooks record is under test.
+  async function payloadOf(argv: string[]) {
+    const program = createProgram();
+    program.exitOverride();
+    const { payload } = await captureTelemetryPayload(
+      "migrate",
+      () =>
+        program.parseAsync(["node", "clerk", ...argv]).then(
+          () => undefined,
+          () => undefined,
+        ),
+      { result: { outcome: "error", exitCode: 1 } },
+    );
+    return payload;
+  }
+
+  test("--app and --instance IDs are recorded before anything resolves", async () => {
+    const payload = await payloadOf([
+      "migrate",
+      "import",
+      missing,
+      "--app",
+      "app_x",
+      "--instance",
+      "ins_y",
+    ]);
+    expect(payload).toMatchObject({ app_id: "app_x", instance_id: "ins_y" });
+  });
+
+  test("an --instance alias is not an ID, and is dropped", async () => {
+    const payload = await payloadOf([
+      "migrate",
+      "import",
+      missing,
+      "--app",
+      "app_x",
+      "--instance",
+      "dev",
+    ]);
+    expect(payload).toMatchObject({ app_id: "app_x", instance_id: null });
+  });
+
+  test("an export subcommand records its platform as the source", async () => {
+    const payload = await payloadOf(["migrate", "export", "supabase", "--db-url", missingDb]);
+    expect(payload.migrate_source).toBe("supabase");
+    expect(payload.stage).toBe("migrate_export_read");
+  });
+});

packages/cli-core/src/commands/migrate/index.ts

@@ -3,6 +3,7 @@ import { isExperimentEnabled, requireExperiment } from "../../lib/experimental.t
 import { parseIntegerOption } from "../../lib/option-parsers.ts";
 import { setMode } from "../../mode.ts";
 import { setAssumeYes } from "./lib/assume-yes.ts";
+import { recordMigrateTarget } from "./telemetry.ts";
 import { registerMigrateExport } from "./export/index.ts";
 import { RUNS_DIR_DESCRIPTION, RUNS_DIR_FLAG } from "./lib/run-store.ts";
 import { run } from "./run.ts";
@@ -70,6 +71,12 @@ export function registerMigrate(program: Program, env: NodeJS.ProcessEnv = proce
     const opts = actionCommand.optsWithGlobals();
     setAssumeYes(Boolean(opts.yes));
     if (opts.json && !opts.mode) setMode("agent");
+    // A hint, so a run that fails before its target resolves still names it.
+    // Aliases like `--instance dev` and app labels are dropped.
+    recordMigrateTarget({
+      appId: typeof opts.app === "string" ? opts.app : undefined,
+      instanceId: typeof opts.instance === "string" ? opts.instance : undefined,
+    });
   });
 
   // Named, not `isDefault`. `import` and `export` are the two directions this

packages/cli-core/src/commands/migrate/run-interactive.test.ts

@@ -22,7 +22,7 @@ import {
 import fs from "node:fs";
 import os from "node:os";
 import path from "node:path";
-import { listageStubs, useCaptureLog } from "../../test/lib/stubs.ts";
+import { captureTelemetryPayload, listageStubs, useCaptureLog } from "../../test/lib/stubs.ts";
 
 const mockSelect = mock(async () => "clerk" as unknown);
 const mockText = mock(async () => "export.json" as unknown);
@@ -513,3 +513,21 @@ describe("unverified identifiers", () => {
     expect(statuses()).toEqual([undefined, ["verified", "reserved"]]);
   });
 });
+
+describe("telemetry", () => {
+  test("a source picked at the prompt is recorded", async () => {
+    const { payload } = await captureTelemetryPayload("migrate import", () =>
+      run({ input: "export.json", secretKey: "sk_test_x" }),
+    );
+    expect(mockSelect).toHaveBeenCalledTimes(1);
+    expect(payload).toMatchObject({ migrate_source: "clerk", stage: "done" });
+  });
+
+  test("declining at the confirm prompt is an abort at migrate_confirm", async () => {
+    confirmAnswer = false;
+    const { payload } = await captureTelemetryPayload("migrate import", () => run(importOptions), {
+      captureError: true,
+    });
+    expect(payload).toMatchObject({ outcome: "abort", stage: "migrate_confirm" });
+  });
+});