# Chart-review prompts (LangSmith hub)

**Prompts are code.** You edit `snapshots/<name>.json` in a PR like any other file, and merging
publishes it to the LangSmith hub and points `staging` at it. The one step that is not automatic is
promotion to **production**, which stays a deliberate human action in the LangSmith UI.

A prompt resolves from one of three places, in this order:

1. **`snapshots/<name>.local.json`** — your per-developer override (gitignored). Wins locally, is
   never read in a deployed build, and is ignored under vitest so it cannot decide a test.
2. **The hub**, at the environment's tag — pulled live when `LANGSMITH_PROMPTS=true`.
3. **`snapshots/<name>.json`** — the committed snapshot. What a PR preview serves, the cold-start
   fallback when the hub is unreachable, and the audit trail of every prompt change.

The CLI engine (hydrate / publish / migrate / sync / push) lives in `@repo/ai/prompts/scripts`
(`createPromptScriptKit`). This package supplies only config: snapshot dir, hub name filter, default
tags. `@repo/ai-agents` has a parallel driver for its own snapshot dir and hub namespace.

## Which version each environment serves

| Where | Resolves | Moved by |
| ----- | -------- | -------- |
| local `pnpm dev` | your override, else the committed snapshot | you |
| local `dev:remote` / `prod:remote` | override hydrated from `:staging` / `:production` | `prompts:hydrate`, at launch |
| **PR preview** | the **committed snapshot on that branch** | the PR |
| **dev-v2** | hub **`:staging`** | **CI, on merge to main** |
| **production** | hub **`:production`** | **a human, in the LangSmith UI** |

So a PR preview shows the prompt that PR proposes, dev-v2 shows what main says, and production
shows what someone deliberately promoted. Those are three different questions and three different
answers.

Propagation is bounded by the SDK's in-process prompt cache — **5 minutes** TTL with a 60s
background refresh (`ttlSeconds ?? 5 * 60`), stale-while-revalidate. Cold instances pick a change
up immediately; warm ones within the window.

## The everyday loop

```
edit snapshots/<name>.json         # in a branch, like any other file
open a PR                          # CI comments the publish plan; the preview serves your edit
merge                              # CI publishes to hub :latest and moves `staging` → dev-v2 has it
promote to Production              # LangSmith UI, when you mean it
```

You do **not** need to touch the hub by hand for any of this. If you want a faster inner loop than
editing the committed file, create an override (`prompts:hydrate <name>`), edit
`<name>.local.json`, and the next server call picks it up without a restart — then fold the change
back into the committed snapshot before you push.

## CI refuses to bury work

Publishing compares hub `:latest` against the snapshot **as it stood at the merge's base**. If the
hub holds something that is neither that nor your branch — someone edited the prompt in the UI —
the publish stops and the job fails rather than overwriting it. Recover by pulling that commit down:

```bash
pnpm sync-prompt-snapshots <name>   # hub :production → committed snapshot; review the diff, commit
```

The push also pins `parentCommitHash`, so a commit landing between the check and the push is
rejected by the API rather than silently ending up underneath ours.

## Commands

Run from `packages/legacy` (or `pnpm --filter @repo/legacy run …`). All load `apps/web`'s env, so
you just need `LANGSMITH_API_KEY` in `apps/web/.env.local`.

| Direction | Command | When |
| --------- | ------- | ---- |
| committed → hub `:latest` + `staging` | `prompts:publish [--dry-run] [--changed] [--base <ref>] [name…]` | what CI runs on merge; run `--dry-run` yourself to see the plan |
| hub → override | `prompts:hydrate [--force] [name…]` | start editing a prompt with hot reload |
| (delete overrides) | `prompts:refresh` | drop overrides so the next hydrate re-pulls |
| override → hub `:latest` | `prompts:migrate [name] [--yes]` | publish a local override without a PR (diff + confirm) |
| hub → committed | `sync-prompt-snapshots [name]` | **rescue only** — recover a UI edit CI refused to bury |
| committed → hub | `push-prompts [name…]` | ⚠️ **disaster recovery only** — no checks, no diff, no base |

`sync-prompt-snapshots` used to be routine hygiene, run after every promotion. It is not any more:
git is where prompts are authored, so pulling the hub down over your snapshots is a recovery
action, not a habit.

## Promotion to production

Promote in the LangSmith UI: open the prompt, pick the commit, **Promote → Production**.

This is a **policy**, not a technical limit. `PATCH /api/v1/repos/-/<name>/tags/production` moves
that tag as willingly as any other, and the docs' "reserved tag" note is a UI restriction only. The
publish command hard-codes `staging` and has no parameter that could ever hold `production`, so CI
cannot promote to prod even by mistake — that is the enforcement.

After promoting, `pnpm sync-prompt-snapshots <name>` refreshes the committed fallback so the
cold-start path matches what prod serves.

## Conventions

- **Naming:** `chart-review-<prompt>`, kebab-case. Slashes are illegal in a hub name (they mean
  `owner/name`); domain grouping is the `chart-review` **tag**.
- **Format:** f-string. Literal braces in a prompt (example JSON, say) must be escaped `{{` / `}}`;
  `{var}` are template variables.
- **Model config travels with the prompt.** A snapshot's `model` is pushed as a bound `ChatOpenAI`,
  so the hub commit carries the same levers the runtime resolves. Levers reach the provider only
  when set — never add a `temperature` to a GPT-5-series prompt, which rejects anything but 1.
- The static `import … .json` makes the committed snapshot a build-time dependency: a *missing*
  snapshot fails the build, and a *malformed* one is caught by `snapshots.test.ts`.

## Gotchas

- **`sh: dotenv: command not found`** — stale `node_modules`; run `pnpm install`.
- **Auth errors / nothing pulled** — `LANGSMITH_API_KEY` missing from `apps/web/.env.local` (or
  `.env.dev` / `.env.prod` for the `:remote` variants).
- **`pnpm dev` does not hydrate.** Only `dev:remote` and `prod:remote` do. A plain `pnpm dev` uses
  whatever override files already exist, else the committed snapshot.
- **An override outlives the mode that created it.** Hydrate is pull-if-missing, so a `.local.json`
  written by `dev:remote` is still there under plain `pnpm dev`. `prompts:refresh` clears them.