Pushes one human-graded audit run into the LangSmith eval dataset — one example per question,
keeping the latest response per question (a question re-run within the run dedupes to its newest
grade) and the persisted asked question text (falling back to the template's current text).
The gold status is the reviewer's grade where they gave one, and the AI's only where they did not.
New examples land in the incoming split (staged-rollout quarantine) plus the patient and
template axis splits (p<patientId>, t<templateId>) — axis splits keep patient- and
template-scoped experiment selections each a single split name (the splits param is a UNION,
so a combined name would compose worse); exact source-run provenance stays in metadata.
Promotion to golden is a human decision in the LangSmith UI, so pushing can never silently
shift a baseline.
PHI note: question/answer text goes straight to LangSmith (BAA in place) and is never logged.
Pushes one human-graded audit run into the LangSmith eval dataset — one example per question, keeping the latest response per question (a question re-run within the run dedupes to its newest grade) and the persisted asked question text (falling back to the template's current text).
The gold status is the reviewer's grade where they gave one, and the AI's only where they did not.
New examples land in the
incomingsplit (staged-rollout quarantine) plus the patient and template axis splits (p<patientId>,t<templateId>) — axis splits keep patient- and template-scoped experiment selections each a single split name (thesplitsparam is a UNION, so a combined name would compose worse); exact source-run provenance stays in metadata. Promotion togoldenis a human decision in the LangSmith UI, so pushing can never silently shift a baseline.PHI note: question/answer text goes straight to LangSmith (BAA in place) and is never logged.