Circle V2 API Docs
    Preparing search index...

    Type Alias RunExpandQuestionEvalOptions

    type RunExpandQuestionEvalOptions = {
        datasetName: string;
        experimentPrefix?: string;
        injectIntent: boolean;
        judgeModel?: ChatOpenAI;
        maxConcurrency?: number;
        metadata?: Record<string, unknown>;
        model?: ChatOpenAI;
        numRepetitions?: number;
    }
    Index

    Properties

    datasetName experimentPrefix? injectIntent judgeModel? maxConcurrency? metadata? model? numRepetitions?

    Properties

    datasetName: string

    LangSmith dataset of { question, goldIntent, goldPlanExpectation, kind, ... } examples.

    experimentPrefix?: string
    injectIntent: boolean

    The A/B lever. false = planner runs blind (baseline). true = classify the question's DateIntent LIVE (classifyDateIntent) and feed it to the planner — exactly the shipped prod behavior, so the arm measures the real change (classifier included), not an idealized gold-intent injection. The example's gold intent is the judge's REFERENCE, not the input.

    judgeModel?: ChatOpenAI

    Judge model; pinned separately so the measuring instrument stays fixed across arms.

    maxConcurrency?: number
    metadata?: Record<string, unknown>
    model?: ChatOpenAI

    Planner model; defaults to the chart-review default (gpt-4.1, temp 0 for reproducibility).

    numRepetitions?: number

    Run each example N times to measure per-example variance (a "lift" must beat the noise).

    Settings

    Member Visibility

    On This Page

    Properties