MLOps

Flagship model-lifecycle for the library: dataset governance intake (parallel per-dataset lineage/consent/retention checks) -> eval-harness design -> executed training/eval runs -> an adversarial eval-review gate that RE-RUNS a sampled eval and diffs metrics -> a policy-gated model promotion with an executed serving smoke -> drift-monitoring setup with an executed drift-detection stub -> an adversarial drift-review gate -> a drift path with severity-routed escalation and a policy-gated rollback/retirement -> kip-backed model-registry memory. This is a brand-new specialization directory (verified: no prior mlops dir anywhere in library/).

Composition map — callable upstream training stages (NOT superseded)

model-lifecycle.js consumes a **trained candidate** (modelVersion + artifactRef) and owns the governance / eval / promotion / drift / retirement lifecycle **around** it. Training itself is a pre-bar point task that can be delegated to either data-science-ml near-miss:

These are mapped as callable upstream stages — **NOT superseded, NOT re-implemented**, nothing deprecated. mlo.training-run (P3, optional, gated on retrain) is the delegation seam.

Module table — `model-lifecycle.js` exports

Style note

All tasks are Style-A kind: 'agent' (zero kind: 'shell'), with per-effect io paths (tasks/<effectId>/input.json|result.json) and labels, and every gate / verification / executed-run output schema declares evidence { type: 'array', minItems: 1 }. Gate combinators (routedBreakpoint, adversarialGate, kipRecall, kipAssert) are imported from `../common-utilities/routed-gate-combinators.js`, not redefined. Timeline, breakpointsHit, and autoApprovals are accumulated in the orchestrator only — agents never write the timeline.

Stage model

Lifecycle stages (`MODEL_STAGES`, verbatim)

['development', 'staging', 'production'] — promotion advances one stage toward production; promotionTargetStage must be a member beyond development (default production). An unknown value throws (no fallback stage).

Promotion table (`STAGE_PROMOTION_POLICY`, verbatim)

Lookups go through stagePromotionPolicy(stage), which **throws** on an unknown stage — there is no fallback promotion policy.

Drift routing table (`DRIFT_ROUTING`, verbatim)

All lookups (stagePromotionPolicy, driftRouting, assertDriftSeverity, governanceCheckLabel) **throw** naming the source on any unknown enum — there are no fallback rows anywhere.

Policy-gated actions

Three actions are policy-gated. Convention: **breakpointId = actionId**, strategy single.

**Fail-closed posture:** there is no alternate execution path around a gate — the retention, promotion, and rollback executors are invoked **only** inside gate.approved === true branches, each with an explicit code comment that no other call site exists. **No gate in this process sets autoApproveAfterN**, and all three policy gates carry explicit code comments stating it must never be added. Any harness-level auto-approval is surfaced in outputs.autoApprovals ({ breakpointId, phase, at }), which is **always present** in outputs, possibly empty.

Quality gates

`mlo.eval-review` (P4) — RE-RUNS a sampled eval, diffs metrics

Runs over the eval report (per-suite metrics + deterministic threshold pass/fail), with the harness and evalSuites reachable in context. Failure (including an owner-rejected escalation) ends the run **before** model-promotion-approval is ever raised (fail closed).

`mlo.drift-review` (P6) — RE-RUNS the drift-detection stub, fires the hooks

Runs over the drift-monitor spec + executed drift-detection run. A live breach in the executed run routes into the P7 drift path.

**IRON-LAW (executed evidence only):** the eval-review gate must re-run a sampled eval and diff; the drift-review gate must re-run the drift-detection stub and fire the hooks — reading a report or spec is not evidence, and passed:true with empty evidence is rejected by the combinator. Fix budget: maxFixAttempts (default 2) rounds of the built-in gateFixerTask; on exhaustion the combinator escalates to the owner via a routed breakpoint (mlo.eval-review.gate-escalation / mlo.drift-review.gate-escalation). A model **never promotes on a read-only review**.

Drift path

Entered only when the executed drift-detection run surfaces a live breach. driftTriageTask classifies SEV1..SEV4 grounded in the executed drift metrics (recommendation only). Then DRIFT_ROUTING:

• **SEV1 / SEV2** -> straight to the model-rollback-approval gate.
• **SEV3 / SEV4** -> one non-policy mlo.drift.remediation-choice breakpoint first (accept-drift/roll-forward vs rollback, expert ml-engineering-lead) — the **only non-policy breakpoint** in this process (sparse-breakpoint rule: the call is genuinely ambiguous at low severity). A roll-forward response ends the run success:false (drift accepted, no rollback gate raised); any other response proceeds to the gate.

On model-rollback-approval approved, rollbackExecutionTask restores exactly currentProductionRef and rollbackVerificationTask executes serving probes proving the restored version serves and the drift symptom is gone (bounded by driftPolicy.maxRollbackAttempts, default 1). **Retirement** of a superseded prior production version is the same gate surface (model-rollback-approval, tagged retirement): a clean promotion to production that supersedes a non-null currentProductionRef requests retirement of the old version through this gate.

kip model-registry memory

- { predicate: 'has-version', object: <modelVersion> } - { predicate: 'outcome', object: 'promoted'|'failed'|'rolled-back', props: { evalReviewPassed, promoted, driftDetected } } - one per eval suite: { predicate: 'eval-metric', object: <suite>, props: { thresholdsPassed } } - { predicate: 'promotion-decision', object: 'promoted-to-<stage>'|'not-promoted', props: { approved, verified } } - only when the drift path ran: { predicate: 'drift-incident', object: <triage.summary>, props: { severity, rollbackVerified } } and { predicate: 'rollback-lesson', object: '<severity>: <rationale>' }

• **Recall (P0)**: kipRecall(ctx, { kipDir, topic: 'model registry: <modelName>@<modelVersion>', kipModel, kind: 'mlops-model-registry' }) — prior model performance, promotion decisions, and drift incidents threaded as priorKnowledge into every downstream agent. An empty store is initialized and reported as factCount: 0, never an error.
• **Assert at close (P8)**, facts built deterministically in the orchestrator (never in an agent), subject model:<modelName>@<modelVersion>:

Both touchpoints are wrapped in if (kipEnabled) (default true). The assert facts are unconditionally non-empty when reached — the has-version and outcome facts always exist.

Inputs / outputs reference

Mirrors the JSDoc @inputs / @outputs in model-lifecycle.js. Required: model { modelName, modelVersion, artifactRef }, a non-empty datasets[] (each { name, uri, purpose }, purpose train|eval|holdout), a non-empty evalSuites[] (each { name, dataset, metrics[] }), and a regressionThresholds object mapping every referenced metric to a threshold (a referenced metric with no threshold **throws** before any run). Optional: promotionTargetStage (default production), retrain (default false), driftPolicy { maxRollbackAttempts, baselineWindow, checkWindow }, maxFixAttempts (default 2), kipEnabled/kipDir/kipModel, artifactsDir.

Usage

babysitter run:create \
  --process-file library/specializations/mlops/model-lifecycle.js \
  --inputs '{
    "model": {
      "modelName": "fraud-scorer",
      "modelVersion": "2.4.0",
      "artifactRef": "s3://models/fraud-scorer/2.4.0/model.pt",
      "currentProductionRef": "s3://models/fraud-scorer/2.3.1/model.pt"
    },
    "datasets": [
      { "name": "txn-train", "uri": "s3://data/txn/train", "purpose": "train", "lineageRef": "dvc://txn@train" },
      { "name": "txn-holdout", "uri": "s3://data/txn/holdout", "purpose": "holdout", "consentBasis": "contract" }
    ],
    "evalSuites": [
      { "name": "accuracy-suite", "dataset": "txn-holdout", "metrics": ["auc", "precision"] }
    ],
    "regressionThresholds": {
      "auc": { "min": 0.9, "baseline": 0.94, "maxRegression": 0.01 },
      "precision": { "min": 0.85 }
    },
    "promotionTargetStage": "production"
  }'

For this run: the two datasets clear lineage/consent/retention in parallel (a flagged retention action would gate dataset-retention-action with the data-governance-officer), the eval harness is authored and every metric is checked for a threshold, accuracy-suite runs against the candidate and is scored deterministically, the mlo.eval-review gate re-runs a sampled eval and diffs it, model-promotion-approval (ml-engineering-lead) gates promotion to production, an executed serving smoke verifies it, mlo.drift-review re-runs the drift-detection stub, and — because a prior currentProductionRef exists — retirement of 2.3.1 is requested through the model-rollback-approval gate tagged retirement.

Non-interactive runs

Nothing policy-gated auto-approves **by design** — no gate in this process sets autoApproveAfterN, and the three policy gates must never gain it. If a non-interactive harness auto-approves a breakpoint at its own level, that approval is recorded in outputs.autoApprovals as { breakpointId, phase, at } with its phase provenance, so the fail-closed posture stays auditable. autoApprovals is always present in outputs, even when empty.