{
  "id": "page:library-observability",
  "_kind": "Page",
  "_file": "wiki/library/observability.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "title": "Observability specialization (Library)",
    "displayName": "Observability specialization (Library)",
    "slug": "library/observability",
    "articlePath": "wiki/library/observability.md",
    "article": "\n# Observability specialization\n\nThe observability specialization is an **SLO-driven reliability practice**. Its flagship,\n`slo-lifecycle.js`, runs the end-to-end loop — SLO design -> telemetry-pipeline delivery ->\nalert tuning -> error-budget review cadence — with every production-affecting step closed by\nan adversarial *executed-evidence* gate and a policy-gated routed breakpoint. Incident\nhandling (detection -> mitigation -> postmortem) is **not** owned here: it lives in the\n[incident-management specialization](../incident-management/incident-lifecycle.js). The\n`sre/` subdirectory holds the cloud-specific SRE point tasks that predate the flagship.\n\n## Module index\n\n| Module | Role | Purpose |\n| --- | --- | --- |\n| `slo-lifecycle.js` | **Flagship process** | End-to-end SLO practice: recall -> inventory -> SLI selection -> error-budget policy -> telemetry pipeline -> alert tuning -> review cadence -> assert. Three adversarial gates, three policy gates. |\n| `incident-lifecycle.js` | **DEPRECATED pointer** | Header-only `@deprecated` re-export of [`../incident-management/incident-lifecycle.js`](../incident-management/incident-lifecycle.js). The incident-management flagship `@absorbs` this module's seed features. |\n| `sre/sre-base.js` | Point task | Cloud-neutral SRE persona: SLO/SLI/error-budget discipline, runbook dispatch, incident-response rigor; emits an incident-escalation breakpoint. Specialized by the three cloud modules below. |\n| `sre/sre-aws.js` | Point task | AWS-specialized SRE persona (CloudWatch, CloudTrail, Auto Scaling, multi-AZ, Route 53 failover, CloudFormation/Terraform IaC). |\n| `sre/sre-azure.js` | Point task | Azure-specialized SRE persona (Azure Monitor, App Insights, Log Analytics, availability zones, Traffic Manager, Site Recovery, ARM/Bicep, AKS scaffolding). |\n| `sre/sre-gcp.js` | Point task | GCP-specialized SRE persona (Cloud Monitoring, Cloud Logging, multi-region + load balancing, Backup for GKE, Deployment Manager/Terraform IaC). |\n\n## `slo-lifecycle.js` — phase walkthrough\n\nData flow:\n\n```\nservice-inventory\n  -> (parallel sli-selection per service) -> error-budget-policy\n  -> slo-design gate -> slo-change-approval\n  -> instrumentation-plan -> pipeline-config -> telemetry gate -> telemetry-pipeline-deploy\n  -> staged pipeline rollout (deploy-executor + stage-verification per stage, sequential)\n  -> alert-noise-audit -> (parallel alert-family-tuning per family) -> alert-tuning gate\n  -> alert-policy-change -> staged alert rollout (shadow canary -> paging-enabled full)\n  -> error-budget-review-cadence -> kip assert\n```\n\n- **P0 — kip recall.** `kipRecall` (kind `observability`) threads prior SLO targets,\n  known-good burn-rate thresholds, and past tuning outcomes into every drafting prompt as\n  `priorKnowledge`. An empty/missing store is initialized and reported `factCount 0` — a\n  fresh brain is not an error.\n- **P1 — SLO design.** `slo.service-inventory` builds the service list (declared or repo\n  discovery); `ctx.parallel.map` fans `slo.sli-selection` out per service;\n  `slo.error-budget-policy` composes the cross-service budget math, burn thresholds, and\n  freeze rules. Closed by the `slo.gate.slo-design` adversarial gate, then the\n  **slo-change-approval** policy gate. The SLO pack is only marked `adopted` when\n  `approved === true`; rejection returns a failed result — there is no unapproved-adoption path.\n- **P2 — instrumentation + telemetry pipeline delivery.** `slo.instrumentation-plan` maps\n  each approved SLI to concrete emit points; `slo.pipeline-config` generates the\n  collector/pipeline configs and reports the exact `validationCommand`. Closed by the\n  `slo.gate.telemetry-pipeline` gate (critics **execute** the dry-run/validate), then the\n  **telemetry-pipeline-deploy** policy gate. `slo.pipeline-deploy-executor` runs **only**\n  under `approved === true`, followed by sequential staged rollout with per-stage\n  `slo.stage-verification`.\n- **P3 — alert tuning.** `slo.alert-noise-audit` classifies existing alerts into families;\n  `ctx.parallel.map` fans `slo.alert-family-tuning` out per family onto multi-window\n  multi-burn-rate alerting derived from the approved error-budget policy. Closed by the\n  `slo.gate.alert-tuning` gate (critics **execute** rule lint + burn-rate simulation), then\n  the **alert-policy-change** policy gate. `slo.alert-rollout-executor` runs **only** under\n  `approved === true`, staged shadow -> paging.\n- **P4 — error-budget review cadence.** `slo.error-budget-review-cadence` composes the\n  recurring budget-burn report template and review cadence (weekly while any service burned\n  >25% of budget last window, else monthly). Documentation, not a production change — **no\n  breakpoint** (sparse-breakpoint rule).\n- **P5 — kip assert.** Learned facts written back under kind `observability`: per-service\n  adopted SLO targets, per-family tuned burn-rate thresholds, and a run-outcome fact.\n\n### Inputs / outputs (summary)\n\nInputs: `services?` / `discover?`, **`telemetryStack` (required — absent throws)**,\n`alertSources?`, `sloHorizon?` (default `{ windowDays: 28 }`), `maxFixAttempts?` (2),\n`kipEnabled?` (true), `kipDir?` (`.a5c/kip`), `kipModel?` (`sonnet`), `artifactsDir?`.\n\nOutputs: `{ success, sloPack, sloDesignGate, telemetry, alerting, reviewCadence,\nautoApprovals (always present), kipFactsAsserted, artifacts, metadata }`.\n\n### Example invocation\n\n```js\nimport { process as sloLifecycle } from './slo-lifecycle.js';\n\nconst result = await sloLifecycle({\n  services: [{ name: 'checkout', tier: 'critical', userJourneys: ['place-order'] }],\n  telemetryStack: {\n    metricsBackend: 'prometheus',\n    tracingBackend: 'tempo',\n    collector: 'otelcol',\n    alertRuleFormat: 'prometheus',\n  },\n  alertSources: [{ system: 'alertmanager', ref: 'prod' }],\n  sloHorizon: { windowDays: 28 },\n}, ctx);\n```\n\n## Policy-gated actions\n\n| actionId | expert | phase | never auto-approves |\n| --- | --- | --- | --- |\n| `slo-change-approval` | `service-owner` | P1 | Yes — SLO adoption is a production-policy change; any harness auto-approval is surfaced in `autoApprovals`. |\n| `telemetry-pipeline-deploy` | `platform-engineering-lead` | P2 | Yes — production pipeline deploys never auto-approve. |\n| `alert-policy-change` | `sre-lead` | P3 | Yes — production alerting changes never auto-approve. |\n\nEach is a `routedBreakpoint` with `breakpointId` equal to the `actionId`, an accountable-role\nexpert, and a `policy-gated` tag — ready for the `adapters/policy` YAML gating layer to\nenforce readiness. The frozen `POLICY_GATE_ROUTING` table + throwing `gateExpert(actionId)`\nlookup guarantees no fallback expert is ever substituted.\n\n## Adversarial gates\n\n| gateId | critics | executed-evidence requirement |\n| --- | --- | --- |\n| `slo.gate.slo-design` | `slo-math-critic`, `slo-coverage-critic` | Recompute the error budget from each target+window; run the measurement-query validator; every archetype must be in `SLI_CATALOG`. |\n| `slo.gate.telemetry-pipeline` | `pipeline-dryrun-critic`, `instrumentation-fidelity-critic` | **Execute** the reported `validationCommand` (collector validate / dry-run) against every config path and quote outputs. |\n| `slo.gate.alert-tuning` | `alert-rule-executor-critic`, `noise-regression-critic` | **Execute** the rule linter (promtool/vendor) over `rulePaths` **and** run the burn-rate simulation from `simulationSpec`. |\n\nEvery gate enforces `evidence` `minItems: 1`; a `passed: true` verdict with empty evidence is\ncoerced to a protocol failure by the combinator. Each gate escalates internally via its own\n`<gateId>.gate-escalation` routed breakpoint when the fix budget is exhausted.\n\n## Staged rollout semantics\n\nBoth rollouts share the frozen `TELEMETRY_ROLLOUT_STAGES = ['canary', 'full']` table with the\nthrowing `stagePolicy(stage)` lookup:\n\n- **Telemetry:** `canary` (10% scope, 30m bake) -> `full` (100% scope, 60m bake).\n- **Alerts:** `canary` = shadow-mode / paging-disabled -> `full` = paging-enabled.\n\nStages are promoted **sequentially**. A failed `slo.stage-verification` (probes actually\nexecuted, `evidence` `minItems: 1`) **halts promotion** and returns a failed result with the\nstage recorded — there is no silent continue.\n\n## kip memory\n\nKind `observability`. **Recalled** at P0: prior SLO targets, known-good burn-rate thresholds,\npast tuning outcomes. **Asserted** at P5: per-service `has-slo` facts, per-family `tuned-alert`\nfacts, and a `run-outcome` fact (gates, attempts, auto-approvals). Future runs recall these to\nseed drafting and skip re-deriving known-good thresholds.\n\n## Deprecation notice\n\n`incident-lifecycle.js` in this directory is a **header-only `@deprecated` pointer** that\nre-exports the incident-management flagship. The old 230-line implementation duplicated the\nincident lifecycle; the incident-management flagship `@absorbs` every seed feature\n(single-workflow lifecycle, severity matrix, non-incident early exit, timeline accumulation,\ncomms phase model, 3-pass diagnosis, SLO breach detection, follow-up issues). Existing callers\nkeep working against the flagship contract (same signal shape, superset of options). The\nreduction was approved through the `incident-lifecycle-deprecation` policy gate\n(expert `library-maintainer`) at generation time. The re-export has no `try`/`catch` or\nconditional import — if the flagship moves, the import breaks loudly.\n\n## Hard rules recap\n\n- **Style-A agent tasks only** — zero `kind: 'shell'` subtasks.\n- **No fallbacks** — throwing lookups (`gateExpert`, `stagePolicy`), `telemetryStack` required,\n  SLI archetypes outside `SLI_CATALOG` are gate issues.\n- **Evidence `minItems: 1`** on every evidence-carrying and verification schema.\n- **Guarded executors** — deploy/rollout executors exist only inside `approved === true`\n  branches; a rejected gate returns a failed result, never a degraded alternate path.\n- **Orchestrator-owned timeline** — agents never write the timeline; combinators are imported\n  from `../common-utilities/`, never re-implemented.\n",
    "documents": [
      "specialization:observability"
    ]
  },
  "outgoingEdges": [
    {
      "from": "page:library-observability",
      "to": "specialization:observability",
      "kind": "documents"
    }
  ],
  "incomingEdges": [
    {
      "from": "page:index",
      "to": "page:library-observability",
      "kind": "contains_page"
    }
  ]
}