I.
Wiki article
Reading · 8 mindocs/testing/adapters-and-runtime-e2e
Agent Adapter And Runtime E2E reference
This strategy covers runtime paths after setup is already satisfied. It separates adapters sessions, transport carriers, genty-core programmatic sessions, and @a5c-ai/genty-platform orchestration. Harness/plugin install coverage lives in Harness And Plugin E2E(./harness-e2e.md), not in agent-platform runtime E2E.
Continue reading
Nearby pages in the same section.
Agent Adapter And Runtime E2E
This strategy covers runtime paths after setup is already satisfied. It separates adapters sessions, transport carriers, genty-core programmatic sessions, and @a5c-ai/genty-platform orchestration. Harness/plugin install coverage lives in Harness And Plugin E2E, not in agent-platform runtime E2E.
Stack Scopes
| Scope | Packages | No-model coverage | Model-backed coverage |
|---|---|---|---|
| Protocol and event contracts | packages/adapters/core, packages/adapters/gateway, packages/transport-adapter | Schema, event ordering, session lifecycle, error envelopes, reconnect behavior | Real event streams from Codex and Claude Code sessions match protocol contracts |
| Adapter translation | packages/adapters/adapters | Prompt normalization, tool-call mapping, stop reasons, model selection, fallback behavior with mock adapters | Live Codex and Claude Code adapters translate real provider output into adapter events |
| Transport runtime | packages/transport-adapter | Route matrix, proxy auth, protocol codecs, runtime env injection, passthrough forwarding, subprocess lifecycle, stream cancellation, timeout/error paths, metrics/cache snapshots | Transport-adapter carries traffic for a real external harness through adapters launch and for an genty-core-backed stream |
| Agent-core bridge | packages/genty/core | Programmatic session creation, mock provider responses, cancellation, usage accounting | Agent-core invokes a real provider and returns events compatible with adapters and agent-platform |
| Hooks muxes | packages/adapters/hooks/* | Adapter normalization, hook payload fixtures, CLI execution, approval/deny/error events | Real harness hook payloads from Codex and Claude Code normalize to the same hook contract |
| Babysitter-agent runtime | packages/genty/platform | Seam contract, phase orchestration, planner/executor mocks, run journal state, task posting, selected backend | agent-platform call/create-run/invoke uses preinstalled or mocked backends; no harness install or plugin install steps are part of this E2E |
| User surfaces | packages/adapters/webui, packages/adapters/ui, packages/adapters/tui | Playwright/Vitest against mock gateway and fixture sessions | Optional manual/live smoke against a model-backed gateway session |
No-Model Runtime Suite
The no-model runtime suite should be built first. It should include:
transport-adapterunit and E2E tests using local HTTP/subprocess fixtures for every supported exposed transport route.transport-adapterruntime tests forstartTransportMuxRuntime,applyTransportMuxToHarnessEnv, proxy auth, redacted env diffs, metrics, cache stats, passthrough path/query preservation, invalid JSON, and upstream failure mapping.adapterslaunch-plan tests for proxy forced, proxy if-needed, native/no-proxy, and proxy-forbidden cases so launch coverage proves the transport-adapter decision seam.adaptersgateway/session tests using existing mock harness scenarios.- Adapter translation tests for Codex, Claude Code, and genty-core-style event streams.
agent-platformseam and orchestration tests with mocked planner/executor calls.- WebUI and TUI session tests using fixture transcripts and mock gateway responses.
- Adapters plugin/session fixtures live in Harness And Plugin E2E; this file consumes their event fixtures only as runtime compatibility inputs.
Candidate command grouping:
npm run test --workspace=@a5c-ai/transport-adapter
npm run test --workspace=@a5c-ai/comm-adapter
npm run test --workspace=@a5c-ai/adapters-codecs
npm run test --workspace=@a5c-ai/adapters-gateway
npm run test --workspace=@a5c-ai/genty-core
npm run test --workspace=@a5c-ai/genty-platform
npm run test:e2e --workspace=@a5c-ai/genty-web-appTransport-Adapter Coverage Matrix
Transport-adapter coverage has to prove the proxy/runtime seam directly before it is used as evidence for adapters or agent-platform paths.
No-Model Transport Coverage
| Surface | Tests to add or keep | Required artifacts |
|---|---|---|
| Supported transport routes | Exercise anthropic, openai-chat, openai-responses, google, bedrock-converse, azure-foundry, vertex-native, and passthrough against fixture engines | Route transcript, response shape snapshot, invalid JSON/auth failure transcript |
| Streaming and non-streaming codecs | Verify text deltas, final events, finish reasons, usage totals, and provider-specific envelopes | Streaming event transcript and non-streaming body snapshot |
| Token and observability routes | Cover count_tokens success/unsupported behavior plus /metrics and /cache/stats | Token count transcript, metrics snapshot, cache stats snapshot |
| Runtime env injection | Call applyTransportMuxToHarnessEnv for every exposed transport | Redacted env diff proving only expected vars changed |
| Adapters launch seam | Cover resolveLaunchPlan for proxy forced, proxy if-needed, native/no-proxy, and forbidden proxy cases | Launch-plan JSON with proxyNeeded, proxyReason, and exposed transport |
| Passthrough forwarding | Preserve path/query/body, inject upstream auth safely, and map upstream failures | Redacted upstream transcript and failure envelope |
Model-Backed Transport Coverage
| Path | Valid stack | Assertion focus |
|---|---|---|
| genty-core stream bridge | genty-core provider backend -> transport-adapter -> fixture consumer | Real or credential-gated stream deltas, final event, cancellation/timeout, and usage metadata survive the proxy layer |
| External harness bridge | adapters launch <harness> <provider> --with-proxy* -> transport-adapter -> target provider | Harness receives proxy env, provider endpoint is not called directly by the harness, sentinel prompt completes, metrics show traffic |
| Babysitter-agent precondition bridge | agent-platform external-harness path only when it delegates through adapters and the selected harness requires proxy translation | Transport-adapter artifacts are supporting evidence for the bridge, while Babysitter run creation/task posting remains asserted by agent-platform tests |
Invalid Transport Claims
- Do not use transport-adapter tests to prove
babysitter harness:installorharness:install-plugin. - Do not use transport-adapter tests to prove agent-native plugin manager support.
- Do not use transport-adapter tests to prove hooks-adapter normalization.
- Do not use transport-adapter tests to prove Babysitter journal terminal state unless a higher-level runtime test also asserts that state.
Live Install Modes
The Publish workflow runs external-harness live E2E through a workflow-owned install-mode axis:
babysitter-plugingenerates plugin artifacts, installs the target withadapters install, installs the local Babysitter SDK, installs the Babysitter plugin for the harness, then launches throughadapters launchwith a/babysitter:callprompt.vanillainstalls the target withadapters install, launches throughadapters launch, and uses a non-plugin prompt so it proves adapters/transport/provider behavior; the vanillaagent-platformrows use thebabysitteradapter withBABYSITTER_HARNESS=genty-core.- Both modes use the same target mapping:
claude-code -> claude,codex -> codex,gemini-cli -> gemini,pi -> pi, and vanilla-onlyagent-platform -> babysitter.
Model-Backed Runtime Suite
The model-backed suite should prove that real providers and real harnesses behave like the no-model contracts expect.
| Test | Required real dependency | Assertion focus |
|---|---|---|
| Transport-adapter + external harness through adapters | Claude Code or Codex-compatible harness, provider credential, and adapters launch --with-proxy or --with-proxy-if-needed | Launch starts transport-adapter, harness receives proxy env, sentinel traffic uses proxy routes, stream completes, metrics snapshot increments |
| Transport-adapter + genty-core | Provider credential for genty-core backend | Agent-core deltas/final events travel through transport-adapter without adapter-only assumptions, including cancellation or timeout evidence |
| Adapters + Codex adapter | Codex CLI or configured Codex runtime and OpenAI credential | Codex output maps to adapter protocol events, including final message and usage metadata when available |
| Adapters + Claude Code adapter | Claude Code CLI plus Foundry/OpenAI credential through transport-adapter | Claude Code output maps to adapter protocol events while model traffic is proxied to GPT-5.5, including tool-call and stop metadata when available |
| Babysitter-agent full run | Provider credentials or mocked backend already available | agent-platform call/create-run creates a bounded process, plans, emits a task, posts a result, completes, and records selected backend evidence without running installer commands |
Model-backed runtime tests must upload redacted event logs, provider/harness version metadata, run IDs, and command durations.
Runtime Path Assertions
Runtime tests must declare which entry path they exercise:
| Path | Entry point | Valid backend combinations | Assertions |
|---|---|---|---|
| Adapters session | adapters run <agent> or createClient().run | Mock adapter, Claude, Codex, Gemini, Cursor, OpenCode, adapters babysitter adapter where registered | Session start/end, event ordering, provider/model config, runtime hooks, capability-gated plugin events |
| Babysitter-agent internal runtime | agent-platform call/create-run --harness genty-core | genty-core backend with mocked or live model provider | Run creation, planning, task posting, terminal state, redacted model trace |
| Babysitter-agent external-harness bridge | agent-platform call/invoke --harness <external> | Harness names mapped in adapterHarnessMap; excludes pi and genty-core | Adapters mapped events, session ID, result, selected harness, no install commands |
| Transport runtime | transport-adapter around genty-core or adapters-launched external harness traffic | Local route fixture, genty-core stream, external harness stream | Route/codec contract, proxy auth, env injection, launch proxy decision, framing, reconnect, cancellation, timeout, backpressure, metrics/cache artifact |
Do not fold plugin setup into the agent-platform runtime assertions. If a runtime job needs an installed external harness or plugin, that is a precondition supplied by a setup job and recorded separately.
Adapter-Specific Assertions
Adapter tests should assert behavior that package-local unit tests cannot prove alone:
- A session can be started, observed, cancelled, and resumed through the adapter boundary.
- Tool-call, text-delta, final-message, usage, and error events preserve ordering and session IDs.
- Adapter-specific errors are normalized before they cross gateway or transport boundaries.
- Model selection is explicit and recorded in the session state.
- Credential absence is detected before provider calls are attempted.
- Mock and live event streams conform to the same protocol fixtures.
Babysitter-Agent Whole-System Assertions
Whole-system tests for @a5c-ai/genty-platform should cover:
- process loading and validation,
- run creation,
- session binding,
- planning phase output shape,
- task effect emission,
- task result posting,
- journal rebuild/repair compatibility,
- terminal run state,
- artifact and log redaction.
The no-model version should use mocks for planner and executor behavior. The model-backed version should use the smallest possible bounded process and real model credentials or a preconfigured external harness. It must not execute harness:install or harness:install-plugin as part of the agent-platform runtime test.
Hooks-Adapter Assertions
Hooks-adapter tests should cover both adapter-local behavior and end-to-end event compatibility:
- each adapter normalizes raw harness hook payloads into the shared hook contract,
- CLI execution preserves stdin/stdout/stderr boundaries and exit codes,
- approval, denial, timeout, and malformed-payload cases are fixture-backed,
- Codex and Claude Code live hook payloads can be redacted and replayed as no-model fixtures,
- adapters UI and TUI approval surfaces consume the same normalized hook events.
Hooks-adapter live coverage should not be promoted until the no-model fixture suite covers the same event types.