{
  "id": "page:docs-agent-mux-archive-comparisons-16-reference-comparison",
  "_kind": "Page",
  "_file": "wiki/docs/agent-mux/archive/comparisons/16-reference-comparison.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/agent-mux/archive/comparisons/16-reference-comparison.md",
    "sourceKind": "repo-docs",
    "title": "Reference Comparison",
    "displayName": "Reference Comparison",
    "slug": "docs/agent-mux/archive/comparisons/16-reference-comparison",
    "articlePath": "wiki/docs/agent-mux/archive/comparisons/16-reference-comparison.md",
    "article": "\n# Reference Comparison\n\n> Archived comparison document. Preserved for historical context; not part of the current normative `reference/` contract.\n\nSurvey of comparable open-source projects and parity / gap analysis vs agent-mux.\n\n## Projects surveyed\n\n| Project | Lang | Shape | Adapters | Relevance |\n|---|---|---|---|---|\n| [paperclipai/paperclip](https://github.com/paperclipai/paperclip) | TS | SDK + adapters | claude, codex, cursor, gemini, opencode, openclaw, pi | Closest structural peer |\n| [BloopAI/vibe-kanban](https://github.com/BloopAI/vibe-kanban) | Rust | Kanban UI executor layer | claude, copilot, cursor, codex, gemini, qwen, droid, opencode, amp | Broadest adapter set |\n| [Th0rgal/sandboxed.sh](https://github.com/Th0rgal/sandboxed.sh) | Rust | Sandbox wrapper | claudecode, gemini, codex, opencode, amp | systemd-nspawn isolation |\n| [hiyenwong/matop](https://github.com/hiyenwong/matop) | Rust | Agent monitor | claude-code, openclaw, opencode | Monitoring pattern |\n| [SihaoLiu/ai-usage](https://github.com/SihaoLiu/ai-usage) | Rust | Usage analytics | claude, codex, gemini | Pricing + usage aggregation |\n| [fotoetienne/gru](https://github.com/fotoetienne/gru) | Rust | Multi-agent runner | claude, codex | `AgentBackend` trait |\n| [ryoppippi/ccusage](https://github.com/ryoppippi/ccusage) | TS | Usage CLI | claude, codex, opencode, pi, amp | JSONL session reader family |\n\n## Key patterns observed\n\n### Spawn flags (claude)\n\nAll references converge on the same critical flags: `--print --verbose --output-format stream-json --include-partial-messages` plus `--session-id` / `--resume`. **We have** `--print` and `--output-format jsonl`. **Gap**: we are missing `--verbose` and `--include-partial-messages`, and we use `jsonl` instead of `stream-json`. Impact: we may miss partial streaming content blocks.\n\n### Session resume distinction\n\ngru separates `build_claude_command()` (new, `--session-id`) from `build_claude_resume_command()` (`--resume`) to avoid \"session already in use\" errors. **We** use `--session-id` unconditionally. **Gap**: resume path should prefer `--resume <id>` when the session already exists on disk.\n\n### Event mapping (claude)\n\ngru's `claude_backend.rs` buffers `ContentBlockStart(ToolUse)` until `ContentBlockStop`, then emits one `ToolUse` event with formatted summary. **We** emit `tool_call_start` immediately on any `tool_use`/`tool_call`. **Nuance**: ours is fine because `inputAccumulated` is a string; gru's is more ergonomic for terminal UIs but not for SDK consumers.\n\n### Codex\n\ngru maps `turn.failed` event → error with nested message extraction; falls back to \"Turn failed\" / \"Unknown Codex error\". **Our** codex-adapter should audit that fallback coverage. (Followup task.)\n\n### Usage/cost parsing\n\nccusage and ai-usage both parse **JSONL session files directly** for token counts (cache-creation vs cache-read tracked separately) and compute cost client-side from a `pricing.json`. **We** have `assembleCostRecord` but only emit cost from the terminal `result` event. **Gap**: we don't separately attribute cache-creation vs cache-read tokens.\n\n### Sandboxing\n\nsandboxed.sh uses `systemd-nspawn` for isolation. **We** support `local`, `docker`, `kubernetes`. **Potentially add**: `nspawn` mode as a 4th invocation target for Linux users who want kernel-level isolation without Docker.\n\n### Rust trait = our BaseAgentAdapter\n\ngru's `AgentBackend` trait (build_command / build_resume_command / parse_event / build_interactive_resume_command) maps 1:1 to our adapter surface. We additionally cover: hooks, plugins (MCP), auth detection, session file discovery, config read/write. **We are a superset.**\n\n### paperclip adapter set\n\npaperclip's 7 adapters: claude-local, codex-local, cursor-local, gemini-local, opencode-local, openclaw-gateway, pi-local. **We match** all 7 by name and add: hermes, omp, copilot, agent-mux-remote (11 total). **We are a superset.**\n\n### vibe-kanban adapter set\n\nvibe-kanban covers: claude, copilot, cursor, codex, gemini, qwen, droid, opencode, amp. **Missing from us**: qwen, droid, amp. **Candidate for future adapters.**\n\n## Concrete gap list (actionable)\n\n1. **claude-adapter flags**: add `--verbose`, `--include-partial-messages`, switch `--output-format` to `stream-json`. Gate on capability + add tests.\n2. **claude resume**: implement `--resume <id>` path separate from `--session-id` when session exists.\n3. **cost attribution**: break out cache-creation vs cache-read tokens in `CostRecord`.\n4. **codex error fallback**: audit `parseEvent` for missing-message fallbacks (\"Turn failed\" / \"Unknown error\").\n5. **new adapters to consider**: qwen, droid, amp — each exists in vibe-kanban and ccusage.\n6. **invocation mode: nspawn** — Linux-only sandboxing option alongside docker/k8s.\n\n## Security / scalability notes\n\n- None of the references do full privilege-dropping in `local` mode; sandboxed.sh delegates to nspawn. We match this baseline.\n- All references read session JSONL lazily and stream line-by-line — we do the same (`parseJsonlSessionFile`).\n- No reference we saw does MCP plugin lifecycle management — **we are ahead** here.\n- No reference we saw exposes a CLI surface as broad as ours (`amux` has run, sessions, hooks, plugins, detect, doctor, config).\n\n## Conclusion\n\nagent-mux is a **structural superset** of every reference project surveyed. The actionable gaps are narrow and mostly in the **claude streaming flags** and **cost attribution granularity**. Filed as followups in the issue tracker.\n\n## Appendix: Per-file deep-dive (2026-04-12)\n\nSource-level comparison of seven reference executors against our adapters in `packages/adapters/src/`. Line numbers below reference the upstream files fetched on 2026-04-12.\n\n### 1. vibe-kanban `crates/executors/src/executors/claude.rs` vs `claude-adapter.ts`\n\n**Spawn flags (upstream L244-275):** `-p`, `--permission-prompt-tool=stdio`, `--permission-mode={mode}`, `--disallowedTools=AskUserQuestion`, `--dangerously-skip-permissions`, `--model`, `--effort`, `--agent`, `--verbose`, `--output-format=stream-json`, `--input-format=stream-json`, `--include-partial-messages`, `--replay-user-messages`. Router mode wraps `npx -y @musistudio/claude-code-router@1.0.66 code`.\n\n**Our `buildSpawnArgs` (L156-201):** only emits `--output-format`, `--model`, `--session-id`, `--max-turns`, `--dangerously-skip-permissions`, `--system-prompt`, `--print`. Missing: `--verbose`, `--input-format=stream-json`, `--include-partial-messages`, `--replay-user-messages`, `--permission-prompt-tool=stdio` / `--permission-mode`, `--disallowedTools`, `--effort`, `--agent`, and the claude-code-router backend entirely.\n\n**parseEvent branches (upstream ClaudeJson L1641-1732, 13 variants):** System, Assistant, User, ToolUse, ToolResult, StreamEvent (message_start/content_block_start/content_block_delta/message_stop), Result, ApprovalRequested, ApprovalResponse, QuestionResponse, ControlRequest/Response/CancelRequest, RateLimitEvent, Unknown.\n\n**Our parseEvent (L203-271):** only handles `assistant|text`, `tool_use|tool_call`, `tool_result`, `thinking`, `error`, `result`. Missing: `system`, `user`, `stream_event` (no message_start / content_block_delta unwrapping — real Claude Code stream-json will be swallowed), `approval_requested`, `approval_response`, `question_response`, `control_request`, `control_response`, `control_cancel_request`, `rate_limit_event`, Unknown fallback.\n\n**Error mapping:** upstream L48 suppresses `[WARN] Fast mode requires the native binary`; L1572-1581 strips ANSI and categorizes non-JSON stderr as SystemMessage. Ours: no suppression list, no non-JSON fallback.\n\n**Session resume (upstream L320-336):** `--resume <id>` + optional `--resume-session-at <uuid>`. Ours uses `--session-id` for both new and resume (L170) — gru `claude_runner.rs` L31-32 comment warns this causes \"session already in use\" errors. No `--resume-session-at` support.\n\n**Auth (upstream L676-694):** reads `~/.claude.json` mtime as availability signal; `env_remove(\"ANTHROPIC_API_KEY\")` (L411) when `disable_api_key=true`. Ours reads only `ANTHROPIC_API_KEY` env; no file-mtime signal, no env_remove toggle, and `authFiles` lists `.claude/settings.json` while upstream uses `~/.claude.json`.\n\n### 2. vibe-kanban `codex.rs` vs `codex-adapter.ts`\n\n**Spawn (upstream L378-387):** `npx -y @openai/codex@0.116.0 app-server [--oss]` plus `apply_overrides`. Env L515-521: `NPM_CONFIG_LOGLEVEL=error`, `NODE_NO_WARNINGS=1`, `NO_COLOR=1`, `RUST_LOG=error`.\n\n**Our `buildSpawnArgs` (L142-165):** `--model`, `--full-auto`, `--quiet <prompt>` one-shot. Upstream uses long-running `app-server` JSON-RPC. Missing: `--oss`, npm/node silence env vars, app-server transport.\n\n**parseEvent:** upstream delegates to JsonRpcPeer. Ours parses `message|text`, `function_call|tool_call`, `function_call_output|tool_result`, `error`. Real codex exec --json (gru `codex_backend.rs` L189-253) emits `thread.started`, `turn.started`, `turn.completed` (usage), `turn.failed`, `item.started` (command_execution/file_change/message/generic), `item.completed`, `error` — **none of these are handled**.\n\n**Error mapping (upstream L552-583):** BrokenPipe suppression, AuthRequired distinct variant, \"missing stdout/stdin\" Io errors, `launch_error`. Ours: no categorization.\n\n**Session resume (upstream L430-452):** `thread_start` vs `thread_fork(fork_params_from(session_id, ...))`. Ours has **no resume plumbing** despite `canResume: true`. Gru uses `codex exec resume --last --json --full-auto` — we do not emit `resume`.\n\n**Auth (upstream L19-27, 465-468, 631-651):** `CODEX_HOME` env → `~/.codex`, `auth.json` mtime for availability, `get_account()` RPC checks `requires_openai_auth`. Our adapter only reads `OPENAI_API_KEY`; missing `CODEX_HOME`, `auth.json` reading; config path should be `config.toml` not `.codex/config.json`.\n\n### 3. vibe-kanban `cursor.rs` vs `cursor-adapter.ts`\n\n**Spawn (upstream L115-129):** `-p`, `--output-format=stream-json`, `--force` OR `--trust`, `--model`. Ours (L122-141): only `--model` and `--prompt`. Missing all stream-json and trust flags. cliCommand is `'cursor'` (L36) but upstream binary is `cursor-agent` — **wrong binary name**.\n\n**parseEvent (upstream L270-500):** System (model-reporting), User (no-op), Assistant (buffer+coalesce), Thinking, ToolCall (Started/Completed subtypes), Result (skip), Unknown → SystemMessage. Ours (L143-181): only `text|message`, `tool_call`, `error`. Missing thinking stream, tool_call split, assistant coalescing, System model-report, Unknown fallback.\n\n**Error mapping (upstream L214-243):** `CURSOR_AUTH_REQUIRED_MSG` → `SetupRequired`. Ours: no auth-stderr detection.\n\n**Session resume (upstream L163-186):** `--resume <session_id>`. Ours emits nothing despite `canResume: true`.\n\n### 4. vibe-kanban `opencode.rs` vs `opencode-adapter.ts`\n\n**Spawn (upstream L109-111):** `npx -y @anomalyco/opencode serve --hostname 127.0.0.1 --port 0` — **HTTP server** transport, not stdout streaming. Ours (L124-143): `opencode --model M --message <prompt>` treated as one-shot streamer — fundamentally wrong transport. Missing: server spawn, URL parsing, `OPENCODE_SERVER_USERNAME`/`OPENCODE_SERVER_PASSWORD` env (L309-310), `build_authenticated_client` (L405), password generation.\n\n**Error mapping (upstream L284-294):** timeout with last 12 lines, premature exit, read-failure. Ours: none.\n\n**Session resume:** upstream passes `resume_session_id` into `RunConfig` (L154, L168). Ours emits nothing.\n\n### 5. gru `src/claude_backend.rs` vs `claude-adapter.ts`\n\n**Spawn (L125-175):** 4 command builders — `build_command` (`--print --verbose --session-id --output-format stream-json --include-partial-messages --dangerously-skip-permissions`), `build_resume_command` (swap `--session-id` → `--resume`), `build_interactive_resume_command` (inherited stdio, no `--print`/`--output-format`), `build_oneshot_command` (`--print --output-format text --max-turns 1 --dangerously-skip-permissions`). Ours: single `buildSpawnArgs`, no interactive/oneshot-text/distinct-resume variants, missing `--verbose` and `--include-partial-messages`.\n\n**parseEvent (L41-117):** MessageStart → Started, ContentBlockStart(ToolUse) buffers, ContentBlockDelta(TextDelta) → TextDelta, ContentBlockDelta(InputJsonDelta) accumulates, ContentBlockStop emits buffered ToolUse via `format_tool_summary`, MessageDelta/MessageStop → MessageComplete, Error, Ping. Ours has none of these stream-json block events — `input_json_delta` accumulation entirely absent.\n\n**Error fallbacks (L223-265):** `format_tool_summary` per-tool fallback strings (`Run: bash command`, `Read: file`, `Tool: {name}`). Ours: no tool-summary formatting.\n\n**Auth:** `GH_HOST` env propagated on all four command variants (L164, tests L188-207). Ours does not propagate `GH_HOST`.\n\n### 6. gru `src/codex_backend.rs` vs `codex-adapter.ts`\n\n**Spawn (L116-162):** `codex exec --json --full-auto [prompt]`; resume `codex exec resume --last --json --full-auto`; oneshot `codex exec --full-auto` (stdin-pipe when prompt==\"-\"). Ours: `codex --quiet --full-auto`; missing `exec` subcommand, `--json`, `resume --last`, stdin-dash convention.\n\n**parseEvent (L189-253):** `thread.started` → Started, `turn.started` → Thinking, `turn.completed` → MessageComplete+usage, `turn.failed` → Error, `item.started`/`item.completed` split by kind, `error`. We match none of these type strings.\n\n**Error fallbacks (L220-250):** `\"Turn failed\"` (L229), `\"Unknown Codex error\"` (L250). Ours: generic passthrough.\n\n**Auth:** `GH_HOST` propagated on all variants (L53-65, L74-79, L107-113). Ours: not propagated.\n\n### 7. gru `src/claude_runner.rs` vs `claude-adapter.ts`\n\nSource-of-truth for claude command shape (L20-44). Both builders pipe stdout, inherit stdin+stderr, set `.env(\"GH_HOST\", ...)`. Confirms gap: our adapter does not inherit stderr, does not set `GH_HOST`, and does not split new-session vs resume to avoid \"session already in use\" errors (L31-32 comment).\n\n### Summary of gaps identified (30)\n\nSee `.a5c/runs/.../state/output.json` for the structured list. Highest-impact clusters: (a) Claude stream-json block-level parsing and flags, (b) Codex `exec --json` subcommand and event vocabulary, (c) Cursor wrong `cliCommand` (`cursor` vs `cursor-agent`) and missing streaming flags, (d) OpenCode wrong transport (HTTP server vs one-shot stdout), (e) absent `GH_HOST`/`CODEX_HOME` env propagation across all OpenAI-adjacent adapters.\n\n## Extended Research: AI Orchestration and Monitoring Ecosystem (2026-04-13)\n\nFollowing the detailed adapter-level analysis above, this section examines the broader ecosystem of AI agent orchestration, monitoring, and harness management platforms to identify architectural patterns and potential feature gaps.\n\n### New projects surveyed\n\n| Category | Project | Lang | Focus | Relevance |\n|---|---|---|---|---|\n| Multi-Agent Orchestration | [LangGraph](https://www.langchain.com/) | Python | Graph-based workflows | Complex orchestration patterns |\n| Multi-Agent Orchestration | [CrewAI](https://crewai.com/) | Python | Role-playing agents | Agent collaboration (44k+ stars) |\n| Multi-Agent Orchestration | [MassGen](https://github.com/massgen/massgen) | Python | Terminal-based scaling | Session memory patterns |\n| Multi-Model Abstraction | [LiteLLM](https://github.com/BerriAI/litellm) | Python | Unified API proxy | 100+ provider abstraction |\n| Multi-Model Abstraction | [Portkey AI](https://portkey.ai/) | - | Enterprise gateway | Advanced observability |\n| Multi-Model Abstraction | [OpenRouter](https://openrouter.ai/) | - | SaaS marketplace | 300+ model coverage |\n| Cost Tracking | [TokenBudget](https://github.com/AIMasterLabs/tokenbudget) | JS | Free cost tracking | Real-time dashboard |\n| Cost Tracking | [Tokscale](https://github.com/junhoyeo/tokscale) | TS | Multi-harness CLI | Similar scope to agent-mux |\n| Cost Tracking | [claude-view](https://recca0120.github.io/en/2026/04/07/claude-view-mission-control/) | JS | Claude dashboard | Real-time monitoring |\n| Session Management | [Pipecat AI](https://github.com/pipecat-ai/pipecat) | Python | Voice/multimodal | Complex state management |\n| Session Management | OpenAI Agents SDK | Python | Session memory | Context preservation |\n| Orchestration | [Haystack](https://haystack.deepset.ai/) | Python | AI pipelines | Production LLM apps |\n| Observability | [Langfuse](https://langfuse.com/) | TS | Framework-agnostic | Span-level tracing |\n| Observability | [Arize Phoenix](https://arize.com/phoenix/) | Python | ML monitoring | Enterprise observability |\n| Observability | Dash0 Agent0 | - | OpenTelemetry-native | AI-powered operations |\n\n### Key architectural patterns observed\n\n#### 1. Abstraction layer positioning\n\n- **API Level** (LiteLLM, OpenRouter, Portkey): Focus on unifying HTTP APIs across providers\n- **Framework Level** (LangChain, CrewAI, Haystack): Provide high-level orchestration abstractions\n- **Harness Level** (agent-mux): **Unique positioning** - abstracts heterogeneous native CLI tools\n\n**Insight**: Agent-mux occupies a distinct layer that others don't address - the gap between raw harnesses and high-level frameworks.\n\n#### 2. Economic models in the ecosystem\n\n- **Pure Open Source**: TokenBudget, LiteLLM, agent-mux\n- **Open Source + Enterprise**: Langfuse, Arize Phoenix  \n- **SaaS with markup**: OpenRouter (5% markup)\n- **Subscription SaaS**: Portkey ($49/month+)\n\n**Insight**: The pure open-source positioning of agent-mux aligns with developer-focused tools rather than enterprise platforms.\n\n#### 3. Observability approaches\n\n- **Real-time dashboards**: claude-view, TokenBudget, Portkey\n- **Span-level tracing**: Langfuse, Arize Phoenix, OpenTelemetry integrations\n- **Cost-first monitoring**: TokenBudget, Tokscale, ai-usage (from original survey)\n\n**Gap identified**: Agent-mux lacks a real-time dashboard component for live monitoring across harnesses.\n\n### Competitive positioning analysis\n\n#### Agent-mux unique strengths confirmed\n\n1. **Multi-harness abstraction at CLI level**: No competitor operates at this specific layer\n2. **Native session file integration**: Reading `~/.claude/projects`, `~/.codex/sessions` etc. directly\n3. **Comprehensive invocation modes**: `local`, `docker`, `ssh`, `k8s` abstraction\n4. **Harness-specific feature preservation**: Hooks, plugins, subagents vs. lowest-common-denominator\n\n#### Emerging opportunity areas\n\n##### 1. Real-time monitoring dashboard\n**Evidence**: claude-view (Claude-specific), Tokscale (multi-harness CLI), TokenBudget (real-time UI)\n**Gap**: No unified dashboard for monitoring runs across all 19+ supported harnesses\n**Recommendation**: Build web-based dashboard similar to claude-view but harness-agnostic\n\n##### 2. Advanced workflow orchestration  \n**Evidence**: LangGraph (44k+ stars) graph-based workflows, CrewAI multi-agent collaboration\n**Gap**: Agent-mux focuses on single-agent runs; limited multi-step orchestration\n**Recommendation**: Consider graph-based workflow support for complex multi-harness processes\n\n##### 3. Enterprise observability features\n**Evidence**: Portkey semantic caching, Arize anomaly detection, Langfuse production monitoring\n**Gap**: Current observability is basic; missing advanced enterprise features\n**Recommendation**: Enhance with semantic caching, guardrails, anomaly detection\n\n##### 4. Cost optimization intelligence\n**Evidence**: All cost-tracking tools show usage but no optimization recommendations\n**Gap**: Industry-wide - everyone tracks costs but none provide optimization suggestions\n**Opportunity**: Agent-mux could be first to provide cost optimization recommendations\n\n##### 5. Agent marketplace integration\n**Evidence**: Existing plugin/skill ecosystems around specific harnesses\n**Gap**: No unified discovery/installation across harnesses\n**Recommendation**: Integrate with existing marketplaces per harness\n\n### Architecture pattern implications\n\n#### Session management patterns\n**From research**: OpenAI Agents SDK automatic context preservation, MassGen memory isolation, Pipecat Flows state management\n**Agent-mux approach**: Native session file reading + resume/fork capabilities  \n**Assessment**: Agent-mux approach is more integrated but could benefit from automatic memory management patterns\n\n#### Event streaming normalization\n**From research**: Most tools either work at API level (losing CLI-specific events) or single-harness (no normalization needed)\n**Agent-mux approach**: Unified `AgentEvent` stream across all adapters\n**Assessment**: This remains a unique and valuable architectural choice\n\n#### Cost attribution granularity  \n**From research**: ai-usage and ccusage parse session JSONL for cache-creation vs cache-read tokens\n**Current gap** (confirmed from original analysis): Agent-mux only emits cost from terminal `result` event\n**Implementation path**: Already identified in original analysis as actionable gap #3\n\n### Recommendations for evolution\n\n#### Immediate (high-impact, moderate effort)\n1. **Real-time dashboard**: Web-based monitoring UI for live runs across all harnesses\n2. **Enhanced cost attribution**: Separate cache-creation vs cache-read tokens (already identified)\n3. **Streaming flags completion**: Claude `--verbose --include-partial-messages stream-json` (already identified)\n\n#### Medium-term (high-value, higher effort)  \n1. **Workflow orchestration**: Graph-based multi-step processes across harnesses\n2. **Advanced observability**: Semantic caching, guardrails, anomaly detection patterns\n3. **Marketplace integration**: Unified discovery/installation across harness ecosystems\n\n#### Long-term (strategic differentiation)\n1. **Cost optimization AI**: First tool to provide intelligent cost optimization recommendations\n2. **Cross-harness session management**: Automatic context preservation across different AI services\n3. **Enterprise security features**: Inspired by emerging agent governance toolkits\n\n### Conclusion\n\nThe extended research confirms that agent-mux occupies a unique and valuable position in the AI tooling ecosystem. While the original adapter-level analysis identified specific implementation gaps, this broader survey reveals strategic opportunities for differentiation and growth.\n\nKey insight: **No competitor operates at the harness abstraction level that agent-mux has carved out**. The closest competitors (Tokscale, claude-view) either focus on monitoring or single-harness scenarios. Agent-mux's multi-harness CLI abstraction with unified event streams remains architecturally unique.\n\nThe opportunity areas identified (real-time monitoring, workflow orchestration, advanced observability) represent natural evolution paths rather than competitive catch-up, positioning agent-mux to lead rather than follow in this space.\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs-agent-mux-archive-comparisons",
      "to": "page:docs-agent-mux-archive-comparisons-16-reference-comparison",
      "kind": "contains_page"
    }
  ]
}