{
  "id": "page:docs-user-guide-features-parallel-execution",
  "_kind": "Page",
  "_file": "wiki/docs/user-guide/features/parallel-execution.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/user-guide/features/parallel-execution.md",
    "sourceKind": "repo-docs",
    "title": "Parallel Execution: Running Tasks Concurrently",
    "displayName": "Parallel Execution: Running Tasks Concurrently",
    "slug": "docs/user-guide/features/parallel-execution",
    "articlePath": "wiki/docs/user-guide/features/parallel-execution.md",
    "article": "\n[Docs](../index.md) › [Features](./index.md) › Parallel Execution\n\n# Parallel Execution: Running Tasks Concurrently\n\n**Version:** 1.1\n**Last Updated:** 2026-01-26\n**Category:** Feature Guide\n\n---\n\n## In Plain English\n\n**Parallel execution means doing multiple things at the same time instead of one after another.**\n\nThink of it like cooking: instead of waiting for the water to boil, THEN chopping vegetables, THEN preheating the oven - you do all three at once. Much faster!\n\n**Example in Babysitter:**\n- Without parallel: Run tests (30s) → Run linter (20s) → Check security (15s) = **65 seconds total**\n- With parallel: Run all three at once = **30 seconds total** (just the slowest one)\n\n**Do you need this as a beginner?** Not really - Babysitter handles this automatically in most pre-built [processes](../reference/glossary.md). This doc is for when you're building custom processes and want to optimize speed.\n\n---\n\n## On this page\n\n- [Overview](#overview)\n- [Use Cases and Scenarios](#use-cases-and-scenarios)\n- [Step-by-Step Instructions](#step-by-step-instructions)\n- [Configuration Options](#configuration-options)\n- [Code Examples and Best Practices](#code-examples-and-best-practices)\n- [Common Pitfalls and Troubleshooting](#common-pitfalls-and-troubleshooting)\n\n---\n\n## Overview\n\nParallel execution enables running multiple independent tasks concurrently within a Babysitter workflow. Instead of executing tasks sequentially, parallel execution dispatches multiple tasks simultaneously and waits for all to complete. This significantly reduces total execution time for workflows with independent operations.\n\n### Why Use Parallel Execution\n\n- **Faster Workflows**: Execute independent tasks simultaneously instead of sequentially\n- **Resource Efficiency**: Maximize utilization of available compute resources\n- **Optimized Quality Checks**: Run lint, test, coverage, and security checks in parallel\n- **Batch Processing**: Process multiple items concurrently with `parallel.map`\n- **Maintained Determinism**: Results remain deterministic despite parallel execution\n\n---\n\n## Use Cases and Scenarios\n\n### Scenario 1: Parallel Quality Checks\n\nRun multiple quality checks simultaneously after implementation.\n\n```javascript\nexport async function process(inputs, ctx) {\n  const impl = await ctx.task(implementTask, { feature: inputs.feature });\n\n  // Run all quality checks in parallel\n  const [coverageResult, lintResult, typeCheckResult, securityResult] = await ctx.parallel.all([\n    () => ctx.task(coverageTask, { testFiles: impl.testFiles }),\n    () => ctx.task(lintTask, { files: impl.filesModified }),\n    () => ctx.task(typeCheckTask, { files: impl.filesModified }),\n    () => ctx.task(securityTask, { files: impl.filesModified })\n  ]);\n\n  return {\n    coverage: coverageResult,\n    lint: lintResult,\n    typeCheck: typeCheckResult,\n    security: securityResult\n  };\n}\n```\n\n**Time savings:**\n- Sequential: ~20 seconds (5s + 5s + 5s + 5s)\n- Parallel: ~5 seconds (all run simultaneously)\n\n### Scenario 2: Batch File Processing\n\nProcess multiple files concurrently using `parallel.map`.\n\n```javascript\nexport async function process(inputs, ctx) {\n  const files = inputs.files;  // ['file1.ts', 'file2.ts', 'file3.ts', ...]\n\n  // Process all files in parallel\n  const results = await ctx.parallel.map(files, file =>\n    ctx.task(processFileTask, { file })\n  );\n\n  return { processed: results.length, results };\n}\n```\n\n### Scenario 3: Multi-Service Deployment\n\nDeploy to multiple services concurrently.\n\n```javascript\nexport async function process(inputs, ctx) {\n  const services = ['api', 'web', 'worker'];\n\n  // Deploy all services in parallel\n  const deployResults = await ctx.parallel.all([\n    () => ctx.task(deployTask, { service: 'api', env: inputs.env }),\n    () => ctx.task(deployTask, { service: 'web', env: inputs.env }),\n    () => ctx.task(deployTask, { service: 'worker', env: inputs.env })\n  ]);\n\n  const success = deployResults.every(r => r.success);\n  return { success, deployments: deployResults };\n}\n```\n\n### Scenario 4: Mixed Sequential and Parallel\n\nCombine sequential dependencies with parallel execution.\n\n```javascript\nexport async function process(inputs, ctx) {\n  // Sequential: Build must complete first\n  const buildResult = await ctx.task(buildTask, { target: 'app' });\n\n  // Parallel: These can run concurrently after build\n  const [testResult, e2eResult, docResult] = await ctx.parallel.all([\n    () => ctx.task(unitTestTask, { files: buildResult.files }),\n    () => ctx.task(e2eTestTask, { url: buildResult.previewUrl }),\n    () => ctx.task(generateDocsTask, { files: buildResult.files })\n  ]);\n\n  // Sequential: Deployment depends on tests passing\n  if (testResult.success && e2eResult.success) {\n    const deployResult = await ctx.task(deployTask, { build: buildResult });\n    return { deployed: true, deployResult };\n  }\n\n  return { deployed: false, reason: 'Tests failed' };\n}\n```\n\n---\n\n## Step-by-Step Instructions\n\n### Step 1: Identify Independent Tasks\n\nAnalyze your workflow to identify tasks that:\n- Do not depend on each other's results\n- Can execute in any order\n- Access different resources or have read-only access\n\n**Independent (can parallelize):**\n- Running lint and tests on the same files\n- Processing multiple independent files\n- Checking multiple quality metrics\n\n**Dependent (must be sequential):**\n- Building before testing\n- Creating a database before seeding it\n- Writing a file before reading it\n\n### Step 2: Use ctx.parallel.all\n\nReplace sequential calls with `ctx.parallel.all` for independent tasks.\n\n**Before (sequential):**\n```javascript\nconst lintResult = await ctx.task(lintTask, { files });\nconst testResult = await ctx.task(testTask, { files });\nconst coverageResult = await ctx.task(coverageTask, { files });\n// Total time: sum of all task durations\n```\n\n**After (parallel):**\n```javascript\nconst [lintResult, testResult, coverageResult] = await ctx.parallel.all([\n  () => ctx.task(lintTask, { files }),\n  () => ctx.task(testTask, { files }),\n  () => ctx.task(coverageTask, { files })\n]);\n// Total time: duration of longest task\n```\n\n### Step 3: Use ctx.parallel.map for Collections\n\nProcess arrays of items concurrently.\n\n```javascript\nconst files = ['a.ts', 'b.ts', 'c.ts', 'd.ts'];\n\n// Process all files in parallel\nconst results = await ctx.parallel.map(files, file =>\n  ctx.task(processFileTask, { file })\n);\n```\n\n### Step 4: Handle Results\n\nParallel execution returns results in the same order as input thunks.\n\n```javascript\nconst [first, second, third] = await ctx.parallel.all([\n  () => ctx.task(taskA, {}),  // first = result of taskA\n  () => ctx.task(taskB, {}),  // second = result of taskB\n  () => ctx.task(taskC, {})   // third = result of taskC\n]);\n```\n\n---\n\n## Configuration Options\n\n### ctx.parallel.all\n\nExecute multiple thunks in parallel and wait for all to complete.\n\n```javascript\nctx.parallel.all<T>(thunks: Array<() => T | Promise<T>>): Promise<T[]>\n```\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `thunks` | `Array<() => T>` | Array of functions that return tasks |\n\n**Returns:** `Promise<T[]>` - Array of results in the same order as input\n\n### ctx.parallel.map\n\nMap items to tasks and execute in parallel.\n\n```javascript\nctx.parallel.map<TItem, TOut>(\n  items: TItem[],\n  fn: (item: TItem) => TOut | Promise<TOut>\n): Promise<TOut[]>\n```\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `items` | `TItem[]` | Array of items to process |\n| `fn` | `(item: TItem) => TOut` | Function mapping item to task |\n\n**Returns:** `Promise<TOut[]>` - Array of results in the same order as input\n\n---\n\n## Code Examples and Best Practices\n\n### Example 1: Parallel Quality Checks in TDD Workflow\n\n```javascript\nexport async function process(inputs, ctx) {\n  const { feature, targetQuality = 85, maxIterations = 5 } = inputs;\n\n  let iteration = 0;\n  let quality = 0;\n\n  while (iteration < maxIterations && quality < targetQuality) {\n    iteration++;\n\n    // Sequential: Tests must run first\n    const tests = await ctx.task(writeTestsTask, { feature, iteration });\n    const impl = await ctx.task(implementTask, { tests, iteration });\n    const testResults = await ctx.task(runTestsTask, { testFiles: tests.testFiles });\n\n    // Parallel: Quality checks are independent\n    const [coverage, lint, typeCheck, security] = await ctx.parallel.all([\n      () => ctx.task(coverageCheckTask, { testFiles: tests.testFiles }),\n      () => ctx.task(lintCheckTask, { files: impl.filesModified }),\n      () => ctx.task(typeCheckTask, { files: impl.filesModified }),\n      () => ctx.task(securityCheckTask, { files: impl.filesModified })\n    ]);\n\n    // Sequential: Scoring depends on all quality checks\n    const score = await ctx.task(agentScoringTask, {\n      tests,\n      testResults,\n      implementation: impl,\n      qualityChecks: { coverage, lint, typeCheck, security }\n    });\n\n    quality = score.overallScore;\n  }\n\n  return { quality, iterations: iteration };\n}\n```\n\n### Example 2: Parallel File Processing with Error Handling\n\n```javascript\nexport async function process(inputs, ctx) {\n  const files = inputs.files;\n\n  // Process files in parallel\n  const results = await ctx.parallel.map(files, async file => {\n    try {\n      const result = await ctx.task(processFileTask, { file });\n      return { file, success: true, result };\n    } catch (error) {\n      // Individual failures don't stop other tasks\n      return { file, success: false, error: error.message };\n    }\n  });\n\n  const successful = results.filter(r => r.success);\n  const failed = results.filter(r => !r.success);\n\n  ctx.log(`Processed ${successful.length}/${files.length} files`);\n\n  return { successful, failed };\n}\n```\n\n### Example 3: Chunked Parallel Processing\n\nProcess large batches in chunks to limit concurrency.\n\n```javascript\nexport async function process(inputs, ctx) {\n  const items = inputs.items;\n  const chunkSize = 10;  // Process 10 items at a time\n\n  const results = [];\n\n  for (let i = 0; i < items.length; i += chunkSize) {\n    const chunk = items.slice(i, i + chunkSize);\n\n    // Process chunk in parallel\n    const chunkResults = await ctx.parallel.map(chunk, item =>\n      ctx.task(processItemTask, { item })\n    );\n\n    results.push(...chunkResults);\n    ctx.log(`Processed ${Math.min(i + chunkSize, items.length)}/${items.length} items`);\n  }\n\n  return { total: results.length, results };\n}\n```\n\n### Example 4: Parallel Final Verification\n\n```javascript\nexport async function process(inputs, ctx) {\n  // ... implementation phases ...\n\n  // Final parallel verification checks\n  const [finalTestResult, finalCoverageResult, integrationTestResult] = await ctx.parallel.all([\n    () => ctx.task(runTestsTask, { testFiles: iterationResults[iteration - 1].tests.testFiles }),\n    () => ctx.task(coverageCheckTask, { testFiles: iterationResults[iteration - 1].tests.testFiles }),\n    () => ctx.task(integrationTestTask, { feature })\n  ]);\n\n  // Agent-based final review using all parallel results\n  const finalReview = await ctx.task(agentFinalReviewTask, {\n    feature,\n    finalTests: finalTestResult,\n    finalCoverage: finalCoverageResult,\n    integrationTests: integrationTestResult\n  });\n\n  return { finalReview };\n}\n```\n\n### Best Practices\n\n1. **Only Parallelize Independent Tasks**: Ensure tasks do not have data dependencies\n2. **Use Thunks (Arrow Functions)**: Wrap task calls in `() =>` to defer execution\n3. **Order Results Match Input Order**: Results array preserves the order of input thunks\n4. **Handle Individual Failures**: Consider try-catch within parallel.map for graceful degradation\n5. **Limit Concurrency for Large Batches**: Process in chunks to avoid overwhelming resources\n6. **Log Progress**: For long-running parallel operations, log progress periodically\n7. **Combine with Sequential When Needed**: Use parallel for independent work, sequential for dependencies\n\n---\n\n## Common Pitfalls and Troubleshooting\n\n### Pitfall 1: Forgetting Thunk Wrappers\n\n**Symptom:** Tasks execute immediately instead of in parallel.\n\n**Wrong:**\n```javascript\n// Tasks execute immediately, not in parallel!\nconst results = await ctx.parallel.all([\n  ctx.task(taskA, {}),  // Executes immediately\n  ctx.task(taskB, {}),  // Executes immediately\n  ctx.task(taskC, {})   // Executes immediately\n]);\n```\n\n**Correct:**\n```javascript\n// Tasks are deferred until parallel.all processes them\nconst results = await ctx.parallel.all([\n  () => ctx.task(taskA, {}),  // Wrapped in thunk\n  () => ctx.task(taskB, {}),  // Wrapped in thunk\n  () => ctx.task(taskC, {})   // Wrapped in thunk\n]);\n```\n\n### Pitfall 2: Parallelizing Dependent Tasks\n\n**Symptom:** Race conditions or undefined results.\n\n**Wrong:**\n```javascript\n// taskB depends on taskA's result!\nconst [a, b] = await ctx.parallel.all([\n  () => ctx.task(taskA, {}),\n  () => ctx.task(taskB, { input: a })  // 'a' is undefined here!\n]);\n```\n\n**Correct:**\n```javascript\n// Sequential for dependent tasks\nconst a = await ctx.task(taskA, {});\nconst b = await ctx.task(taskB, { input: a });\n```\n\n### Pitfall 3: One Failure Stops All\n\n**Symptom:** If one parallel task fails, all results are lost.\n\n**Default behavior:**\n```javascript\n// If any task throws, the entire parallel.all fails\nconst results = await ctx.parallel.all([\n  () => ctx.task(taskA, {}),\n  () => ctx.task(taskB, {}),  // If this fails...\n  () => ctx.task(taskC, {})   // ... we lose taskA and taskC results too\n]);\n```\n\n**Solution - Handle errors individually:**\n```javascript\nconst results = await ctx.parallel.map(['A', 'B', 'C'], async id => {\n  try {\n    return { id, success: true, result: await ctx.task(getTask(id), {}) };\n  } catch (error) {\n    return { id, success: false, error: error.message };\n  }\n});\n```\n\n### Pitfall 4: Result Order Confusion\n\n**Symptom:** Results don't match expected tasks.\n\n**Remember:** Results are always in the same order as input thunks.\n\n```javascript\nconst [first, second, third] = await ctx.parallel.all([\n  () => ctx.task(slowTask, {}),   // Takes 10s, but 'first' = its result\n  () => ctx.task(fastTask, {}),   // Takes 1s, but 'second' = its result\n  () => ctx.task(mediumTask, {})  // Takes 5s, but 'third' = its result\n]);\n\n// Order is preserved regardless of completion time\n// first = slowTask result\n// second = fastTask result\n// third = mediumTask result\n```\n\n### Pitfall 5: Too Many Parallel Tasks\n\n**Symptom:** Resource exhaustion, timeouts, or throttling.\n\n**Cause:** Starting hundreds of tasks simultaneously.\n\n**Solution:** Chunk large batches:\n\n```javascript\nconst items = Array.from({ length: 1000 }, (_, i) => i);\nconst chunkSize = 50;\n\nfor (let i = 0; i < items.length; i += chunkSize) {\n  const chunk = items.slice(i, i + chunkSize);\n  await ctx.parallel.map(chunk, item => ctx.task(processTask, { item }));\n}\n```\n\n---\n\n## Related Documentation\n\n- [Process Definitions](./process-definitions.md) - Learn how to structure workflows\n- [Quality Convergence](./quality-convergence.md) - Parallel checks in TDD loops\n- [Journal System](./journal-system.md) - How parallel tasks are recorded\n\n---\n\n## Summary\n\nParallel execution dramatically reduces workflow duration by running independent tasks concurrently. Use `ctx.parallel.all` for fixed sets of tasks and `ctx.parallel.map` for processing collections. Remember to wrap task calls in thunks, only parallelize independent operations, and handle errors gracefully. For large batches, process in chunks to manage resource utilization.\n\n---\n\n## Next steps\n\n- **Next:** [Process Definitions](./process-definitions.md)\n- **Related:** [Quality Convergence](./quality-convergence.md), [Best Practices](./best-practices.md)\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs-user-guide-features",
      "to": "page:docs-user-guide-features-parallel-execution",
      "kind": "contains_page"
    }
  ]
}