From acba1a2da0f32056b5eead232541410f91bba27b Mon Sep 17 00:00:00 2001 From: beo3000 Date: Sat, 21 Mar 2026 10:28:52 +0100 Subject: [PATCH] planning --- .../plans/2026-03-21-batch-orchestrator.md | 452 ++++++++++++++++++ .../2026-03-21-batch-orchestrator-design.md | 309 ++++++++++++ .../2026-03-21-batch-orchestrator-future.md | 39 ++ 3 files changed, 800 insertions(+) create mode 100644 docs/superpowers/plans/2026-03-21-batch-orchestrator.md create mode 100644 docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md create mode 100644 docs/superpowers/specs/2026-03-21-batch-orchestrator-future.md diff --git a/docs/superpowers/plans/2026-03-21-batch-orchestrator.md b/docs/superpowers/plans/2026-03-21-batch-orchestrator.md new file mode 100644 index 0000000..436b1cf --- /dev/null +++ b/docs/superpowers/plans/2026-03-21-batch-orchestrator.md @@ -0,0 +1,452 @@ +# Batch Orchestrator Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Implement the batch orchestrator for re:struct — a `/project:run-batch` command that semi-autonomously processes planned tasks from task.md using subagents. + +**Architecture:** A command file (`.claude/commands/run-batch.md`) encodes the orchestrator flow as a Claude prompt instruction. A rules file (`rules/batch-orchestration.md`) codifies the constraints. Both are markdown — no application code. The orchestrator leverages Claude Code's native Agent tool for subagent dispatch. + +**Tech Stack:** Claude Code commands (markdown prompts), Claude Code Agent tool, project rules system + +**Spec:** `docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md` + +--- + +## File Structure + +| File | Action | Responsibility | +|------|--------|----------------| +| `rules/batch-orchestration.md` | Create | Orchestrator/subagent contract rules, loaded via CLAUDE.md | +| `.claude/commands/run-batch.md` | Create | The orchestrator command — full flow from parsing to consolidation | +| `.claude/settings.json` | Create | Project-level permissions for batch execution | +| `CLAUDE.md` | Modify | Reference new rule, add run-batch to Workflow-Commands | + +--- + +### Task 1: Create `rules/batch-orchestration.md` + +**Files:** +- Create: `rules/batch-orchestration.md` + +This file defines the hard constraints that apply during any batch orchestration run. +It is referenced from CLAUDE.md and automatically loaded into context. + +- [ ] **Step 1: Write the rules file** + +```markdown +# Batch Orchestration Rules + +## Orchestrator Constraints +- The orchestrator only reads task.md and subagent summaries — never full output files +- The orchestrator only processes tasks from the active phase (defined in CLAUDE.md "Aktiver Scope") +- Shared state files (status.md, decisions-log.md, task.md) are only written by the orchestrator, never by subagents +- Phase-specific shared artifacts (e.g., capabilities-inventory.md) are updated only by the orchestrator during consolidation +- The orchestrator never triggers /project:phase-transition or starts tasks from the next phase +- When all DoD items of the active phase are complete: hard stop, inform user +- Missing batch parameters (items, checkpoint mode) are always queried interactively before starting + +## Subagent Constraints +- Each subagent writes only its own assigned output file +- Subagents never write to task.md, status.md, decisions-log.md, or CLAUDE.md +- Each subagent returns a 2-3 sentence summary as its return value + +## Task Parsing +- The orchestrator parses the first section in task.md that contains ### sub-headings with - [ ] checkboxes (the work section) +- Flat checkbox lists without sub-headings are treated as Definition of Done and ignored for dispatching +- Tasks under the same ### heading form a parallelization group (can run concurrently) +- Tasks under different ### headings are sequential (Group N+1 starts after Group N completes and checkpoint passes) + +## Context Loading +- Base context (always): project rules of active phase + status.md +- Phase-specific context: follow phase rules if they define context requirements (e.g., discovery-waves.md) +- Task-specific context: if task declares (context: file1, file2), load those; otherwise load output files from prior groups +- Prefer condensed documents over raw sources + +## Checkpoint Modes +- after-each (default): pause after each group, show summaries, wait for user input +- after-batch:N: run N groups, then pause +- at-end: run all groups, consolidated review at end +- Phase boundary: always a hard stop regardless of selected mode + +## Error Handling +- On subagent failure: mark task as failed in checkpoint summary, do not block other parallel tasks in the same group +- Failed tasks stay - [ ] in task.md (not marked as completed) +- Failures are reported at the next checkpoint with the error details +``` + +- [ ] **Step 2: Verify the rules file covers all spec constraints** + +Cross-check against spec sections: +- Orchestrator Flow (steps 1-5) ✓ +- Subagent Contract (boundaries) ✓ +- Task Parsing (conventions) ✓ +- Context Loading Strategy ✓ +- Checkpoint Modes ✓ +- Phase Protection ✓ +- Error Handling ✓ + +- [ ] **Step 3: Commit** + +```bash +git add rules/batch-orchestration.md +git commit -m "feat: add batch orchestration rules" +``` + +--- + +### Task 2: Create `.claude/commands/run-batch.md` + +**Files:** +- Create: `.claude/commands/run-batch.md` + +This is the main orchestrator command. It's a Claude prompt instruction that encodes +the complete flow. When invoked, Claude follows these instructions step by step. + +**Important:** This file must be self-contained enough that Claude can follow it without +having seen the spec. The rules file provides the constraints; this file provides the procedure. + +- [ ] **Step 1: Write the command file — Preamble and Phase Detection** + +The command file starts with reading CLAUDE.md and determining the active phase: + +```markdown +Du bist der Batch-Orchestrator. Führe die folgenden Schritte der Reihe nach aus. + +Beachte dabei strikt die Regeln aus @rules/batch-orchestration.md + +## Schritt 1 — Phase und Tasks erkennen + +1. Lies CLAUDE.md und bestimme die aktive Phase und das Arbeitsverzeichnis +2. Lies die task.md der aktiven Phase +3. Finde den Arbeitsabschnitt: den ersten Abschnitt mit ### Unterüberschriften und - [ ] Checkboxen +4. Ignoriere flache Checkbox-Listen ohne Unterüberschriften (das ist die Definition of Done) +5. Erstelle eine nummerierte Liste aller offenen Tasks (- [ ]) mit: + - Nummer + - Task-Beschreibung + - Gruppenname (### Überschrift) + - Output-Pfad (falls im Text erkennbar) +``` + +- [ ] **Step 2: Write the command file — Permission Check** + +```markdown +## Schritt 1.5 — Berechtigungen prüfen + +Prüfe ob die folgenden Tools ohne Nachfrage verfügbar sind: Read, Write, Edit, Glob, Grep, Agent. + +Falls Berechtigungen fehlen, zeige dem User: +> ⚠ Für den Batch-Modus werden folgende Berechtigungen benötigt: [Liste] +> Bitte konfiguriere sie in `.claude/settings.json` oder `.claude/settings.local.json`: +> ```json +> { "permissions": { "allow": ["Read", "Write", "Edit", "Glob", "Grep", "Agent"] } } +> ``` + +Brich dann ab. Starte den Batch NICHT ohne bestätigte Berechtigungen. +``` + +- [ ] **Step 3: Write the command file — Parameter Query** + +```markdown +## Schritt 2 — Parameter abfragen + +Falls der User beim Aufruf keine Parameter angegeben hat, oder Parameter fehlen, frage interaktiv: + +Zeige die Task-Liste aus Schritt 1 und frage: + +> Ich sehe N offene Tasks in Phase X: +> [nummerierte Liste mit Gruppenname] +> +> **Welche Tasks?** (alle / Nummern z.B. 1-3 / Gruppenname z.B. wave-3) +> **Checkpoint-Modus?** (after-each / after-batch:N / at-end) + +Warte auf die Antworten des Users. + +Zeige dann eine Zusammenfassung des geplanten Batch-Laufs: +> **Batch-Plan:** +> - Tasks: [ausgewählte Tasks] +> - Gruppen: [Gruppenreihenfolge, parallele Tasks markiert] +> - Checkpoint: [gewählter Modus] +> - Phase: [aktive Phase] +> +> Soll ich starten? + +Warte auf Bestätigung. +``` + +- [ ] **Step 4: Write the command file — Group Processing Loop** + +```markdown +## Schritt 3 — Gruppen abarbeiten + +Arbeite die Gruppen in der Reihenfolge ihrer ### Überschriften ab. + +### Für jede Gruppe: + +**Context bestimmen:** +- Basis: rules der aktiven Phase + status.md +- Phasen-spezifisch: falls Phase-Rules Context definieren (z.B. discovery-waves.md), folge diesen +- Task-spezifisch: falls der Task (context: datei1, datei2) deklariert, lade diese +- Fallback: alle existierenden Output-Dateien vorheriger Gruppen im aktuellen Batch + +**Tasks dispatchen:** +- Falls die Gruppe nur einen Task hat: dispatche einen Subagent +- Falls die Gruppe mehrere Tasks hat: dispatche alle als parallele Subagents + +Für jeden Subagent verwende das Agent-Tool mit folgendem Prompt-Muster: + +> Du bist ein Subagent im re:struct Framework. +> +> **Dein Auftrag:** [Task-Beschreibung aus task.md] +> **Output-Datei:** [Pfad] +> **Context-Dateien zum Lesen:** [Liste der Context-Dateien] +> **Regeln:** Beachte @rules/framework-constraints.md und @rules/language-conventions.md +> +> Schreibe dein Ergebnis in die Output-Datei. +> Gib am Ende eine Zusammenfassung in 2-3 Sätzen zurück (NUR die Zusammenfassung als Return-Value, nicht in die Datei). + +**Auf Abschluss warten:** +- Warte bis alle Subagents der Gruppe fertig sind +- Sammle die Summaries und eventuelle Fehler + +**Fehlerbehandlung:** +- Falls ein Subagent fehlschlägt: notiere den Fehler, blockiere aber nicht andere parallele Tasks +- Fehlgeschlagene Tasks werden beim Checkpoint gemeldet +``` + +- [ ] **Step 5: Write the command file — Checkpoint Handling** + +```markdown +### Checkpoint-Logik + +Nach jeder abgeschlossenen Gruppe: prüfe ob ein Checkpoint fällig ist. + +**after-each:** Immer Checkpoint nach jeder Gruppe. +**after-batch:N:** Checkpoint nach jeder N-ten Gruppe. Zähler beginnt bei 1. +**at-end:** Kein Checkpoint bis alle Gruppen fertig sind. + +**Bei einem Checkpoint:** + +Zeige dem User: +> **Checkpoint — Gruppe "[Gruppenname]" abgeschlossen** +> +> | Task | Output | Status | Summary | +> |------|--------|--------|---------| +> | [Name] | [Dateipfad] | ✅ / ❌ | [2-3 Sätze] | +> +> **Befehle:** `weiter` | `stopp` | `zeig mir [datei]` + +- Bei `weiter`: nächste Gruppe starten +- Bei `stopp`: gehe zu Schritt 4 (Konsolidierung) +- Bei `zeig mir [datei]`: Datei lesen und anzeigen, dann erneut fragen + +**Phasen-Checkpoint (immer aktiv):** +Prüfe nach jeder Gruppe ob alle offenen Tasks der Phase erledigt sind (prüfe die DoD-Sektion). +Falls ja: STOPP, unabhängig vom Checkpoint-Modus: +> **Alle Tasks dieser Phase sind abgeschlossen.** +> Ein Phasenwechsel ist möglich. Nutze `/project:phase-transition` wenn du bereit bist. + +Gehe dann zu Schritt 4. +``` + +- [ ] **Step 6: Write the command file — Consolidation** + +```markdown +## Schritt 4 — Konsolidierung + +Führe die folgenden Updates durch (nur der Orchestrator, nie Subagents): + +1. **task.md aktualisieren:** + - Markiere erfolgreich abgeschlossene Tasks mit [x] + - Fehlgeschlagene Tasks bleiben [ ] + +2. **status.md aktualisieren:** + - Aktuellen Stand in 2-3 Sätzen + - Liste der abgeschlossenen Tasks + - Falls Fehler auftraten: dokumentiere sie + - Nächste Schritte (verbleibende offene Tasks) + +3. **decisions-log.md:** + - Falls während des Batch Entscheidungen relevant wurden: Einträge erstellen + - Format: siehe docs/templates/decisions-log-entry.md + +4. **Phasen-spezifische Shared Artifacts:** + - Falls die Phase geteilte Dokumente hat (z.B. capabilities-inventory.md in Discovery): + aktualisiere sie mit den konsolidierten Ergebnissen aus den Subagent-Summaries + +## Schritt 5 — Commit-Vorschlag + +Zeige dem User: +> **Batch abgeschlossen.** Zusammenfassung: +> - [N] Tasks erfolgreich, [M] fehlgeschlagen +> - Aktualisiert: task.md, status.md [, decisions-log.md] [, weitere] +> - Output-Dateien: [Liste der neuen Dateien] +> +> Soll ich einen Commit erstellen? + +Falls ja: erstelle einen Commit mit einer Nachricht die den Batch beschreibt, +z.B. `feat: complete Wave 2 entry-points analysis` +``` + +- [ ] **Step 7: Assemble the complete command file** + +Combine Steps 1-6 into the final `.claude/commands/run-batch.md` file. +Ensure the flow is continuous and there are no gaps between sections. + +- [ ] **Step 8: Verify the command file against the spec** + +Cross-check every section of the spec against the command file: +- [ ] Task parsing (work section vs DoD) — Schritt 1 +- [ ] Permission check — Schritt 1.5 +- [ ] Interactive parameter query — Schritt 2 +- [ ] Group processing with parallelization — Schritt 3 +- [ ] Subagent contract (input/output/boundaries) — Subagent prompt template +- [ ] Context loading strategy (base/phase/task) — Context bestimmen +- [ ] Checkpoint modes (after-each, after-batch:N, at-end) — Checkpoint-Logik +- [ ] Phase protection (hard stop) — Phasen-Checkpoint +- [ ] Error handling (failed tasks) — Fehlerbehandlung +- [ ] Consolidation (task.md, status.md, decisions-log, shared artifacts) — Schritt 4 +- [ ] Commit proposal — Schritt 5 +- [ ] session-start/end replacement — implicitly handled (no reference to session-start needed) + +- [ ] **Step 9: Commit** + +```bash +git add .claude/commands/run-batch.md +git commit -m "feat: add run-batch orchestrator command" +``` + +--- + +### Task 3: Create `.claude/settings.json` + +**Files:** +- Create: `.claude/settings.json` + +Project-level permissions that ensure the batch orchestrator and its subagents +can operate without permission prompts. This file is separate from the existing +`.claude/settings.local.json` (which contains user-specific permissions). + +- [ ] **Step 1: Write settings.json** + +```json +{ + "permissions": { + "allow": [ + "Read", + "Glob", + "Grep", + "Edit", + "Write", + "Agent" + ] + } +} +``` + +- [ ] **Step 2: Verify no conflict with settings.local.json** + +Read `.claude/settings.local.json` and confirm the two files don't conflict. +`settings.local.json` has specific Bash and Skill permissions; `settings.json` +adds the core tool permissions. These are additive, no conflict expected. + +- [ ] **Step 3: Commit** + +```bash +git add .claude/settings.json +git commit -m "feat: add project-level permissions for batch orchestration" +``` + +--- + +### Task 4: Update CLAUDE.md + +**Files:** +- Modify: `CLAUDE.md` (two sections) + +- [ ] **Step 1: Add batch-orchestration rule reference** + +In the `## Regeln` section, add: +``` +@rules/batch-orchestration.md +``` + +- [ ] **Step 2: Add run-batch to Workflow-Commands** + +In the `## Workflow-Commands` section, add: +``` +- `/project:run-batch` — Batch von geplanten Tasks semi-autonom abarbeiten (Checkpoints, Parallelisierung) +``` + +- [ ] **Step 3: Verify CLAUDE.md is consistent** + +Read the updated CLAUDE.md and verify: +- The rule reference is in the Regeln section alongside existing rules +- The command is listed alongside existing commands +- No other sections need updating + +- [ ] **Step 4: Commit** + +```bash +git add CLAUDE.md +git commit -m "docs: reference batch orchestration rule and command in CLAUDE.md" +``` + +--- + +### Task 5: Verification Dry-Run + +**Files:** +- Read-only: all created/modified files + +- [ ] **Step 1: Verify file structure** + +Confirm all expected files exist: +``` +rules/batch-orchestration.md ✓ +.claude/commands/run-batch.md ✓ +.claude/settings.json ✓ +CLAUDE.md (updated) ✓ +``` + +- [ ] **Step 2: Verify rules are loadable** + +Read CLAUDE.md and confirm `@rules/batch-orchestration.md` resolves correctly +(the file exists at the referenced path). + +- [ ] **Step 3: Verify command is invocable** + +Confirm `.claude/commands/run-batch.md` exists in the commands directory +alongside the existing commands (session-start, session-end, decision, etc.). + +- [ ] **Step 4: Read through the complete command file end-to-end** + +Read `run-batch.md` as if you were Claude encountering it for the first time. +Check for: +- Is the flow unambiguous? Can every step be followed without guessing? +- Are there references to files or concepts that aren't explained? +- Is the subagent prompt template complete? +- Are all checkpoint modes handled? + +- [ ] **Step 5: Trace through an example scenario** + +Mentally trace through this scenario: +- Phase 1 Discovery is active +- task.md has Wave 1 (1 task), Wave 2 (1 task), Wave 3 (3 parallel tasks) +- User says: `/project:run-batch items:all checkpoint:after-each` + +Verify the command file produces this behavior: +1. Reads task.md, finds 5 tasks in 3 groups +2. Checks permissions +3. Shows task list, asks for confirmation +4. Dispatches Wave 1 subagent → checkpoint → user says "weiter" +5. Dispatches Wave 2 subagent → checkpoint → user says "weiter" +6. Dispatches 3 Wave 3 subagents in parallel → checkpoint +7. All tasks done → phase checkpoint → hard stop +8. Consolidation → commit proposal + +- [ ] **Step 6: Final commit (if any fixes were needed)** + +```bash +git add -A +git commit -m "fix: adjustments from dry-run verification" +``` diff --git a/docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md b/docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md new file mode 100644 index 0000000..4782605 --- /dev/null +++ b/docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md @@ -0,0 +1,309 @@ +# Batch Orchestrator for re:struct + +## Problem Statement + +When tasks are pre-planned (e.g., discovery waves, user stories, architecture decisions), +the human operator is reduced to a manual scheduler: + +``` +session-start → "do task X" → session-end → clear context → commit → repeat +``` + +This is repetitive and adds no value when the next steps are already defined. + +## Solution Overview + +A lightweight in-session orchestrator that reads planned tasks from `task.md`, +dispatches subagents for execution (fresh context per task), and pauses at +configurable checkpoints for human review. + +**Approach:** Slim orchestrator + subagent isolation (Approach 3 from evaluation). + +- Subagents do the heavy work with their own context +- Orchestrator reads only task.md + subagent summaries (never full output files) +- Parallelization within groups, sequential between groups +- Phase boundary is a hard stop — never crossed automatically + +## Task Parsing from task.md + +### Which section does the orchestrator parse? + +A `task.md` may contain multiple sections with checkboxes — e.g., a "Waves" section with +sub-headings AND a flat "Definition of Done" section. These serve different purposes: + +- **Work section** (with `###` sub-headings): actionable tasks with grouping — this is what + the orchestrator parses. Identified by having sub-headings (`###`) under it. +- **Definition of Done section** (flat list): acceptance criteria for phase completion — + used only for phase-boundary checks, not for dispatching work. + +**Convention:** The orchestrator parses the **first section in task.md that contains +`###` sub-headings with `- [ ]` checkboxes**. This is the work plan. Flat checkbox lists +without sub-headings are treated as DoD and ignored for dispatching. + +Example `task.md` with both sections: + +```markdown +## Analyse-Wellen ← WORK SECTION (orchestrator parses this) +### Wave 1 — Structure +- [ ] structure-map.md + +### Wave 2 — Entry Points +- [ ] entry-points.md + +### Wave 3 — Module Deep Dives +- [ ] Module: auth +- [ ] Module: invoicing +- [ ] Module: reporting + +## Ergebnisse (Definition of Done) ← DoD SECTION (phase check only) +- [ ] structure-map.md +- [ ] domain-model.md +- [ ] capabilities-inventory.md +``` + +### Conventions + +- `- [ ]` = open task, eligible for orchestrator +- `- [x]` = completed, skipped +- Tasks under the same `###` heading = **parallelization group** (can run concurrently) +- Tasks under different `###` headings = **sequential** (Group 2 starts after Group 1) +- Each task requires an **output path** — either explicit in the text or derivable from task name +- Each task must contain enough context for the subagent: what to do, which sources to read + +### What the orchestrator does NOT read from task.md + +- Checkpoint mode and batch size — provided by the user at invocation time +- The Definition of Done section — only relevant for phase-boundary checks + +## Checkpoint Modes + +Three modes, selected by the user at batch start: + +| Mode | Behavior | Typical Use | +|------|----------|-------------| +| `after-each` (default) | Pause after each group, show summaries, wait for "continue" | New/uncertain tasks, want to review every output | +| `after-batch:N` | Run N groups, then pause | "Do the next 3, then I'll check" | +| `at-end` | Run all groups, consolidated review at the end | Well-understood tasks, high confidence | + +### Checkpoint Behavior + +At each checkpoint the orchestrator: + +1. Shows per completed task: task name, output file path, 2-3 sentence summary (provided by subagent) +2. User can respond: + - `weiter` (continue) — next task / next batch + - `stopp` (stop) — orchestrator consolidates current results and ends + - `zeig mir [file]` (show me) — review an output file in detail before deciding +3. After final checkpoint: consolidation (status.md, decisions-log, task.md checkboxes, commit proposal) + +### Checkpoint with Parallel Tasks + +| Mode | Behavior with group of 3 parallel tasks | +|------|----------------------------------------| +| `after-each` | All 3 run in parallel, checkpoint when **all 3 complete** — show all summaries | +| `after-batch:N` | Batch counter counts groups, not individual tasks | +| `at-end` | Run through, checkpoint only after all groups | + +Parallelization does not change checkpoint rhythm — checkpoints refer to groups, not individual subagents. + +## Phase Protection + +- The orchestrator only processes tasks from the **active phase** (defined in CLAUDE.md "Aktiver Scope") +- When the last open task of a phase completes: orchestrator **stops unconditionally** and informs the user that all DoD items are done and a phase transition is possible +- The orchestrator **never** triggers `/project:phase-transition` itself or starts tasks from the next phase +- Even in `at-end` mode: phase boundary = hard stop, no override possible + +This is an implicit fourth checkpoint type: the **phase checkpoint**, which always applies regardless of selected mode. + +## Relationship to session-start / session-end + +The batch orchestrator **replaces** the manual session-start/end cycle for batch work: + +- **session-start is not required** before run-batch. The orchestrator reads task.md and + CLAUDE.md directly — it does not need the user to restore context manually. +- **session-end is incorporated** into the orchestrator's consolidation step (Step 4). + The consolidation performs the same updates as session-end: status.md, decisions-log.md, + task.md checkboxes. A separate session-end after run-batch is not needed. +- **The user may still use session-start** before run-batch if they want to review context + first — but it is optional, not required. + +In short: `/project:run-batch` is a self-contained replacement for the +`session-start → work → session-end → clear → commit` cycle. + +## Orchestrator Flow + +``` +User: "Work through the open discovery tasks" (or /project:run-batch) + │ + ├─ 1. Read task.md of active phase, parse open tasks + │ + ├─ 1.5. Permission check: + │ • Verify required tools are allowed (Read, Write, Edit, Glob, Grep, Agent) + │ • If permissions missing → inform user, suggest settings.json config, abort + │ + ├─ 2. Check batch parameters — anything missing? → Ask interactively: + │ • "Which tasks? (all open / specific items / range)" + │ • "Checkpoint mode? (after-each / after-batch:N / at-end)" + │ • Show summary + wait for confirmation + │ + ├─ 3. Process groups sequentially: + │ │ + │ ├─ Group 1: + │ │ ├─ Dispatch tasks in parallel (subagents) + │ │ ├─ Wait for completion + │ │ ├─ On subagent failure: mark task as failed, include error + │ │ │ in checkpoint summary, do not block other parallel tasks + │ │ ├─ Checkpoint per selected mode + │ │ └─ On "stop" → consolidation → end + │ │ + │ ├─ Group 2: + │ │ └─ ... (only after Group 1 completes + checkpoint passes) + │ │ + │ └─ Phase boundary reached? + │ → HARD STOP, inform user + │ + ├─ 4. Consolidation (orchestrator only, never subagents): + │ • task.md: mark completed items [x], failed items stay [ ] + │ • status.md: update current state (include failures if any) + │ • decisions-log.md: add entries if applicable + │ • Phase-specific shared artifacts (e.g., capabilities-inventory.md + │ in Discovery): update with consolidated findings from subagents + │ + └─ 5. Commit proposal to user +``` + +## Subagent Contract + +### Context Loading Strategy + +Each subagent needs context to do its work. The orchestrator determines context through +a layered approach: + +**Base context (always loaded):** +- Project rules of the active phase (framework-constraints, language-conventions, parallel-agents) +- status.md of the active phase (current state summary) + +**Phase-specific context (loaded by convention):** +- If phase-specific rules define context requirements (e.g., `discovery-waves.md` specifies + "Wave 2 loads structure-map.md"), the orchestrator follows those rules. + +**Task-specific context (optional, from task.md):** +- Tasks in task.md can declare explicit context dependencies: + ```markdown + - [ ] domain-model.md (context: structure-map.md, entry-points.md) + ``` +- If no explicit context is declared: the orchestrator loads all existing output files + from prior groups in the current batch as context. + +**Context budget principle:** The orchestrator always prefers condensed documents +(structure-map.md, status.md) over raw sources. Raw code is only passed when +the task explicitly requires it (e.g., a module deep-dive task references a source path). + +### Input (provided by orchestrator) + +- Exactly one task with clear assignment +- Context files per the loading strategy above +- Output file path +- Project rules (framework-constraints, language-conventions, parallel-agents) + +### Output (returned to orchestrator) + +- Output file written to disk at the specified path +- Summary (2-3 sentences) as return value to orchestrator + +### Boundaries + +- Subagent writes **only** its own output file — never shared state +- Subagent does **not** update task.md, status.md, or decisions-log.md +- Orchestrator reads **only** subagent summaries — never full output files + +## Skill Interface + +### Command + +``` +/project:run-batch → asks everything interactively +/project:run-batch items:1-3 checkpoint:after-each +/project:run-batch items:all checkpoint:at-end +/project:run-batch items:wave-3 checkpoint:after-batch:2 +``` + +### Natural Language Triggers (examples) + +``` +"Arbeite die nächsten 3 offenen Tasks ab, zeig mir nach jedem das Ergebnis" +"Führe alle offenen Wave-3-Module parallel durch" +"Mach die Discovery-Tasks durch, Checkpoint am Ende" +``` + +### Interactive Parameter Query + +When parameters are missing or incomplete, the orchestrator asks before starting: + +``` +Orchestrator: "Ich sehe 5 offene Tasks in Phase 1 — Discovery: + 1. [ ] structure-map.md (Wave 1) + 2. [ ] entry-points.md (Wave 2) + 3. [ ] Module: auth (Wave 3) + 4. [ ] Module: invoicing (Wave 3) + 5. [ ] Module: reporting (Wave 3) + +Welche Tasks soll ich abarbeiten? (alle / Nummern / Bereich) +Checkpoint-Modus? (after-each / after-batch:N / at-end)" +``` + +## Parallelization Rules + +- Tasks under the same heading = one parallelization group +- Within a group: all tasks dispatched simultaneously as subagents +- Between groups: strictly sequential — Group 2 starts only after Group 1 fully completes AND checkpoint passes +- Checkpoint mode applies **per group**, not per individual task within parallel groups + +## Permissions + +Required permissions in `.claude/settings.json`: + +```json +{ + "permissions": { + "allow": [ + "Read", + "Glob", + "Grep", + "Edit", + "Write", + "Agent" + ] + } +} +``` + +- Configured once in settings.json, no runtime flags needed +- Subagents inherit the parent session's permission context +- Orchestrator checks permissions at startup (Step 1.5) and aborts with guidance if missing + +## Artifacts + +| Artifact | Purpose | +|----------|---------| +| `rules/batch-orchestration.md` | Rules for orchestrator/subagent contract | +| `.claude/commands/run-batch.md` | The skill/command definition | +| `.claude/settings.json` | Permission configuration | +| `docs/superpowers/specs/2026-03-21-batch-orchestrator-design.md` | This spec | +| `docs/superpowers/specs/2026-03-21-batch-orchestrator-future.md` | Future improvements | + +## Decisions Made + +| Decision | Rationale | +|----------|-----------| +| In-session orchestrator with subagents (not external script) | Keeps checkpoints interactive, stays in Claude ecosystem | +| Slim orchestrator (summaries only, no full output reads) | Prevents context overflow with many tasks | +| task.md as single source of truth | No redundant runbook file, existing structure is machine-readable | +| Heading-based parallelization groups | Already matches existing task.md structure | +| Phase boundary = hard stop | Phase transitions require human intent per framework rules | +| Missing parameters always queried interactively | Prevents accidental batch runs with wrong settings | +| Permission check at startup | Prevents hanging on permission prompts mid-batch | +| Default checkpoint mode: after-each | Safest option, user opts into more autonomy explicitly | +| Orchestrator parses work section (with ###), not DoD section | Avoids ambiguity between actionable tasks and acceptance criteria | +| run-batch replaces session-start/end cycle | Eliminates redundant manual steps; consolidation is built into the flow | +| Layered context loading for subagents | Base context always available; phase-specific and task-specific context keeps the approach generic across phases | diff --git a/docs/superpowers/specs/2026-03-21-batch-orchestrator-future.md b/docs/superpowers/specs/2026-03-21-batch-orchestrator-future.md new file mode 100644 index 0000000..1229de6 --- /dev/null +++ b/docs/superpowers/specs/2026-03-21-batch-orchestrator-future.md @@ -0,0 +1,39 @@ +# Batch Orchestrator — Future Improvements + +Improvements identified during design but explicitly out of scope for the initial implementation. + +## Live Subagent Streaming + +**What:** Real-time visibility into subagent work as it happens, not just after completion. + +**Why deferred:** Claude Code subagents currently return results only on completion. Live streaming would require changes to the Agent tool or a workaround (e.g., file-watching with periodic reads). + +**Possible approaches:** +- Subagent writes incremental progress to a `.progress` file; orchestrator polls periodically +- Future Claude Code feature: streaming subagent output to parent + +## Cross-Wave Parallelization + +**What:** When Wave 3 Module A completes, start Wave 4 work for Module A while Wave 3 Module B is still running. + +**Why deferred:** Creates complex dependency graphs and merge conflicts on shared output files. The sequential group model is simpler and sufficient for current needs. + +**Prerequisite:** Clear per-module output isolation across phases, not just within a phase. + +## Automatic Retry on Subagent Failure + +**What:** If a subagent fails (context overflow, tool error), automatically retry with adjusted parameters. + +**Why deferred:** Failure modes need to be observed in practice first before designing retry logic. Current approach: orchestrator reports failure, user decides. + +## Batch Resume After Interruption + +**What:** If a batch is interrupted (crash, user closes VS Code), resume from where it left off. + +**Why deferred:** task.md checkboxes already track completion. A partial batch leaves some items `[x]` and others `[ ]`. Restarting `/project:run-batch` naturally picks up remaining items. Explicit resume logic adds complexity with little benefit over this natural behavior. + +## Dynamic Parallelization Hints + +**What:** Allow task.md to contain explicit parallelization overrides beyond the heading-based grouping (e.g., `` to force sequential within a group). + +**Why deferred:** Heading-based grouping covers known use cases. Override syntax adds complexity that may never be needed.