summaryrefslogtreecommitdiff
path: root/Omni
AgeCommit message (Collapse)Author
2025-11-20doc: add design docs for open epicsBen Sima
- Omni/Task/DESIGN.md: Task manager improvements - Biz/PodcastItLater/DESIGN.md: Architecture and features - Biz/PodcastItLater/TESTING.md: Test strategy - Network/Wai/Middleware/Braid/DESIGN.md: Keep-alive design
2025-11-20doc: update agent design to require timestamps in logsBen Sima
2025-11-20doc: add design document for Multi-Agent System 2.0Ben Sima
2025-11-20fix: remove null noise from worker monitorBen Sima
- Use 'empty' in jq to properly filter ignored log lines
2025-11-20fix: enable debug logs for worker agentBen Sima
- Allows monitoring tool execution details in amp.log
2025-11-20feat: automate worker workflow in bashBen Sima
- Replaced agent-driven workflow with robust bash script - Script handles task claiming, branching, syncing, and submission - Agent is now only responsible for implementation and testing - Eliminates context loss and process adherence issues
2025-11-20fix: merge local live branch in worker guideBen Sima
- We share the .git directory, so we can merge 'live' directly - Avoids dependency on 'origin' which might be stale or unused
2025-11-20fix: prevent data loss in worker loopBen Sima
- Changed 'git reset --hard' to 'git merge' in worker guide - This prevents the worker from wiping its own status updates (commits to base branch) when starting a new task loop
2025-11-20fix: handle existing task branches in worker guideBen Sima
- Workers should reuse existing branches if they crashed/restarted - Changed 'git checkout -b' to 'git checkout || git checkout -b'
2025-11-20fix: instruct worker to reset to latest live codeBen Sima
- Workers need to pull the latest merged code before starting a new task - Added 'git reset --hard origin/live' to Step 1 of the loop
2025-11-20Merge branch 'task/t-1gcR9RV' into liveBen Sima
2025-11-20Move agent log file to cabdirBen Sima
2025-11-20feat: implement task harvesting logicBen Sima
- Added Omni/Agent/harvest-tasks.sh for Planner to pull updates from workers - Updated WORKER_AGENT_GUIDE.md to instruct workers to commit status updates to their base branch
2025-11-20fix: capture worker agent logs to fileBen Sima
- Added --log-file amp.log to worker command - Allows users to monitor agent progress with tail -f
2025-11-20chore: task breakdown and cleanupBen Sima
- Updated Omni/Task/Core.hs to filter parent tasks from 'task ready' - Broke down 'PodcastItLater: Path to Paid Product' into 5 subtasks - Broke down 'General Code Quality Refactor' into 3 subtasks - Broke down 'Complete comprehensive test suite' into 2 subtasks - Fixed namespace naming in tasks.jsonl
2025-11-20feat: implement t-1gcR9RVBen Sima
Amp-Thread-ID: https://ampcode.com/threads/T-c683858e-5ed5-4d8e-8b9f-842ebdcec0a3 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20fix: enable dangerously-allow-all for worker agentBen Sima
- The autonomous worker needs to execute commands without interactive prompts - Added --dangerously-allow-all flag to start-worker.sh
2025-11-20fix: improve start-worker.sh scriptBen Sima
- Use 'git worktree list' to reliably find main repo and amp binary - Run amp in a loop with -x flag for autonomous execution - Support flexible worker path resolution
2025-11-20Fix duplicate worktree commandBen Sima
2025-11-20feat: implement multi-agent workflow infrastructureBen Sima
- Add 'Review' status to Task tool - Add Omni/Agent/ directory with setup and sync scripts - Add WORKER_AGENT_GUIDE.md - Configure custom merge driver for tasks.jsonl
2025-11-20bild: fix --plan to output only JSON without loggingBen Sima
- Remove Log.wipe from --plan output path - Add isPlanMode parameter to analyzeAll to skip LogC.withLineManager when planning - Update all analyzeAll call sites to pass isPlanMode flag - Remove grep workaround from repl.sh (no longer needed) Now 'bild --plan' outputs clean JSON that can be piped directly to jq. This fixes typecheck.sh and other tools that rely on clean JSON output. Amp-Thread-ID: https://ampcode.com/threads/T-4e6225cf-3e78-4538-963c-5377bbbccee8 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20typecheck: add Haskell support to typecheck.shBen Sima
- Implement Haskell typechecking using 'ghc -fno-code' - Use repl.sh to provision correct GHC environment with dependencies - Follows same pattern as Python typechecking (via repl.sh) - Typechecks without generating code (fast feedback) Now typecheck.sh supports both Python (.py) and Haskell (.hs) files. Amp-Thread-ID: https://ampcode.com/threads/T-4e6225cf-3e78-4538-963c-5377bbbccee8 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20task: implement epic progress trackingBen Sima
- Add progress display to 'task show' for epics (X/Y with percentage) - Add progress display to 'task list --type=epic' showing [X/Y] - Progress already shown in 'task tree' as [X/Y] - Calculate completed/total child tasks for epics - Clean up test tasks accidentally created in production database All 31 tests passing. Amp-Thread-ID: https://ampcode.com/threads/T-4e6225cf-3e78-4538-963c-5377bbbccee8 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20task: implement stats commandBen Sima
- Add 'task stats' command to show task statistics - Display total tasks, status breakdown (open/in-progress/done) - Show epic count, ready vs blocked tasks - Show task counts by priority (P0-P4) and namespace - Support both human-readable and JSON output (--json flag) - Add tests for stats command and stats --json - TaskStats data type with ToJSON/FromJSON instances All 31 tests passing. Amp-Thread-ID: https://ampcode.com/threads/T-4e6225cf-3e78-4538-963c-5377bbbccee8 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20task: prettier tree visualization and flag ordering fixesBen Sima
- Implement box-drawing characters (├──, └──, │) for task tree visualization - Fix 'task create' flag ordering by using [options] in docopt (same as 'task list') - Document TASK_TEST_MODE environment variable in AGENTS.md Testing section - Add test case for multi-flag ordering on 'task create' - Clean up test tasks polluted in production database All 29 tests passing. Amp-Thread-ID: https://ampcode.com/threads/T-4e6225cf-3e78-4538-963c-5377bbbccee8 Co-authored-by: Amp <amp@ampcode.com>
2025-11-20Fix docopt flag ordering issue by using [options]Ben Sima
- Changed 'task list' usage from explicit flags to [options] - Allows flags to be specified in any order (fixes t-10KNtTF) - All combinations now work: --namespace --status, --status --namespace, etc. - Updated AGENTS.md examples to use correct status argument format - All tests passing (28/28)
2025-11-20Add task show command for detailed task inspectionBen Sima
- Implemented 'task show <id>' command with human-readable output - Shows all task fields: title, type, status, priority, timestamps - Displays dependencies with their types - Supports --json flag for programmatic use - Added CLI tests for show command - Includes priority descriptions (Critical/High/Medium/Low/Backlog)
2025-11-20task: sync databaseBen Sima
2025-11-16Fix repl.sh and typecheck.sh to properly parse bild --plan JSONBen Sima
The issue was that bild --plan outputs progress indicators before the JSON, causing jq to fail parsing. Fixed by: - Using grep to extract only lines starting with '{' (the JSON output) - This filters out progress lines like '[…] target' - Restored typecheck.sh to use repl.sh for proper environment setup Now typecheck.sh correctly provisions the environment via repl.sh instead of trying to use bild or raw python.
2025-11-16Add 'Add to feed' button on episode pages and fix typecheck.shBen Sima
- Episode pages now show 'Add to feed' button for logged-in users who don't have the episode - Added POST /episode/{id}/add-to-feed endpoint - Tracks 'added' metric when user adds episode to their feed - Added Database.track_episode_metric() function for metrics tracking - Fixed typecheck.sh to use bild instead of broken repl.sh approach Tasks completed: t-gc9aud, t-gcbqDl
2025-11-15Fix run.sh to handle bild --plan JSON outputBen Sima
The bild --plan command now outputs progress indicators before JSON. Updated run.sh to use 'tail -1' to extract only the JSON line and fixed the file path check to use the full path. Amp-Thread-ID: https://ampcode.com/threads/T-cc5d29f0-454e-4864-8d7e-1ad69a42afa9 Co-authored-by: Amp <amp@ampcode.com>
2025-11-15Expand terminalLock scope to protect all terminal I/OBen Sima
Problem: Still getting segfaults ("free(): invalid pointer") even with ncurses calls protected. The mutex only covered ANSI calls, but IORef reads and IO.hPutStr operations were happening outside the lock. Root cause: Race conditions in concurrent terminal output: 1. Multiple threads reading namespaceLines IORef concurrently 2. Interleaved IO.hPutStr calls corrupting output buffer 3. Rainbow.hPutChunks not being thread-safe Solution: Expand terminalLock scope to cover entire output operations: - Move IORef reads inside the lock - Protect all IO.hPutStr and Rainbow.hPutChunks calls - Lock both SingleLine and MultiLine modes - Ensure atomicity from IORef read through all I/O to flush This makes each updateLine/updateLineState call atomic, preventing any interleaving of terminal operations between threads. Changes: - updateLine SingleLine: wrap entire output in withMVar terminalLock - updateLine MultiLine: move nsMap read inside lock - updateLineState SingleLine: wrap entire output in withMVar terminalLock - updateLineState MultiLine: move nsMap read inside lock Tested: 10/10 successful runs of `bild --time 0 **/*` without segfaults.
2025-11-15Improve terminalLock comment for clarityBen Sima
Clarify that the mutex protects against ncurses thread-safety issues that cause segfaults during concurrent builds. This commit also forces a new Nix derivation hash to ensure the mutex fix is actually used (previous builds were cached with the old version).
2025-11-15Fix segfault: add mutex for ANSI terminal operationsBen Sima
Problem: Intermittent segfaults when running `bild --time 0 **/*` with many concurrent builds. Core dumps showed crashes in libncursesw's free() function during terminal cleanup. Root cause: ANSI.getTerminalSize and other ANSI terminal library calls are not thread-safe. With mapConcurrentlyBounded running up to 8 analyses concurrently, multiple threads were calling ANSI functions simultaneously, causing memory corruption in the ncurses library. Solution: Add global MVar terminalLock to serialize all ANSI terminal operations. Wrap all ANSI function calls (cursor movement, line clearing, etc.) with withMVar terminalLock. Changes: - Add terminalLock :: MVar () in Omni/Log/Concurrent.hs - Wrap all ANSI calls in withMVar terminalLock: - initializeLines: cursor column, clear line - updateLine: cursor up/down, column set, clear line - updateLineState: cursor up/down, column set, clear line - withLineManager: cursor up Tested: 5 consecutive runs of `bild --time 0 **/*` complete without segfaults (previously failed 1-2 out of 3 runs).
2025-11-15Fix NixOS integration: separate package building from OS buildsBen Sima
Problem: Calling bild.run inside NixOS configs triggered IFD during OS evaluation. ANSI escape codes from bild broke JSON parsing in Nix sandbox, causing build failures. Root cause: bild.run uses IFD (Import From Derivation) which runs bild --plan during Nix evaluation. When this happened inside NixOS service definitions, it ran recursively and bild output ANSI codes that corrupted the JSON analysis output. Solution: Two-phase architecture + NO_COLOR support 1. Biz/Packages.nix: Pre-builds all packages outside NixOS context 2. Biz.nix: Accepts packages as function argument (default: Packages.nix) 3. Omni/Bild.nix: Sets NO_COLOR=1 in analysis derivation 4. Omni/Log/Terminal.hs: Respects NO_COLOR env var 5. Omni/Log/Terminal.hs: Skip getTerminalSize when NO_COLOR set to avoid escape code output 6. Omni/Log/Concurrent.hs: Skip line initialization without ANSI support Now NixOS builds succeed: - Package IFD happens once at top level - No recursive builds during service evaluation - Clean JSON output from bild --plan in Nix sandbox - NixOS configs reference pre-analyzed packages Changes: - Add Biz/Packages.nix - standalone package builder - Update Biz.nix to accept packages argument - Update Biz/Dragons/Analysis.nix to use Packages.nix - Remove Biz/Targets.nix (replaced by Packages.nix) - Add NO_COLOR support throughout logging stack - Fix ANSI.getTerminalSize outputting escape codes when NO_COLOR set Amp-Thread-ID: https://ampcode.com/threads/T-bc0f6fc7-46bf-4aa2-892e-dd62e7251d4b Co-authored-by: Amp <amp@ampcode.com>
2025-11-15fix: handle stdin EOF in terminal detection and subprocess spawningBen Sima
ANSI.getTerminalSize queries the terminal by writing escape codes and reading the response from stdin. When stdin is unavailable or at EOF (common in non-interactive contexts), this causes "hWaitForInput: end of file" errors. Additionally, Conduit.streamingProcess returns a stdin handle that was never being closed, which could cause child processes to block waiting for input. Fixes: - Catch IOException in detectTerminal when getTerminalSize fails - Close subprocess stdin handle immediately after spawn - Fallback to default terminal size (80x24) when detection fails This unblocks all bild commands that were failing with stdin errors.
2025-11-15refactor(bild): simplify terminal output - use --loud flagBen Sima
- Remove BILD_OUTPUT_MODE environment variable - Simplify to 2 modes: MultiLine (≥80 cols) and SingleLine (<80) - Use existing --loud flag to disable concurrent UI entirely - When --loud: show all compiler output line-by-line (for debugging) - When not --loud: adaptive concurrent UI based on terminal width This is simpler and uses the existing --loud flag instead of adding new configuration. The --loud flag was already meant for debugging, so it makes sense to use it to show all output. Note: Omni/Bild.nix updated to include new Omni/Log/Terminal.hs module. Blocked by: stdin bug (see _/llm/STDIN_BUG.md) - cannot test build yet.
2025-11-15feat(bild): adaptive terminal output with width detectionBen Sima
- Add Omni/Log/Terminal module to detect terminal capabilities - Implement 3 output modes: RichMultiLine (≥80 cols), SingleLine (40-79), SimpleFallback (<40 or dumb terminals) - Terminal width auto-detection via System.Console.ANSI.getTerminalSize - Text truncation with ellipsis to prevent line wrapping - Manual override via BILD_OUTPUT_MODE env var (simple|single|rich|auto) Fixes UI corruption on small terminals (mobile SSH, narrow windows). Wide terminals keep existing multi-line behavior but with truncation. Design doc: _/llm/CONCURRENT_LOG_DESIGN.md
2025-11-14fix(bild): per-module cp overwrites with -f and chmod after each copyBen Sima
When building per-module derivations, copying dependency .hi files was failing silently because: 1. First dep copies read-only Alpha.hi from nix store 2. Second dep also has Alpha.hi (transitive dep) and tries to overwrite 3. cp fails with permission denied but error is hidden by '|| true' Fixed by: - Add -f flag to cp to force overwrite - Run chmod -R +w after each dependency copy - This allows later deps to overwrite shared files like Alpha.hi Tested: Biz/Que/Host.hs now builds successfully with per-module derivations
2025-11-14fix(bild): work around Nix patch-shebangs.sh bugBen Sima
Set dontPatchShebangs=true for per-module link derivation to avoid unbound variable error in Nix's patch-shebangs.sh script. The script declares 'local update' but never initializes it, causing bash set -u to fail. Our ELF binaries don't need shebangs patched anyway. Tested: All Haskell targets now build successfully including Omni/Lint.hs which uses makeWrapper with rundeps.
2025-11-14fix(bild): per-module builds + exit code propagationBen Sima
- Fix nixBuild to propagate exit code from realise step - Previously used >> which discarded exit code - Now checks realise result before running symlink - Fixes false success checkmarks on build failures - Fix per-module Nix derivations - Use cp -rL instead of tar (src is directory not tarball) - Add coreutils and findutils to pkgs - Copy deps to . and use -i. for GHC - Use find with --parents to preserve module hierarchy - Set dontStrip=true to avoid fixup script errors - Tested: Example.hs, Task.hs, Dragons.hs, Bild.hs all build - Known issue: makeWrapper fixup scripts have unbound vars (Nix bug)
2025-11-14Fix module builder: manually unpack tarballBen Sima
When using custom builder, unpackPhase function is not available. Use tar xzf to manually extract the source archive.
2025-11-14Fix module builder: call unpackPhase explicitly and cd to sourceBen Sima
When using custom builder, standard phases don't run automatically. Call unpackPhase explicitly and cd to 'source' directory where files are unpacked. Fixes 'sourceRoot: unbound variable' error in module compilation.
2025-11-14Fix module compilation: use custom builder to avoid GHC setup hooksBen Sima
The haskell.ghcWith package has setup hooks that override buildPhase even when explicitly set. Solution: use custom builder = stdenv.shell with args instead of relying on mkDerivation phases. Changes: - Module derivations: Use custom builder with single -c script - Combines unpack, build, and install into one script - Explicitly call ghc with full path to avoid hook interference - Remove unused objectPaths computation (now using ghc --make with source) This fixes builds for Omni/Lint.hs, Omni/Task.hs, and all other Haskell targets with complex dependency graphs.
2025-11-14Fix per-module link: copy .hi files locally instead of using -i pathsBen Sima
GHC --make was treating -i /nix/store/.../hidir paths as compilation targets. Solution: Copy all .hi files to local directory and use -i. only. Also make hsGraph optional with 'or null' for backward compatibility with old bild binaries during bootstrap. Amp-Thread-ID: https://ampcode.com/threads/T-fe68faaf-1c1d-4c43-a377-1cf5e6cffb3a Co-authored-by: Amp <amp@ampcode.com>
2025-11-14Fix per-module link phase: put source file before flagsBen Sima
GHC --make interprets arguments before flags as targets. Moving the entry point source file to the beginning prevents -i paths from being treated as compilation targets. This fixes the 'is not a module name or a source file' error.
2025-11-14Implement per-module Nix derivations for incremental Haskell buildsBen Sima
This is the core architecture transformation from Phase 3 of the performance plan. Each Haskell module is now built as a separate Nix derivation, enabling true incremental builds where only changed modules and their dependents are rebuilt. Implementation: - buildHsModuleGraph: Analyzes transitive module dependencies and builds DAG - TH detection: Falls back to monolithic build if Template Haskell detected - SCC cycle detection: Falls back if import cycles found - Per-module Nix builder: Each module -> separate derivation with .hi and .o - Module dependencies: Copy .hi files to build dir, use -i flags for imports - Final link: Use ghc --make with entry point source + -i paths to .hi files - Entry point fix: Explicitly analyze entry point module separately from deps Architecture: - Module compilation: ghc -c with -i paths to dependency .hi files - Source filtering: Each module derivation includes only its source file - Dependency DAG: Expressed as recursive Nix attrset with lib.fix - Link phase: ghc --make with entry source file + all .hi search paths - Fallback: Monolithic ghc --make when hsGraph is null (TH/cycles) Performance characteristics: - Change one module -> rebuild only that + dependents + relink - Nix handles DAG scheduling and caching automatically - Parallel module compilation (Nix orchestrates) - Content-addressed caching across machines Testing: - Added test_buildHsModuleGraph unit test - Verified with Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers correct subset This completes Phase 2 and Phase 3 core optimizations from the plan.
2025-11-14Add per-module Haskell build foundationBen Sima
Implements Steps 1-4 of per-module Nix architecture: - Add HsModuleGraph and HsModuleNode types to represent module DAG - Implement buildHsModuleGraph that: - Parses imports for each module in the transitive closure - Detects Template Haskell usage (language pragma or $( patterns) - Detects import cycles using SCC analysis - Returns Nothing (fallback to monolithic) if TH or cycles found - Wire hsGraph into Target type for all builders - Populate hsGraph for Haskell targets during analysis Module graph is now emitted in --plan output showing per-module dependencies. This enables the next step: per-module Nix derivations in Builder.nix for true incremental builds. Part of Phase 3 from the performance plan.
2025-11-14Add persistent ghc-pkg cache to speed up analysisBen Sima
Implements optimization #3 from Phase 2 of the performance plan. Changes: - Cache stored in _/var/ghc-pkg-cache-<hash>.json - Hash based on GHC version + GHC_PACKAGE_PATH for automatic invalidation - Loads cache at startup, saves on successful completion - Uses atomic write (tmp + rename) to prevent corruption - Gracefully handles missing/corrupt cache files - Accumulates cache entries across builds - Works with parallel builds (in-memory IORef + disk persistence) Performance impact: - Eliminates redundant ghc-pkg invocations across runs - Near-zero ghc-pkg overhead once cache is populated - No impact on single-run performance (still uses in-memory IORef)
2025-11-14Tighten Nix source filtering to prevent spurious rebuildsBen Sima
Only include directories that are ancestors of source files in allSources. Previously accepted all directories, causing rebuilds when any new directory was added to the repo. Implementation: - Precompute normalized source paths and their ancestor directories - Filter directories against allowedDirs whitelist - Normalize paths in file filter for consistency - Keep existing skip list behavior for _ and .direnv This is optimization #2 from Phase 2 of the performance plan.
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-16 15:17:42 +0000