summaryrefslogtreecommitdiff
path: root/Omni
AgeCommit message (Collapse)Author
40 hoursFix module builder: call unpackPhase explicitly and cd to sourceBen Sima
When using custom builder, standard phases don't run automatically. Call unpackPhase explicitly and cd to 'source' directory where files are unpacked. Fixes 'sourceRoot: unbound variable' error in module compilation.
40 hoursFix module compilation: use custom builder to avoid GHC setup hooksBen Sima
The haskell.ghcWith package has setup hooks that override buildPhase even when explicitly set. Solution: use custom builder = stdenv.shell with args instead of relying on mkDerivation phases. Changes: - Module derivations: Use custom builder with single -c script - Combines unpack, build, and install into one script - Explicitly call ghc with full path to avoid hook interference - Remove unused objectPaths computation (now using ghc --make with source) This fixes builds for Omni/Lint.hs, Omni/Task.hs, and all other Haskell targets with complex dependency graphs.
41 hoursFix per-module link: copy .hi files locally instead of using -i pathsBen Sima
GHC --make was treating -i /nix/store/.../hidir paths as compilation targets. Solution: Copy all .hi files to local directory and use -i. only. Also make hsGraph optional with 'or null' for backward compatibility with old bild binaries during bootstrap. Amp-Thread-ID: https://ampcode.com/threads/T-fe68faaf-1c1d-4c43-a377-1cf5e6cffb3a Co-authored-by: Amp <amp@ampcode.com>
42 hoursFix per-module link phase: put source file before flagsBen Sima
GHC --make interprets arguments before flags as targets. Moving the entry point source file to the beginning prevents -i paths from being treated as compilation targets. This fixes the 'is not a module name or a source file' error.
42 hoursImplement per-module Nix derivations for incremental Haskell buildsBen Sima
This is the core architecture transformation from Phase 3 of the performance plan. Each Haskell module is now built as a separate Nix derivation, enabling true incremental builds where only changed modules and their dependents are rebuilt. Implementation: - buildHsModuleGraph: Analyzes transitive module dependencies and builds DAG - TH detection: Falls back to monolithic build if Template Haskell detected - SCC cycle detection: Falls back if import cycles found - Per-module Nix builder: Each module -> separate derivation with .hi and .o - Module dependencies: Copy .hi files to build dir, use -i flags for imports - Final link: Use ghc --make with entry point source + -i paths to .hi files - Entry point fix: Explicitly analyze entry point module separately from deps Architecture: - Module compilation: ghc -c with -i paths to dependency .hi files - Source filtering: Each module derivation includes only its source file - Dependency DAG: Expressed as recursive Nix attrset with lib.fix - Link phase: ghc --make with entry source file + all .hi search paths - Fallback: Monolithic ghc --make when hsGraph is null (TH/cycles) Performance characteristics: - Change one module -> rebuild only that + dependents + relink - Nix handles DAG scheduling and caching automatically - Parallel module compilation (Nix orchestrates) - Content-addressed caching across machines Testing: - Added test_buildHsModuleGraph unit test - Verified with Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers correct subset This completes Phase 2 and Phase 3 core optimizations from the plan.
43 hoursAdd per-module Haskell build foundationBen Sima
Implements Steps 1-4 of per-module Nix architecture: - Add HsModuleGraph and HsModuleNode types to represent module DAG - Implement buildHsModuleGraph that: - Parses imports for each module in the transitive closure - Detects Template Haskell usage (language pragma or $( patterns) - Detects import cycles using SCC analysis - Returns Nothing (fallback to monolithic) if TH or cycles found - Wire hsGraph into Target type for all builders - Populate hsGraph for Haskell targets during analysis Module graph is now emitted in --plan output showing per-module dependencies. This enables the next step: per-module Nix derivations in Builder.nix for true incremental builds. Part of Phase 3 from the performance plan.
43 hoursAdd persistent ghc-pkg cache to speed up analysisBen Sima
Implements optimization #3 from Phase 2 of the performance plan. Changes: - Cache stored in _/var/ghc-pkg-cache-<hash>.json - Hash based on GHC version + GHC_PACKAGE_PATH for automatic invalidation - Loads cache at startup, saves on successful completion - Uses atomic write (tmp + rename) to prevent corruption - Gracefully handles missing/corrupt cache files - Accumulates cache entries across builds - Works with parallel builds (in-memory IORef + disk persistence) Performance impact: - Eliminates redundant ghc-pkg invocations across runs - Near-zero ghc-pkg overhead once cache is populated - No impact on single-run performance (still uses in-memory IORef)
44 hoursTighten Nix source filtering to prevent spurious rebuildsBen Sima
Only include directories that are ancestors of source files in allSources. Previously accepted all directories, causing rebuilds when any new directory was added to the repo. Implementation: - Precompute normalized source paths and their ancestor directories - Filter directories against allowedDirs whitelist - Normalize paths in file filter for consistency - Keep existing skip list behavior for _ and .direnv This is optimization #2 from Phase 2 of the performance plan.
46 hoursImplement concurrent analysis with [+] state indicatorBen Sima
- Add Analyzing state to BuildState enum - Refactor from sequential foldM analyze to concurrent analyzeAll - Initialize all lines with [+] during analysis phase - Update to [...] (Pending) after each analysis completes - Uses mapConcurrentlyBounded with concurrency of 8 for analysis - Remove Log.info from analyzeOne (now handled by line state) - Analysis now runs in parallel, improving efficiency - Flow: [+] analyzing → [...] pending → [~] building → [✓]/[x] complete
46 hoursRemove blank lines after build completesBen Sima
- Replace cursor down movement with single newline at end - Cursor now stays at bottom of status lines - No extra blank space between build output and next prompt
46 hoursFix cursor movement to avoid updating wrong linesBen Sima
- Remove save/restore cursor in favor of explicit movement - Always move up from bottom position, update line, move back down - Simplify initializeLines to just print lines sequentially - Cursor now stays at bottom and moves up/down for each update - Fixes issue where lines were being updated far up the terminal
46 hoursFix cursor positioning to preserve shell promptBen Sima
- Use hCursorDown in initializeLines instead of hCursorUp - Simplify updateLine and updateLineState cursor movement - Cursor now moves up from bottom position correctly - Shell prompt no longer gets erased
47 hoursRedesign LineManager to show one line per namespaceBen Sima
- Allocate one line per namespace (not per concurrent job) - Add Pending state shown as [...] when build hasn't started - Initialize all namespace lines at start showing [...] - Update to [~] when building, [✓]/[x] when complete - Each namespace keeps its line throughout the build - At end, all namespaces show their final status - --jobs controls concurrency, not line count
47 hoursFix concurrent terminal update issuesBen Sima
- Remove [+] display from reserveLine (was causing mangled output from concurrent writes) - Add [+] display in analyze function where it's single-threaded - Simplify cleanup to just move cursor down without clearing lines - This eliminates race conditions and terminal clearing issues
47 hoursFix prompt preservation and cleanup extra lines at endBen Sima
- Add initial newline to preserve terminal prompt - Clear each line individually at end instead of just moving cursor - This prevents extra blank lines from remaining on screen
47 hoursFix blank lines in parallel build outputBen Sima
- Remove initial blank line before build starts - Remove trailing blank line after build completes - Remove newlines from status indicators in releaseLine - Status lines now stay on their reserved lines without extra spacing
47 hoursClean up build output and add [+] for analyzing phaseBen Sima
- Remove all label prefixes from build logs (bild:, nix:, etc) - Show [+] Namespace when first reserving a line (analyzing) - Show [~] Namespace: output when building with subprocess output - Show [✓] Namespace (green) on success - Show [x] Namespace (red) on failure
47 hoursRemove info and good log messages from parallel build outputBen Sima
- Remove Log.info from analyze function - Remove Log.info from build target compilers (Guile, NixBuild, Rustc, Sbcl) - Remove Log.good from proc and nixBuild success handlers - Remove 'info: bild:' prefix from streaming subprocess output - Output now shows only [✓]/[x]/[~] status lines with subprocess stdout
48 hoursFix parallel build output format with status indicators and colorsBen Sima
- Add initial line break to preserve terminal prompt - Change format to [✓]/[x]/[~] Namespace: output - Add colors: green for success, red for failure - Fix extra lines by adding newlines in releaseLine - Fix currentLine initialization to 0 instead of maxLines Amp-Thread-ID: https://ampcode.com/threads/T-39671965-c412-4a2e-8084-9d09128fd865 Co-authored-by: Amp <amp@ampcode.com>
2 daysAdd ansi-terminal dep and Omni/Log/Concurrent to Bild.nixBen Sima
- Add ansi-terminal to ghcPackageSetBild dependencies - Add Omni/Log/Concurrent.hs to source fileset - Fix unused mLineNum warning in buildTarget - Alphabetize deps list (hostname moved up) Parallel builds now fully functional with proper Nix dependencies.
2 daysComplete multi-line concurrent logging integrationBen Sima
- Fix BuildStatus to use newtype with single field - Clean up unused imports and fields - logs function now uses updateCurrentLine for LineManager support - Fallback to single-line output when no LineManager All tests passing. Ready to test with different terminal types. Tasks: t-1a1Eiay
2 daysIntegrate LineManager with logging systemBen Sima
- Add global IORef for currentLineManager and namespaceLines mapping - Update logs function to use LogC.updateCurrentLine - Add updateCurrentLine and releaseCurrentLine helpers - Fallback to normal printing when no LineManager active - Simplify buildTarget to use global helpers instead of threading Tasks: t-1a1EaJy
2 daysThread LineManager through build functionsBen Sima
- Reserve line for each concurrent build - Release line on completion with success/failed state - Fix hlint eta reduce warning Task: t-1a1E3j1
2 daysCreate Omni/Log/Concurrent module for multi-line outputBen Sima
- Implement LineManager abstraction with IORef state - Line reservation/update/release functions - ANSI cursor positioning for concurrent updates - Terminal capability detection (ANSI vs dumb) - Graceful fallback for non-ANSI terminals Tasks: t-1a1DzES, t-1a1DGY0, t-1a1DOev, t-1a1DVM5
2 daysAdd bounded parallel target buildsBen Sima
- Add mapConcurrentlyBounded using QSemN for bounded parallelism - Refactor build function to extract buildTarget worker - Build up to --jobs targets concurrently - Preserves all existing functionality - Output will be interleaved (will fix with LineManager next) Related tasks: - t-1a0OVBs: mapConcurrentlyBounded helper - t-1a16ame: refactor build function - t-1a1DdSB: replace forM with concurrent map
2 daysBild: breadth-first search and ghc-pkg cachingBen Sima
Replaced the old slow depth-first search with a breadth-first search for detecting imports. This should be way faster when building a single namespace because it doesn't have to visit the same file multiple times. The ghc-pkg caching means we only have to run ghc-pkg once per bild invocation.
3 daysFix Python import detection to handle transitive dependenciesBen Sima
detectPythonImports now recursively analyzes imported modules to find transitive dependencies, matching the behavior of detectHaskellImports. Previously it only detected direct imports, which caused build failures when Python modules had nested dependencies. - Changed signature from [Text] -> IO (Set FilePath) to Analysis -> [Text] -> IO (Set FilePath) - Added filepaths, findDeps, and onlyPython helper functions - Recursively calls analyze() on imported modules to find transitive deps - Updated tests to pass empty Analysis map
3 daysCleanup some logging setup codeBen Sima
I think the calls to Log.setup() were accidentally creating multiple loggers, hopefully this fixes the problem.
4 daysFix billing migration to work with SQLite ALTER TABLE limitationsBen Sima
- Remove UNIQUE constraints from ALTER TABLE statements (SQLite doesn't support adding constraints via ALTER TABLE) - Add better logging for migration failures (debug level) - Rely on application logic to ensure uniqueness instead of DB constraint - This fixes the silent failure where stripe_customer_id and stripe_subscription_id columns weren't being added
4 daysAdd complete Stripe billing integration to PodcastItLaterBen Sima
- Implement Biz.PodcastItLater.Billing with checkout sessions, billing portal, webhook handling - Add subscription database schema: plan_tier, stripe fields, period dates, stripe_events table - Three-tier pricing: free (10/month), personal (/month, 50 articles), pro (9/month, unlimited) - Usage tracking and enforcement with tier-based limits - Full billing UI with plan display, usage stats, pricing cards, upgrade buttons - Dashboard shows current tier with billing button - Update Web.nix with Stripe environment variables - Fix POST redirects to Stripe with 303 status code for CloudFront compatibility Amp-Thread-ID: https://ampcode.com/threads/T-c139e5b5-1901-4cd6-8030-5623bfe1df35 Co-authored-by: Amp <amp@ampcode.com>
4 daysFix exit condition for run.shBen Sima
Without this, run.sh would continue if bild failed, and you might end up running an out of date artifact.
7 daysfeat: Add stripe to Python deps and document dependency processBen Sima
- Add stripe to Omni/Bild/Deps/Python.nix (alphabetically sorted) - Fix all type annotations in Billing.py for mypy - Document how to add Python packages in AGENTS.md - Add billing routes to Web.py (checkout, portal, webhook) This enables Stripe integration in PodcastItLater. Related to task t-144e7lF
7 daysShow epic progress count instead of Epic label and checkboxBen Sima
Changed epic display in tree view from: t-PpXWsU [Epic] [ ] Task Manager Improvements To: t-PpXWsU [6/11] Task Manager Improvements The [6/11] shows completed/total child tasks, giving immediate visual feedback on epic progress. Regular tasks still use checkbox indicators: [ ] open, [~] in-progress, [✓] done.
7 daysMove namespace label between status and title in tree viewBen Sima
Changed tree output format from: t-abc123 [ ] Task title [Omni/Task.hs] To: t-abc123 [ ] [Omni/Task.hs] Task title This makes the namespace more prominent and groups all metadata (status + namespace) together before the title.
7 daysImprove task tree visualization displayBen Sima
Changes: 1. Remove [Task] label - only show [Epic] for epics, cleaner output 2. Truncate long titles to fit 80 columns with ... ellipsis 3. Better spacing with type label included in layout calculation Created task for future improvement: prettier box-drawing characters (├──, └──) which would require Data.Tree library investigation. Current output is clean and readable within standard terminal width.
7 daysImplement task tree visualization commandBen Sima
Add 'task tree' command to show hierarchical task structure: Usage: task tree # Show all epics with their children task tree <id> # Show specific epic/task with children Features: - Visual status indicators: [ ] open, [~] in-progress, [✓] done - Shows task type: [Epic] or [Task] - Indented display for hierarchy - Shows namespace associations Example output: t-PpXWsU [Epic] [ ] Task Manager Improvements [Omni/Task.hs] t-PpYZt2 [Task] [ ] Implement child ID generation t-PpZGVf [Task] [✓] Add filtering by type and parent Updated AGENTS.md with usage examples. Closes task t-PpZlbL
7 daysFix task auto-commit with pre-commit hookBen Sima
Implemented proper pre-commit hook that: - Calls 'task export --flush' to consolidate tasks - Auto-stages .tasks/tasks.jsonl if modified - Runs before every commit Added reminder message after 'task update' to inform users that task changes will be committed on next git commit. Updated AGENTS.md to document the auto-commit behavior. This fixes the bug where task status updates (e.g., marking tasks as Done) were not being committed to git.
7 daysAdd enhanced filtering to task list commandBen Sima
Implement --status and --namespace filters for task list: New filters: - --status: Filter by open, in-progress, or done - --namespace: Filter by namespace (e.g., Omni/Task) All filters can be combined: - task list --parent=t-abc123 --status=open - task list --type=epic --status=done - task list --namespace="Omni/Task" --status=open Updated listTasks signature to accept all filter parameters and apply them in sequence. Updated AGENTS.md with examples. Closes task t-PpZGVf
7 daysProtect production task database from tests and add migrationBen Sima
- Add TASK_TEST_MODE environment variable to use separate test database - All file operations now use getTasksFilePath to respect test mode - Tests use .tasks/tasks-test.jsonl instead of production database - Add automatic migration from old task format (taskProject field) to new format - Migrated tasks convert taskProject to WorkTask type with empty parent - Old [Text] dependencies converted to [Dependency] with Blocks type - Restore actual tasks from commit 3bf1691 (were lost during testing) This prevents accidental data loss when running tests and provides backward compatibility for existing task databases.
7 daysAdd enhanced dependency types to task managerBen Sima
Implement four dependency types based on beads patterns: - Blocks: Hard dependency, blocks ready work queue (default) - DiscoveredFrom: Work discovered during implementation (doesn't block) - ParentChild: Epic/task relationships (blocks ready work) - Related: Soft relationships (doesn't block) Key changes: - New Dependency data type with depId and depType fields - New DependencyType enum with four relationship types - Updated CLI with --dep-type and --discovered-from flags - Enhanced getReadyTasks to respect only blocking dependency types - Added comprehensive tests for all dependency behaviors - Updated AGENTS.md with usage examples and patterns The discovered-from pattern is especially important for AI agents to maintain context of work found during implementation while keeping it immediately available in the ready work queue. Amp-Thread-ID: https://ampcode.com/threads/T-178b273a-3ac7-416c-a964-db89bac3c8f7 Co-authored-by: Amp <amp@ampcode.com>
7 daysImplement epic and task types to replace project fieldBen Sima
Major refactoring of task data model: - Added TaskType enum (Epic | WorkTask) - Replaced taskProject with taskType and taskParent fields - Epics are containers for tasks (hierarchical organization) - Tasks can have optional parent epics - Updated createTask signature to accept type and parent - Updated CLI: --type=epic|task and --parent=<id> options - Updated list command to filter by type and parent - Updated printTask to display type and parent info - Fixed naming collision (WorkTask instead of Task constructor) Example usage: task create "Auth System" --type=epic task create "Design API" --type=task --parent=t-abc123 task list --type=epic task list --parent=t-abc123 Completed task: t-8WR5Zg Amp-Thread-ID: https://ampcode.com/threads/T-85f4ee29-a529-4f59-ac6f-6ffec75b6a56 Co-authored-by: Amp <amp@ampcode.com>
8 daysUse Omni.Namespace for validated namespace argumentsBen Sima
Updated namespace handling to use the Omni.Namespace module: - Import Omni.Namespace in Task CLI - Parse and validate namespace strings using fromHaskellModule - Convert to proper path format using toPath - Ensures namespaces are well-formed (e.g., Omni/Task -> Omni/Task.hs) Example: task create "Fix bug" project --namespace="Omni/Task" # Creates task with validated namespace Omni/Task.hs This provides type safety and ensures all task namespaces correspond to actual code namespaces in the monorepo. Amp-Thread-ID: https://ampcode.com/threads/T-85f4ee29-a529-4f59-ac6f-6ffec75b6a56 Co-authored-by: Amp <amp@ampcode.com>
8 daysAdd optional namespace field to tasksBen Sima
Tasks can now be associated with specific namespaces in the monorepo: - Added taskNamespace (Maybe Text) field to Task data type - Updated createTask to accept optional namespace parameter - Added --namespace CLI option to task create command - Display namespace in task list output (e.g., [Omni/Task]) - Updated tests to pass Nothing for namespace - Updated AGENTS.md documentation Example usage: task create "Fix bug" project --namespace="Omni/Task" Completed task: t-j0k1L2 Amp-Thread-ID: https://ampcode.com/threads/T-85f4ee29-a529-4f59-ac6f-6ffec75b6a56 Co-authored-by: Amp <amp@ampcode.com>
8 daysAdd task manager for AI agentsBen Sima
Implemented a dependency-aware task tracker inspired by beads: - Task CRUD operations (create, list, update, ready) - Dependency tracking and ready work detection - JSONL storage with git sync via hooks - Export/import for cross-machine synchronization - Short base62-encoded task IDs (e.g., t-1ky7gJ2) Added comprehensive AGENTS.md documentation: - Task manager usage and workflows - Development tools (bild, lint, repl.sh) - Git-branchless workflow guidelines - Coding conventions Integrated with git hooks for auto-sync: - post-merge/post-checkout: import tasks - pre-commit/pre-push: export tasks Also includes beads design analysis document for reference. Completed tasks: - t-a1b2c3: Show help text when invoked without args - t-d4e5f6: Move dev instructions to AGENTS.md - t-g7h8i9: Implement shorter task IDs - t-p6q7r8: Add git-branchless workflow docs https: //ampcode.com/threads/T-85f4ee29-a529-4f59-ac6f-6ffec75b6a56 Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-85f4ee29-a529-4f59-ac6f-6ffec75b6a56
2025-09-05Strip comment lines and diff in commit-msg hookBen Sima
Without this, my custom git commit template will get mangled into the commit message with the call to fmt. I guess git strips all of this stuff *after* it runs the commit-msg hook. Oy vey. diff --git a/Omni/Ide/hooks/commit-msg b/Omni/Ide/hooks/commit-msg index c15b4a1..bfbb06f 100755 --- a/Omni/Ide/hooks/commit-msg +++ b/Omni/Ide/hooks/commit-msg @@ -1,6 +1,7 @@ #!/usr/bin/env bash temp=$(mktemp) -fmt -w 72 -u "$1" > "$temp" +# strip comment lines and everything after >8 cut line +sed '/^#/d; /^# -\+ >8 -\+/,$d' "$1" | fmt -w 72 -u > "$temp" mv "$temp" "$1" if ! gitlint --ignore-stdin --staged --msg-filename "$1" run-hook; then backup="$CODEROOT"/.git/COMMIT_EDITMSG.backup
2025-09-05Enhance worker memory managementBen Sima
Check to prevent processing of large articles, truncate oversized content, defer jobs during high memory usage, use streaming TTS generation and upload to minimize memory consumption.
2025-09-05Fix fmt spacingBen Sima
The -s and -u flags tell fmt to fill the paragraph by merging lines. Without this, fmt might just add line breaks, and this is no good.
2025-09-04Refactor Admin and Database path stuffBen Sima
Moved the Admin related stuff to a separate file. Removed the repetitive `db_path` arg everywhere and replaced it with correct assumptions, similar to whats in other apps.
2025-09-04Disable dbus connection attempt during deploymentBen Sima
When I deploy, I get an error like this: Error: Failed to open dbus connection Caused by: Failed to connect to socket /run/user/1000/bus: Connection refused According to Claude, this is because dbus is trying to do stuff while I'm running the `systemd-run` command, but that loses my user context so dbus errors out, and I can disable dbus by setting this variable to nothing. I guess we'll see if it works!
2025-09-04Wrap all commit messages with fmtBen Sima
Sometimes aider will write commit messages without wrapping them at 80 chars, and then the commit fails the gitlint hook, and aider can't finish the commit. To fix this I can just auto-wrap before we check it.