summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
45 hoursImplement per-module Nix derivations for incremental Haskell buildsBen Sima
This is the core architecture transformation from Phase 3 of the performance plan. Each Haskell module is now built as a separate Nix derivation, enabling true incremental builds where only changed modules and their dependents are rebuilt. Implementation: - buildHsModuleGraph: Analyzes transitive module dependencies and builds DAG - TH detection: Falls back to monolithic build if Template Haskell detected - SCC cycle detection: Falls back if import cycles found - Per-module Nix builder: Each module -> separate derivation with .hi and .o - Module dependencies: Copy .hi files to build dir, use -i flags for imports - Final link: Use ghc --make with entry point source + -i paths to .hi files - Entry point fix: Explicitly analyze entry point module separately from deps Architecture: - Module compilation: ghc -c with -i paths to dependency .hi files - Source filtering: Each module derivation includes only its source file - Dependency DAG: Expressed as recursive Nix attrset with lib.fix - Link phase: ghc --make with entry source file + all .hi search paths - Fallback: Monolithic ghc --make when hsGraph is null (TH/cycles) Performance characteristics: - Change one module -> rebuild only that + dependents + relink - Nix handles DAG scheduling and caching automatically - Parallel module compilation (Nix orchestrates) - Content-addressed caching across machines Testing: - Added test_buildHsModuleGraph unit test - Verified with Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers correct subset This completes Phase 2 and Phase 3 core optimizations from the plan.
46 hoursAdd per-module Haskell build foundationBen Sima
Implements Steps 1-4 of per-module Nix architecture: - Add HsModuleGraph and HsModuleNode types to represent module DAG - Implement buildHsModuleGraph that: - Parses imports for each module in the transitive closure - Detects Template Haskell usage (language pragma or $( patterns) - Detects import cycles using SCC analysis - Returns Nothing (fallback to monolithic) if TH or cycles found - Wire hsGraph into Target type for all builders - Populate hsGraph for Haskell targets during analysis Module graph is now emitted in --plan output showing per-module dependencies. This enables the next step: per-module Nix derivations in Builder.nix for true incremental builds. Part of Phase 3 from the performance plan.
47 hoursAdd persistent ghc-pkg cache to speed up analysisBen Sima
Implements optimization #3 from Phase 2 of the performance plan. Changes: - Cache stored in _/var/ghc-pkg-cache-<hash>.json - Hash based on GHC version + GHC_PACKAGE_PATH for automatic invalidation - Loads cache at startup, saves on successful completion - Uses atomic write (tmp + rename) to prevent corruption - Gracefully handles missing/corrupt cache files - Accumulates cache entries across builds - Works with parallel builds (in-memory IORef + disk persistence) Performance impact: - Eliminates redundant ghc-pkg invocations across runs - Near-zero ghc-pkg overhead once cache is populated - No impact on single-run performance (still uses in-memory IORef)
47 hoursTighten Nix source filtering to prevent spurious rebuildsBen Sima
Only include directories that are ancestors of source files in allSources. Previously accepted all directories, causing rebuilds when any new directory was added to the repo. Implementation: - Precompute normalized source paths and their ancestor directories - Filter directories against allowedDirs whitelist - Normalize paths in file filter for consistency - Keep existing skip list behavior for _ and .direnv This is optimization #2 from Phase 2 of the performance plan.
2 daysFix untypable circular importBen Sima
2 daysImplement concurrent analysis with [+] state indicatorBen Sima
- Add Analyzing state to BuildState enum - Refactor from sequential foldM analyze to concurrent analyzeAll - Initialize all lines with [+] during analysis phase - Update to [...] (Pending) after each analysis completes - Uses mapConcurrentlyBounded with concurrency of 8 for analysis - Remove Log.info from analyzeOne (now handled by line state) - Analysis now runs in parallel, improving efficiency - Flow: [+] analyzing → [...] pending → [~] building → [✓]/[x] complete
2 daysRemove blank lines after build completesBen Sima
- Replace cursor down movement with single newline at end - Cursor now stays at bottom of status lines - No extra blank space between build output and next prompt
2 daysFix cursor movement to avoid updating wrong linesBen Sima
- Remove save/restore cursor in favor of explicit movement - Always move up from bottom position, update line, move back down - Simplify initializeLines to just print lines sequentially - Cursor now stays at bottom and moves up/down for each update - Fixes issue where lines were being updated far up the terminal
2 daysFix cursor positioning to preserve shell promptBen Sima
- Use hCursorDown in initializeLines instead of hCursorUp - Simplify updateLine and updateLineState cursor movement - Cursor now moves up from bottom position correctly - Shell prompt no longer gets erased
2 daysRedesign LineManager to show one line per namespaceBen Sima
- Allocate one line per namespace (not per concurrent job) - Add Pending state shown as [...] when build hasn't started - Initialize all namespace lines at start showing [...] - Update to [~] when building, [✓]/[x] when complete - Each namespace keeps its line throughout the build - At end, all namespaces show their final status - --jobs controls concurrency, not line count
2 daysFix concurrent terminal update issuesBen Sima
- Remove [+] display from reserveLine (was causing mangled output from concurrent writes) - Add [+] display in analyze function where it's single-threaded - Simplify cleanup to just move cursor down without clearing lines - This eliminates race conditions and terminal clearing issues
2 daysFix prompt preservation and cleanup extra lines at endBen Sima
- Add initial newline to preserve terminal prompt - Clear each line individually at end instead of just moving cursor - This prevents extra blank lines from remaining on screen
2 daysFix blank lines in parallel build outputBen Sima
- Remove initial blank line before build starts - Remove trailing blank line after build completes - Remove newlines from status indicators in releaseLine - Status lines now stay on their reserved lines without extra spacing
2 daysClean up build output and add [+] for analyzing phaseBen Sima
- Remove all label prefixes from build logs (bild:, nix:, etc) - Show [+] Namespace when first reserving a line (analyzing) - Show [~] Namespace: output when building with subprocess output - Show [✓] Namespace (green) on success - Show [x] Namespace (red) on failure
2 daysRemove info and good log messages from parallel build outputBen Sima
- Remove Log.info from analyze function - Remove Log.info from build target compilers (Guile, NixBuild, Rustc, Sbcl) - Remove Log.good from proc and nixBuild success handlers - Remove 'info: bild:' prefix from streaming subprocess output - Output now shows only [✓]/[x]/[~] status lines with subprocess stdout
2 daysFix parallel build output format with status indicators and colorsBen Sima
- Add initial line break to preserve terminal prompt - Change format to [✓]/[x]/[~] Namespace: output - Add colors: green for success, red for failure - Fix extra lines by adding newlines in releaseLine - Fix currentLine initialization to 0 instead of maxLines Amp-Thread-ID: https://ampcode.com/threads/T-39671965-c412-4a2e-8084-9d09128fd865 Co-authored-by: Amp <amp@ampcode.com>
2 daysAdd ansi-terminal dep and Omni/Log/Concurrent to Bild.nixBen Sima
- Add ansi-terminal to ghcPackageSetBild dependencies - Add Omni/Log/Concurrent.hs to source fileset - Fix unused mLineNum warning in buildTarget - Alphabetize deps list (hostname moved up) Parallel builds now fully functional with proper Nix dependencies.
2 daysDocument parallel builds implementation in planBen Sima
Updated PARALLEL_BUILDS_PLAN.md with: - Implementation status (all phases complete) - Global state approach details - Testing results - Future enhancement ideas Task: t-1a1ELGl, Epic: t-19ZF6A8 (complete)
2 daysComplete multi-line concurrent logging integrationBen Sima
- Fix BuildStatus to use newtype with single field - Clean up unused imports and fields - logs function now uses updateCurrentLine for LineManager support - Fallback to single-line output when no LineManager All tests passing. Ready to test with different terminal types. Tasks: t-1a1Eiay
2 daysIntegrate LineManager with logging systemBen Sima
- Add global IORef for currentLineManager and namespaceLines mapping - Update logs function to use LogC.updateCurrentLine - Add updateCurrentLine and releaseCurrentLine helpers - Fallback to normal printing when no LineManager active - Simplify buildTarget to use global helpers instead of threading Tasks: t-1a1EaJy
2 daysThread LineManager through build functionsBen Sima
- Reserve line for each concurrent build - Release line on completion with success/failed state - Fix hlint eta reduce warning Task: t-1a1E3j1
2 daysCreate Omni/Log/Concurrent module for multi-line outputBen Sima
- Implement LineManager abstraction with IORef state - Line reservation/update/release functions - ANSI cursor positioning for concurrent updates - Terminal capability detection (ANSI vs dumb) - Graceful fallback for non-ANSI terminals Tasks: t-1a1DzES, t-1a1DGY0, t-1a1DOev, t-1a1DVM5
2 daysAdd bounded parallel target buildsBen Sima
- Add mapConcurrentlyBounded using QSemN for bounded parallelism - Refactor build function to extract buildTarget worker - Build up to --jobs targets concurrently - Preserves all existing functionality - Output will be interleaved (will fix with LineManager next) Related tasks: - t-1a0OVBs: mapConcurrentlyBounded helper - t-1a16ame: refactor build function - t-1a1DdSB: replace forM with concurrent map
2 daysBild: breadth-first search and ghc-pkg cachingBen Sima
Replaced the old slow depth-first search with a breadth-first search for detecting imports. This should be way faster when building a single namespace because it doesn't have to visit the same file multiple times. The ghc-pkg caching means we only have to run ghc-pkg once per bild invocation.
2 daysIgnore node_modules (amp code installation)Ben Sima
3 daysFix Python import detection to handle transitive dependenciesBen Sima
detectPythonImports now recursively analyzes imported modules to find transitive dependencies, matching the behavior of detectHaskellImports. Previously it only detected direct imports, which caused build failures when Python modules had nested dependencies. - Changed signature from [Text] -> IO (Set FilePath) to Analysis -> [Text] -> IO (Set FilePath) - Added filepaths, findDeps, and onlyPython helper functions - Recursively calls analyze() on imported modules to find transitive deps - Updated tests to pass empty Analysis map
3 daysUnify navigation across PodcastItLater pagesBen Sima
- Created reusable PageLayout component in UI.py with consistent header/navbar - Added Home link and Admin dropdown menu (Queue Status, Manage Users) - Updated all pages to use PageLayout: home, account, admin queue, admin users - Added demo@example.com to admin whitelist for testing - Added dark mode styling for table headers - Fixed component children syntax for Ludic framework - Proper type annotations instead of type: ignore comments
3 daysFix CORS error for Stripe portal redirectBen Sima
Change 'Manage Subscription' from HTMX link to regular form POST. HTMX AJAX requests can't follow redirects to external domains like Stripe.
3 daysSimplify Stripe portal error handling and fix account page paddingBen Sima
- Remove user-facing error messages for portal configuration - Just log the error server-side and return 500 status - Use Bootstrap card-header class for proper padding on section headers - This fixes icons touching the card borders Portal errors will now be logged but won't break the UI. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysAdd error handling for unconfigured Stripe billing portalBen Sima
- Catch Stripe exceptions when portal not configured - Redirect to account page with user-friendly error message - Display error alert on account page when present - Change portal return URL to /account instead of / Fixes issue when Stripe billing portal settings haven't been configured in test/production dashboard. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysImplement full account management pageBen Sima
- Display account information (email, creation date) - Show subscription details (plan, status, features) - Display cancellation warning if subscription ending - Add Upgrade button for free users - Add Manage Subscription button for paid users (goes to Stripe portal) - Add logout button in Actions section - Replace Coming Soon placeholder with functional UI Addresses account management epic tasks. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysAdd remove button to queue status itemsBen Sima
- Add 'Remove' button for non-pending queue items (error, processing, cancelled) - Keep 'Cancel' button for pending items - Update delete_queue_item to check referer and return appropriate response - Use HX-Trigger for queue updates from user page, HX-Redirect for admin - Add confirmation dialog for remove action - Allow admins to delete any job, users can only delete their own Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysSimplify navbar to clean nav linksBen Sima
- Remove 'Logged in as' user email display - Convert button-styled links to standard Bootstrap nav-link style - Left-align navigation links (removed me-auto from wrapper) - Remove logout button from navbar (will be in account page) - Use proper ul/li structure for navbar items The navbar is now cleaner and follows Bootstrap navbar conventions. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysImprove dev mode login UXBen Sima
- Add mx-1 (horizontal margin) to code element for better text spacing - Pre-fill email input with demo@example.com in dev/test mode - Makes it faster to test login flow in development Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysAdd dev taskBen Sima
3 daysImprove type safety in Billing moduleBen Sima
- Add explicit type annotations to TIER_LIMITS dict - Add type annotation to tier_info dict in get_tier_info - Remove unnecessary type: ignore comments - Use .get() consistently for dict access Improves type safety and removes 3 type: ignore suppressions. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysExtract format_duration utility to UI moduleBen Sima
Moved format_duration function from Web.py to UI.py for better code organization. This is a UI utility function used for displaying episode durations, so it belongs in the shared UI module rather than the web-specific module. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysReplace bare exception catches with specific exceptionsBen Sima
- Replace Exception with httpx.HTTPError, httpx.TimeoutException, re.error in extract_og_metadata - Replace Exception with ValueError, KeyError in auth verification - Replace Exception with httpx errors and ValueError in submit_article - Replace Exception with ValueError, KeyError, AttributeError in RSS feed generation - Replace Exception with ValueError, KeyError in cancel/retry/delete job handlers Improves error handling specificity and removes BLE001 linter suppressions. Amp-Thread-ID: https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6 Co-authored-by: Amp <amp@ampcode.com>
3 daysUpdate agents and plan out next workBen Sima
3 daysSimplify billing to single paid planBen Sima
- Remove /billing page and BillingPage component - Add callout box on home page showing articles remaining for free users - Upgrade Now button goes directly to Stripe checkout - Change tier from 'pro' to 'paid' throughout - Update redirect URLs to go to / instead of /billing - Remove Billing button from navbar, add Manage Account link for all users - Add /account route with coming soon page - Hide payment banner for paid users Amp-Thread-ID: https://ampcode.com/threads/T-7de89e42-947c-4243-be19-0cb75be607e7 Co-authored-by: Amp <amp@ampcode.com>
3 daysSwitch to just one paid planBen Sima
I don't need the complexity of multiple plans, just one for now to test.
3 daysFix tests I thinkBen Sima
amp worked on this overnight, and I cleaned it up a bit this morning. I think its all correct now.
3 daysMove end to end test to own fileBen Sima
This is a big test, and probably over-dependencied, but it's nice to have an end-to-end test as an overall bug-catcher.
3 daysCleanup some logging setup codeBen Sima
I think the calls to Log.setup() were accidentally creating multiple loggers, hopefully this fixes the problem.
4 daysPodcastItLater end to end testBen Sima
4 daysDocument database migrations and external API testing patternsBen Sima
Add to AGENTS.md: - SQLite migration limitations (no UNIQUE constraints via ALTER TABLE) - Migration logging best practices - Testing external API integrations (Stripe pattern) - Mock webhook testing approach - Local webhook testing with CLI tools - Handling API version differences with fallback field names - Database inspection for debugging integrations These patterns emerged from implementing and debugging the Stripe billing integration.
4 daysUpdate webhook test to match real Stripe API behaviorBen Sima
- Use billing_cycle_anchor instead of current_period_start/end in test - Reflects actual Stripe API v2025-10-29 subscription structure - Test validates upgrade flow works with newer API versions - Removes debug logging (list of available keys)
4 daysFix subscription webhook to handle Stripe API version differencesBen Sima
- Use billing_cycle_anchor or start_date as fallback for period start - Calculate period end by adding 1 month if not provided - Handles newer Stripe API versions that don't include current_period_* fields - Fixes issue where subscription.created webhook couldn't update user plan - Works with Stripe API Version 2025-10-29.clover and newer
4 daysAdd detailed logging for missing subscription period datesBen Sima
Log available keys when period dates are missing to help debug why subscription webhooks fail to update user plan.
4 daysAdd full checkout flow test with proper client_reference_idBen Sima
- Test simulates complete Stripe checkout flow end-to-end - Step 1: checkout.session.completed with client_reference_id - Step 2: customer.subscription.created with subscription details - Verifies customer linking and plan upgrade to pro tier - Tests ensure webhook flow works without calling real Stripe API - Validates that client_reference_id properly links session to user