| Age | Commit message (Collapse) | Author |
|---|
|
The bild --plan command now outputs progress indicators before JSON.
Updated run.sh to use 'tail -1' to extract only the JSON line and
fixed the file path check to use the full path.
Amp-Thread-ID:
https://ampcode.com/threads/T-cc5d29f0-454e-4864-8d7e-1ad69a42afa9
Co-authored-by: Amp <amp@ampcode.com>
|
|
Problem: Still getting segfaults ("free(): invalid pointer") even
with ncurses calls protected. The mutex only covered ANSI calls, but
IORef reads and IO.hPutStr operations were happening outside the lock.
Root cause: Race conditions in concurrent terminal output: 1. Multiple
threads reading namespaceLines IORef concurrently 2. Interleaved
IO.hPutStr calls corrupting output buffer 3. Rainbow.hPutChunks not
being thread-safe
Solution: Expand terminalLock scope to cover entire output operations:
- Move IORef reads inside the lock - Protect all IO.hPutStr and
Rainbow.hPutChunks calls - Lock both SingleLine and MultiLine modes -
Ensure atomicity from IORef read through all I/O to flush
This makes each updateLine/updateLineState call atomic, preventing
any interleaving of terminal operations between threads.
Changes: - updateLine SingleLine: wrap entire output in withMVar
terminalLock - updateLine MultiLine: move nsMap read inside lock -
updateLineState SingleLine: wrap entire output in withMVar terminalLock
- updateLineState MultiLine: move nsMap read inside lock
Tested: 10/10 successful runs of `bild --time 0 **/*` without
segfaults.
|
|
Clarify that the mutex protects against ncurses thread-safety issues
that cause segfaults during concurrent builds.
This commit also forces a new Nix derivation hash to ensure the
mutex fix is actually used (previous builds were cached with the
old version).
|
|
Problem: Intermittent segfaults when running `bild --time 0 **/*` with
many concurrent builds. Core dumps showed crashes in libncursesw's
free() function during terminal cleanup.
Root cause: ANSI.getTerminalSize and other ANSI terminal library
calls are not thread-safe. With mapConcurrentlyBounded running up to
8 analyses concurrently, multiple threads were calling ANSI functions
simultaneously, causing memory corruption in the ncurses library.
Solution: Add global MVar terminalLock to serialize all ANSI terminal
operations. Wrap all ANSI function calls (cursor movement, line
clearing, etc.) with withMVar terminalLock.
Changes: - Add terminalLock :: MVar () in Omni/Log/Concurrent.hs -
Wrap all ANSI calls in withMVar terminalLock:
- initializeLines: cursor column, clear line - updateLine: cursor
up/down, column set, clear line - updateLineState: cursor up/down,
column set, clear line - withLineManager: cursor up
Tested: 5 consecutive runs of `bild --time 0 **/*` complete without
segfaults (previously failed 1-2 out of 3 runs).
|
|
Problem: Calling bild.run inside NixOS configs triggered IFD during
OS evaluation. ANSI escape codes from bild broke JSON parsing in Nix
sandbox, causing build failures.
Root cause: bild.run uses IFD (Import From Derivation) which runs
bild --plan during Nix evaluation. When this happened inside NixOS
service definitions, it ran recursively and bild output ANSI codes
that corrupted the JSON analysis output.
Solution: Two-phase architecture + NO_COLOR support
1. Biz/Packages.nix: Pre-builds all packages outside NixOS context
2. Biz.nix: Accepts packages as function argument (default:
Packages.nix) 3. Omni/Bild.nix: Sets NO_COLOR=1 in analysis
derivation 4. Omni/Log/Terminal.hs: Respects NO_COLOR env var
5. Omni/Log/Terminal.hs: Skip getTerminalSize when NO_COLOR set
to avoid
escape code output
6. Omni/Log/Concurrent.hs: Skip line initialization without ANSI
support
Now NixOS builds succeed: - Package IFD happens once at top level -
No recursive builds during service evaluation - Clean JSON output
from bild --plan in Nix sandbox - NixOS configs reference pre-analyzed
packages
Changes: - Add Biz/Packages.nix - standalone package builder - Update
Biz.nix to accept packages argument - Update Biz/Dragons/Analysis.nix
to use Packages.nix - Remove Biz/Targets.nix (replaced by
Packages.nix) - Add NO_COLOR support throughout logging stack -
Fix ANSI.getTerminalSize outputting escape codes when NO_COLOR set
Amp-Thread-ID:
https://ampcode.com/threads/T-bc0f6fc7-46bf-4aa2-892e-dd62e7251d4b
Co-authored-by: Amp <amp@ampcode.com>
|
|
ANSI.getTerminalSize queries the terminal by writing escape codes and
reading the response from stdin. When stdin is unavailable or at EOF
(common in non-interactive contexts), this causes "hWaitForInput:
end of file" errors.
Additionally, Conduit.streamingProcess returns a stdin handle that
was never being closed, which could cause child processes to block
waiting for input.
Fixes: - Catch IOException in detectTerminal when getTerminalSize
fails - Close subprocess stdin handle immediately after spawn -
Fallback to default terminal size (80x24) when detection fails
This unblocks all bild commands that were failing with stdin errors.
|
|
- Remove BILD_OUTPUT_MODE environment variable - Simplify to 2
modes: MultiLine (≥80 cols) and SingleLine (<80) - Use existing
--loud flag to disable concurrent UI entirely - When --loud: show
all compiler output line-by-line (for debugging) - When not --loud:
adaptive concurrent UI based on terminal width
This is simpler and uses the existing --loud flag instead of adding
new configuration. The --loud flag was already meant for debugging,
so it makes sense to use it to show all output.
Note: Omni/Bild.nix updated to include new Omni/Log/Terminal.hs module.
Blocked by: stdin bug (see _/llm/STDIN_BUG.md) - cannot test build yet.
|
|
- Add Omni/Log/Terminal module to detect terminal capabilities -
Implement 3 output modes: RichMultiLine (≥80 cols), SingleLine
(40-79),
SimpleFallback (<40 or dumb terminals)
- Terminal width auto-detection via System.Console.ANSI.getTerminalSize
- Text truncation with ellipsis to prevent line wrapping - Manual
override via BILD_OUTPUT_MODE env var (simple|single|rich|auto)
Fixes UI corruption on small terminals (mobile SSH, narrow windows).
Wide terminals keep existing multi-line behavior but with truncation.
Design doc: _/llm/CONCURRENT_LOG_DESIGN.md
|
|
When building per-module derivations, copying dependency .hi files
was failing silently because: 1. First dep copies read-only Alpha.hi
from nix store 2. Second dep also has Alpha.hi (transitive dep) and
tries to overwrite 3. cp fails with permission denied but error is
hidden by '|| true'
Fixed by: - Add -f flag to cp to force overwrite - Run chmod -R +w
after each dependency copy - This allows later deps to overwrite
shared files like Alpha.hi
Tested: Biz/Que/Host.hs now builds successfully with per-module
derivations
|
|
Set dontPatchShebangs=true for per-module link derivation to avoid
unbound variable error in Nix's patch-shebangs.sh script.
The script declares 'local update' but never initializes it, causing
bash set -u to fail. Our ELF binaries don't need shebangs patched
anyway.
Tested: All Haskell targets now build successfully including
Omni/Lint.hs which uses makeWrapper with rundeps.
|
|
- Fix nixBuild to propagate exit code from realise step
- Previously used >> which discarded exit code - Now checks realise
result before running symlink - Fixes false success checkmarks on
build failures
- Fix per-module Nix derivations
- Use cp -rL instead of tar (src is directory not tarball) - Add
coreutils and findutils to pkgs - Copy deps to . and use -i. for
GHC - Use find with --parents to preserve module hierarchy - Set
dontStrip=true to avoid fixup script errors
- Tested: Example.hs, Task.hs, Dragons.hs, Bild.hs all build - Known
issue: makeWrapper fixup scripts have unbound vars (Nix bug)
|
|
When using custom builder, unpackPhase function is not available.
Use tar xzf to manually extract the source archive.
|
|
When using custom builder, standard phases don't run automatically.
Call unpackPhase explicitly and cd to 'source' directory where files
are unpacked.
Fixes 'sourceRoot: unbound variable' error in module compilation.
|
|
The haskell.ghcWith package has setup hooks that override buildPhase
even when explicitly set. Solution: use custom builder = stdenv.shell
with args instead of relying on mkDerivation phases.
Changes: - Module derivations: Use custom builder with single -c script
- Combines unpack, build, and install into one script - Explicitly
call ghc with full path to avoid hook interference - Remove unused
objectPaths computation (now using ghc --make with source)
This fixes builds for Omni/Lint.hs, Omni/Task.hs, and all other
Haskell targets with complex dependency graphs.
|
|
GHC --make was treating -i /nix/store/.../hidir paths as compilation targets.
Solution: Copy all .hi files to local directory and use -i. only.
Also make hsGraph optional with 'or null' for backward compatibility with
old bild binaries during bootstrap.
Amp-Thread-ID: https://ampcode.com/threads/T-fe68faaf-1c1d-4c43-a377-1cf5e6cffb3a
Co-authored-by: Amp <amp@ampcode.com>
|
|
GHC --make interprets arguments before flags as targets. Moving the
entry point source file to the beginning prevents -i paths from being
treated as compilation targets.
This fixes the 'is not a module name or a source file' error.
|
|
This is the core architecture transformation from Phase 3 of the
performance plan. Each Haskell module is now built as a separate
Nix derivation, enabling true incremental builds where only changed
modules and their dependents are rebuilt.
Implementation: - buildHsModuleGraph: Analyzes transitive module
dependencies and builds DAG - TH detection: Falls back to monolithic
build if Template Haskell detected - SCC cycle detection: Falls
back if import cycles found - Per-module Nix builder: Each module ->
separate derivation with .hi and .o - Module dependencies: Copy .hi
files to build dir, use -i flags for imports - Final link: Use ghc
--make with entry point source + -i paths to .hi files - Entry point
fix: Explicitly analyze entry point module separately from deps
Architecture: - Module compilation: ghc -c with -i paths to dependency
.hi files - Source filtering: Each module derivation includes only
its source file - Dependency DAG: Expressed as recursive Nix attrset
with lib.fix - Link phase: ghc --make with entry source file + all .hi
search paths - Fallback: Monolithic ghc --make when hsGraph is null
(TH/cycles)
Performance characteristics: - Change one module -> rebuild only
that + dependents + relink - Nix handles DAG scheduling and caching
automatically - Parallel module compilation (Nix orchestrates) -
Content-addressed caching across machines
Testing: - Added test_buildHsModuleGraph unit test - Verified with
Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers
correct subset
This completes Phase 2 and Phase 3 core optimizations from the plan.
|
|
Implements Steps 1-4 of per-module Nix architecture: - Add
HsModuleGraph and HsModuleNode types to represent module DAG -
Implement buildHsModuleGraph that:
- Parses imports for each module in the transitive closure -
Detects Template Haskell usage (language pragma or $( patterns) -
Detects import cycles using SCC analysis - Returns Nothing (fallback
to monolithic) if TH or cycles found
- Wire hsGraph into Target type for all builders - Populate hsGraph
for Haskell targets during analysis
Module graph is now emitted in --plan output showing per-module
dependencies. This enables the next step: per-module Nix derivations
in Builder.nix for true incremental builds.
Part of Phase 3 from the performance plan.
|
|
Implements optimization #3 from Phase 2 of the performance plan.
Changes: - Cache stored in _/var/ghc-pkg-cache-<hash>.json - Hash
based on GHC version + GHC_PACKAGE_PATH for automatic invalidation -
Loads cache at startup, saves on successful completion - Uses atomic
write (tmp + rename) to prevent corruption - Gracefully handles
missing/corrupt cache files - Accumulates cache entries across builds -
Works with parallel builds (in-memory IORef + disk persistence)
Performance impact: - Eliminates redundant ghc-pkg invocations across
runs - Near-zero ghc-pkg overhead once cache is populated - No impact
on single-run performance (still uses in-memory IORef)
|
|
Only include directories that are ancestors of source files in
allSources. Previously accepted all directories, causing rebuilds
when any new directory was added to the repo.
Implementation: - Precompute normalized source paths and their ancestor
directories - Filter directories against allowedDirs whitelist -
Normalize paths in file filter for consistency - Keep existing skip
list behavior for _ and .direnv
This is optimization #2 from Phase 2 of the performance plan.
|
|
- Add Analyzing state to BuildState enum - Refactor from sequential
foldM analyze to concurrent analyzeAll - Initialize all lines with
[+] during analysis phase - Update to [...] (Pending) after each
analysis completes - Uses mapConcurrentlyBounded with concurrency of
8 for analysis - Remove Log.info from analyzeOne (now handled by line
state) - Analysis now runs in parallel, improving efficiency - Flow:
[+] analyzing → [...] pending → [~] building → [✓]/[x] complete
|
|
- Replace cursor down movement with single newline at end - Cursor
now stays at bottom of status lines - No extra blank space between
build output and next prompt
|
|
- Remove save/restore cursor in favor of explicit movement - Always
move up from bottom position, update line, move back down - Simplify
initializeLines to just print lines sequentially - Cursor now stays
at bottom and moves up/down for each update - Fixes issue where lines
were being updated far up the terminal
|
|
- Use hCursorDown in initializeLines instead of hCursorUp - Simplify
updateLine and updateLineState cursor movement - Cursor now moves up
from bottom position correctly - Shell prompt no longer gets erased
|
|
- Allocate one line per namespace (not per concurrent job) - Add
Pending state shown as [...] when build hasn't started - Initialize all
namespace lines at start showing [...] - Update to [~] when building,
[✓]/[x] when complete - Each namespace keeps its line throughout
the build - At end, all namespaces show their final status - --jobs
controls concurrency, not line count
|
|
- Remove [+] display from reserveLine (was causing mangled output
from concurrent writes) - Add [+] display in analyze function where
it's single-threaded - Simplify cleanup to just move cursor down
without clearing lines - This eliminates race conditions and terminal
clearing issues
|
|
- Add initial newline to preserve terminal prompt - Clear each line
individually at end instead of just moving cursor - This prevents
extra blank lines from remaining on screen
|
|
- Remove initial blank line before build starts - Remove trailing blank
line after build completes - Remove newlines from status indicators
in releaseLine - Status lines now stay on their reserved lines without
extra spacing
|
|
- Remove all label prefixes from build logs (bild:, nix:, etc) -
Show [+] Namespace when first reserving a line (analyzing) - Show [~]
Namespace: output when building with subprocess output - Show [✓]
Namespace (green) on success - Show [x] Namespace (red) on failure
|
|
- Remove Log.info from analyze function - Remove Log.info from build
target compilers (Guile, NixBuild, Rustc, Sbcl) - Remove Log.good
from proc and nixBuild success handlers - Remove 'info: bild:' prefix
from streaming subprocess output - Output now shows only [✓]/[x]/[~]
status lines with subprocess stdout
|
|
- Add initial line break to preserve terminal prompt - Change format
to [✓]/[x]/[~] Namespace: output - Add colors: green for success,
red for failure - Fix extra lines by adding newlines in releaseLine -
Fix currentLine initialization to 0 instead of maxLines
Amp-Thread-ID:
https://ampcode.com/threads/T-39671965-c412-4a2e-8084-9d09128fd865
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Add ansi-terminal to ghcPackageSetBild dependencies - Add
Omni/Log/Concurrent.hs to source fileset - Fix unused mLineNum warning
in buildTarget - Alphabetize deps list (hostname moved up)
Parallel builds now fully functional with proper Nix dependencies.
|
|
- Fix BuildStatus to use newtype with single field - Clean up unused
imports and fields - logs function now uses updateCurrentLine
for LineManager support - Fallback to single-line output when no
LineManager
All tests passing. Ready to test with different terminal types.
Tasks: t-1a1Eiay
|
|
- Add global IORef for currentLineManager and namespaceLines
mapping - Update logs function to use LogC.updateCurrentLine -
Add updateCurrentLine and releaseCurrentLine helpers - Fallback to
normal printing when no LineManager active - Simplify buildTarget to
use global helpers instead of threading
Tasks: t-1a1EaJy
|
|
- Reserve line for each concurrent build - Release line on completion
with success/failed state - Fix hlint eta reduce warning
Task: t-1a1E3j1
|
|
- Implement LineManager abstraction with IORef state - Line
reservation/update/release functions - ANSI cursor positioning for
concurrent updates - Terminal capability detection (ANSI vs dumb) -
Graceful fallback for non-ANSI terminals
Tasks: t-1a1DzES, t-1a1DGY0, t-1a1DOev, t-1a1DVM5
|
|
- Add mapConcurrentlyBounded using QSemN for bounded parallelism -
Refactor build function to extract buildTarget worker - Build up to
--jobs targets concurrently - Preserves all existing functionality -
Output will be interleaved (will fix with LineManager next)
Related tasks: - t-1a0OVBs: mapConcurrentlyBounded helper - t-1a16ame:
refactor build function - t-1a1DdSB: replace forM with concurrent map
|
|
Replaced the old slow depth-first search with a breadth-first search
for detecting imports. This should be way faster when building a
single namespace because it doesn't have to visit the same file
multiple times.
The ghc-pkg caching means we only have to run ghc-pkg once per bild
invocation.
|
|
detectPythonImports now recursively analyzes imported modules to find
transitive dependencies, matching the behavior of detectHaskellImports.
Previously it only detected direct imports, which caused build failures
when Python modules had nested dependencies.
- Changed signature from [Text] -> IO (Set FilePath)
to Analysis -> [Text] -> IO (Set FilePath)
- Added filepaths, findDeps, and onlyPython helper functions -
Recursively calls analyze() on imported modules to find transitive
deps - Updated tests to pass empty Analysis map
|
|
I think the calls to Log.setup() were accidentally creating multiple
loggers, hopefully this fixes the problem.
|
|
- Remove UNIQUE constraints from ALTER TABLE statements
(SQLite doesn't support adding constraints via ALTER TABLE)
- Add better logging for migration failures (debug level) - Rely on
application logic to ensure uniqueness instead of DB constraint -
This fixes the silent failure where stripe_customer_id and
stripe_subscription_id columns weren't being added
|
|
- Implement Biz.PodcastItLater.Billing with checkout sessions, billing
portal, webhook handling - Add subscription database schema: plan_tier,
stripe fields, period dates, stripe_events table - Three-tier pricing:
free (10/month), personal (/month, 50 articles), pro (9/month,
unlimited) - Usage tracking and enforcement with tier-based limits
- Full billing UI with plan display, usage stats, pricing cards,
upgrade buttons - Dashboard shows current tier with billing button -
Update Web.nix with Stripe environment variables - Fix POST redirects
to Stripe with 303 status code for CloudFront compatibility
Amp-Thread-ID:
https://ampcode.com/threads/T-c139e5b5-1901-4cd6-8030-5623bfe1df35
Co-authored-by: Amp <amp@ampcode.com>
|
|
Without this, run.sh would continue if bild failed, and you might
end up running an out of date artifact.
|
|
- Add stripe to Omni/Bild/Deps/Python.nix (alphabetically sorted) -
Fix all type annotations in Billing.py for mypy - Document how to add
Python packages in AGENTS.md - Add billing routes to Web.py (checkout,
portal, webhook)
This enables Stripe integration in PodcastItLater.
Related to task t-144e7lF
|
|
Changed epic display in tree view from:
t-PpXWsU [Epic] [ ] Task Manager Improvements
To:
t-PpXWsU [6/11] Task Manager Improvements
The [6/11] shows completed/total child tasks, giving immediate visual
feedback on epic progress. Regular tasks still use checkbox indicators:
[ ] open, [~] in-progress, [✓] done.
|
|
Changed tree output format from:
t-abc123 [ ] Task title [Omni/Task.hs]
To:
t-abc123 [ ] [Omni/Task.hs] Task title
This makes the namespace more prominent and groups all metadata
(status + namespace) together before the title.
|
|
Changes: 1. Remove [Task] label - only show [Epic] for epics, cleaner
output 2. Truncate long titles to fit 80 columns with ... ellipsis
3. Better spacing with type label included in layout calculation
Created task for future improvement: prettier box-drawing characters
(├──, └──) which would require Data.Tree library
investigation.
Current output is clean and readable within standard terminal width.
|
|
Add 'task tree' command to show hierarchical task structure:
Usage:
task tree # Show all epics with their children task tree <id> #
Show specific epic/task with children
Features: - Visual status indicators: [ ] open, [~] in-progress,
[✓] done - Shows task type: [Epic] or [Task] - Indented display
for hierarchy - Shows namespace associations
Example output:
t-PpXWsU [Epic] [ ] Task Manager Improvements [Omni/Task.hs]
t-PpYZt2 [Task] [ ] Implement child ID generation t-PpZGVf [Task]
[✓] Add filtering by type and parent
Updated AGENTS.md with usage examples.
Closes task t-PpZlbL
|
|
Implemented proper pre-commit hook that: - Calls 'task export --flush'
to consolidate tasks - Auto-stages .tasks/tasks.jsonl if modified -
Runs before every commit
Added reminder message after 'task update' to inform users that task
changes will be committed on next git commit.
Updated AGENTS.md to document the auto-commit behavior.
This fixes the bug where task status updates (e.g., marking tasks as
Done) were not being committed to git.
|
|
Implement --status and --namespace filters for task list:
New filters: - --status: Filter by open, in-progress, or done -
--namespace: Filter by namespace (e.g., Omni/Task)
All filters can be combined: - task list --parent=t-abc123
--status=open - task list --type=epic --status=done - task list
--namespace="Omni/Task" --status=open
Updated listTasks signature to accept all filter parameters and apply
them in sequence. Updated AGENTS.md with examples.
Closes task t-PpZGVf
|
