| Age | Commit message (Collapse) | Author |
|---|
|
Clarify that the mutex protects against ncurses thread-safety issues
that cause segfaults during concurrent builds.
This commit also forces a new Nix derivation hash to ensure the
mutex fix is actually used (previous builds were cached with the
old version).
|
|
Problem: Intermittent segfaults when running `bild --time 0 **/*` with
many concurrent builds. Core dumps showed crashes in libncursesw's
free() function during terminal cleanup.
Root cause: ANSI.getTerminalSize and other ANSI terminal library
calls are not thread-safe. With mapConcurrentlyBounded running up to
8 analyses concurrently, multiple threads were calling ANSI functions
simultaneously, causing memory corruption in the ncurses library.
Solution: Add global MVar terminalLock to serialize all ANSI terminal
operations. Wrap all ANSI function calls (cursor movement, line
clearing, etc.) with withMVar terminalLock.
Changes: - Add terminalLock :: MVar () in Omni/Log/Concurrent.hs -
Wrap all ANSI calls in withMVar terminalLock:
- initializeLines: cursor column, clear line - updateLine: cursor
up/down, column set, clear line - updateLineState: cursor up/down,
column set, clear line - withLineManager: cursor up
Tested: 5 consecutive runs of `bild --time 0 **/*` complete without
segfaults (previously failed 1-2 out of 3 runs).
|
|
Problem: Calling bild.run inside NixOS configs triggered IFD during
OS evaluation. ANSI escape codes from bild broke JSON parsing in Nix
sandbox, causing build failures.
Root cause: bild.run uses IFD (Import From Derivation) which runs
bild --plan during Nix evaluation. When this happened inside NixOS
service definitions, it ran recursively and bild output ANSI codes
that corrupted the JSON analysis output.
Solution: Two-phase architecture + NO_COLOR support
1. Biz/Packages.nix: Pre-builds all packages outside NixOS context
2. Biz.nix: Accepts packages as function argument (default:
Packages.nix) 3. Omni/Bild.nix: Sets NO_COLOR=1 in analysis
derivation 4. Omni/Log/Terminal.hs: Respects NO_COLOR env var
5. Omni/Log/Terminal.hs: Skip getTerminalSize when NO_COLOR set
to avoid
escape code output
6. Omni/Log/Concurrent.hs: Skip line initialization without ANSI
support
Now NixOS builds succeed: - Package IFD happens once at top level -
No recursive builds during service evaluation - Clean JSON output
from bild --plan in Nix sandbox - NixOS configs reference pre-analyzed
packages
Changes: - Add Biz/Packages.nix - standalone package builder - Update
Biz.nix to accept packages argument - Update Biz/Dragons/Analysis.nix
to use Packages.nix - Remove Biz/Targets.nix (replaced by
Packages.nix) - Add NO_COLOR support throughout logging stack -
Fix ANSI.getTerminalSize outputting escape codes when NO_COLOR set
Amp-Thread-ID:
https://ampcode.com/threads/T-bc0f6fc7-46bf-4aa2-892e-dd62e7251d4b
Co-authored-by: Amp <amp@ampcode.com>
|
|
Problem: Calling bild.run inside NixOS configs triggers recursive
builds during OS image creation, causing slow IFD evaluations that
worsen as complexity grows.
Solution: Create Biz/Targets.nix that pre-declares all buildable
targets as an attribute set. NixOS configs now import and reference
these targets directly, eliminating recursive builds during evaluation.
Changes: - Add Biz/Targets.nix exposing storybook, podcastitlater-web,
podcastitlater-worker, dragons-analysis
- Update Biz.nix to import targets and reference them - Update
Biz/Dragons/Analysis.nix to use targets pattern
Benefits: - All bild.run calls happen once at top level during targets
evaluation - NixOS service configs reference pre-built derivations -
Scalable: adding targets doesn't slow individual builds - Explicit:
clear what gets built for each OS
Amp-Thread-ID:
https://ampcode.com/threads/T-bc0f6fc7-46bf-4aa2-892e-dd62e7251d4b
Co-authored-by: Amp <amp@ampcode.com>
|
|
ANSI.getTerminalSize queries the terminal by writing escape codes and
reading the response from stdin. When stdin is unavailable or at EOF
(common in non-interactive contexts), this causes "hWaitForInput:
end of file" errors.
Additionally, Conduit.streamingProcess returns a stdin handle that
was never being closed, which could cause child processes to block
waiting for input.
Fixes: - Catch IOException in detectTerminal when getTerminalSize
fails - Close subprocess stdin handle immediately after spawn -
Fallback to default terminal size (80x24) when detection fails
This unblocks all bild commands that were failing with stdin errors.
|
|
- Remove BILD_OUTPUT_MODE environment variable - Simplify to 2
modes: MultiLine (≥80 cols) and SingleLine (<80) - Use existing
--loud flag to disable concurrent UI entirely - When --loud: show
all compiler output line-by-line (for debugging) - When not --loud:
adaptive concurrent UI based on terminal width
This is simpler and uses the existing --loud flag instead of adding
new configuration. The --loud flag was already meant for debugging,
so it makes sense to use it to show all output.
Note: Omni/Bild.nix updated to include new Omni/Log/Terminal.hs module.
Blocked by: stdin bug (see _/llm/STDIN_BUG.md) - cannot test build yet.
|
|
- Add Omni/Log/Terminal module to detect terminal capabilities -
Implement 3 output modes: RichMultiLine (≥80 cols), SingleLine
(40-79),
SimpleFallback (<40 or dumb terminals)
- Terminal width auto-detection via System.Console.ANSI.getTerminalSize
- Text truncation with ellipsis to prevent line wrapping - Manual
override via BILD_OUTPUT_MODE env var (simple|single|rich|auto)
Fixes UI corruption on small terminals (mobile SSH, narrow windows).
Wide terminals keep existing multi-line behavior but with truncation.
Design doc: _/llm/CONCURRENT_LOG_DESIGN.md
|
|
When building per-module derivations, copying dependency .hi files
was failing silently because: 1. First dep copies read-only Alpha.hi
from nix store 2. Second dep also has Alpha.hi (transitive dep) and
tries to overwrite 3. cp fails with permission denied but error is
hidden by '|| true'
Fixed by: - Add -f flag to cp to force overwrite - Run chmod -R +w
after each dependency copy - This allows later deps to overwrite
shared files like Alpha.hi
Tested: Biz/Que/Host.hs now builds successfully with per-module
derivations
|
|
Set dontPatchShebangs=true for per-module link derivation to avoid
unbound variable error in Nix's patch-shebangs.sh script.
The script declares 'local update' but never initializes it, causing
bash set -u to fail. Our ELF binaries don't need shebangs patched
anyway.
Tested: All Haskell targets now build successfully including
Omni/Lint.hs which uses makeWrapper with rundeps.
|
|
- Fix nixBuild to propagate exit code from realise step
- Previously used >> which discarded exit code - Now checks realise
result before running symlink - Fixes false success checkmarks on
build failures
- Fix per-module Nix derivations
- Use cp -rL instead of tar (src is directory not tarball) - Add
coreutils and findutils to pkgs - Copy deps to . and use -i. for
GHC - Use find with --parents to preserve module hierarchy - Set
dontStrip=true to avoid fixup script errors
- Tested: Example.hs, Task.hs, Dragons.hs, Bild.hs all build - Known
issue: makeWrapper fixup scripts have unbound vars (Nix bug)
|
|
When using custom builder, unpackPhase function is not available.
Use tar xzf to manually extract the source archive.
|
|
When using custom builder, standard phases don't run automatically.
Call unpackPhase explicitly and cd to 'source' directory where files
are unpacked.
Fixes 'sourceRoot: unbound variable' error in module compilation.
|
|
The haskell.ghcWith package has setup hooks that override buildPhase
even when explicitly set. Solution: use custom builder = stdenv.shell
with args instead of relying on mkDerivation phases.
Changes: - Module derivations: Use custom builder with single -c script
- Combines unpack, build, and install into one script - Explicitly
call ghc with full path to avoid hook interference - Remove unused
objectPaths computation (now using ghc --make with source)
This fixes builds for Omni/Lint.hs, Omni/Task.hs, and all other
Haskell targets with complex dependency graphs.
|
|
GHC --make was treating -i /nix/store/.../hidir paths as compilation targets.
Solution: Copy all .hi files to local directory and use -i. only.
Also make hsGraph optional with 'or null' for backward compatibility with
old bild binaries during bootstrap.
Amp-Thread-ID: https://ampcode.com/threads/T-fe68faaf-1c1d-4c43-a377-1cf5e6cffb3a
Co-authored-by: Amp <amp@ampcode.com>
|
|
GHC --make interprets arguments before flags as targets. Moving the
entry point source file to the beginning prevents -i paths from being
treated as compilation targets.
This fixes the 'is not a module name or a source file' error.
|
|
This is the core architecture transformation from Phase 3 of the
performance plan. Each Haskell module is now built as a separate
Nix derivation, enabling true incremental builds where only changed
modules and their dependents are rebuilt.
Implementation: - buildHsModuleGraph: Analyzes transitive module
dependencies and builds DAG - TH detection: Falls back to monolithic
build if Template Haskell detected - SCC cycle detection: Falls
back if import cycles found - Per-module Nix builder: Each module ->
separate derivation with .hi and .o - Module dependencies: Copy .hi
files to build dir, use -i flags for imports - Final link: Use ghc
--make with entry point source + -i paths to .hi files - Entry point
fix: Explicitly analyze entry point module separately from deps
Architecture: - Module compilation: ghc -c with -i paths to dependency
.hi files - Source filtering: Each module derivation includes only
its source file - Dependency DAG: Expressed as recursive Nix attrset
with lib.fix - Link phase: ghc --make with entry source file + all .hi
search paths - Fallback: Monolithic ghc --make when hsGraph is null
(TH/cycles)
Performance characteristics: - Change one module -> rebuild only
that + dependents + relink - Nix handles DAG scheduling and caching
automatically - Parallel module compilation (Nix orchestrates) -
Content-addressed caching across machines
Testing: - Added test_buildHsModuleGraph unit test - Verified with
Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers
correct subset
This completes Phase 2 and Phase 3 core optimizations from the plan.
|
|
Implements Steps 1-4 of per-module Nix architecture: - Add
HsModuleGraph and HsModuleNode types to represent module DAG -
Implement buildHsModuleGraph that:
- Parses imports for each module in the transitive closure -
Detects Template Haskell usage (language pragma or $( patterns) -
Detects import cycles using SCC analysis - Returns Nothing (fallback
to monolithic) if TH or cycles found
- Wire hsGraph into Target type for all builders - Populate hsGraph
for Haskell targets during analysis
Module graph is now emitted in --plan output showing per-module
dependencies. This enables the next step: per-module Nix derivations
in Builder.nix for true incremental builds.
Part of Phase 3 from the performance plan.
|
|
Implements optimization #3 from Phase 2 of the performance plan.
Changes: - Cache stored in _/var/ghc-pkg-cache-<hash>.json - Hash
based on GHC version + GHC_PACKAGE_PATH for automatic invalidation -
Loads cache at startup, saves on successful completion - Uses atomic
write (tmp + rename) to prevent corruption - Gracefully handles
missing/corrupt cache files - Accumulates cache entries across builds -
Works with parallel builds (in-memory IORef + disk persistence)
Performance impact: - Eliminates redundant ghc-pkg invocations across
runs - Near-zero ghc-pkg overhead once cache is populated - No impact
on single-run performance (still uses in-memory IORef)
|
|
Only include directories that are ancestors of source files in
allSources. Previously accepted all directories, causing rebuilds
when any new directory was added to the repo.
Implementation: - Precompute normalized source paths and their ancestor
directories - Filter directories against allowedDirs whitelist -
Normalize paths in file filter for consistency - Keep existing skip
list behavior for _ and .direnv
This is optimization #2 from Phase 2 of the performance plan.
|
|
|
|
- Add Analyzing state to BuildState enum - Refactor from sequential
foldM analyze to concurrent analyzeAll - Initialize all lines with
[+] during analysis phase - Update to [...] (Pending) after each
analysis completes - Uses mapConcurrentlyBounded with concurrency of
8 for analysis - Remove Log.info from analyzeOne (now handled by line
state) - Analysis now runs in parallel, improving efficiency - Flow:
[+] analyzing → [...] pending → [~] building → [✓]/[x] complete
|
|
- Replace cursor down movement with single newline at end - Cursor
now stays at bottom of status lines - No extra blank space between
build output and next prompt
|
|
- Remove save/restore cursor in favor of explicit movement - Always
move up from bottom position, update line, move back down - Simplify
initializeLines to just print lines sequentially - Cursor now stays
at bottom and moves up/down for each update - Fixes issue where lines
were being updated far up the terminal
|
|
- Use hCursorDown in initializeLines instead of hCursorUp - Simplify
updateLine and updateLineState cursor movement - Cursor now moves up
from bottom position correctly - Shell prompt no longer gets erased
|
|
- Allocate one line per namespace (not per concurrent job) - Add
Pending state shown as [...] when build hasn't started - Initialize all
namespace lines at start showing [...] - Update to [~] when building,
[✓]/[x] when complete - Each namespace keeps its line throughout
the build - At end, all namespaces show their final status - --jobs
controls concurrency, not line count
|
|
- Remove [+] display from reserveLine (was causing mangled output
from concurrent writes) - Add [+] display in analyze function where
it's single-threaded - Simplify cleanup to just move cursor down
without clearing lines - This eliminates race conditions and terminal
clearing issues
|
|
- Add initial newline to preserve terminal prompt - Clear each line
individually at end instead of just moving cursor - This prevents
extra blank lines from remaining on screen
|
|
- Remove initial blank line before build starts - Remove trailing blank
line after build completes - Remove newlines from status indicators
in releaseLine - Status lines now stay on their reserved lines without
extra spacing
|
|
- Remove all label prefixes from build logs (bild:, nix:, etc) -
Show [+] Namespace when first reserving a line (analyzing) - Show [~]
Namespace: output when building with subprocess output - Show [✓]
Namespace (green) on success - Show [x] Namespace (red) on failure
|
|
- Remove Log.info from analyze function - Remove Log.info from build
target compilers (Guile, NixBuild, Rustc, Sbcl) - Remove Log.good
from proc and nixBuild success handlers - Remove 'info: bild:' prefix
from streaming subprocess output - Output now shows only [✓]/[x]/[~]
status lines with subprocess stdout
|
|
- Add initial line break to preserve terminal prompt - Change format
to [✓]/[x]/[~] Namespace: output - Add colors: green for success,
red for failure - Fix extra lines by adding newlines in releaseLine -
Fix currentLine initialization to 0 instead of maxLines
Amp-Thread-ID:
https://ampcode.com/threads/T-39671965-c412-4a2e-8084-9d09128fd865
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Add ansi-terminal to ghcPackageSetBild dependencies - Add
Omni/Log/Concurrent.hs to source fileset - Fix unused mLineNum warning
in buildTarget - Alphabetize deps list (hostname moved up)
Parallel builds now fully functional with proper Nix dependencies.
|
|
Updated PARALLEL_BUILDS_PLAN.md with: - Implementation status (all
phases complete) - Global state approach details - Testing results -
Future enhancement ideas
Task: t-1a1ELGl, Epic: t-19ZF6A8 (complete)
|
|
- Fix BuildStatus to use newtype with single field - Clean up unused
imports and fields - logs function now uses updateCurrentLine
for LineManager support - Fallback to single-line output when no
LineManager
All tests passing. Ready to test with different terminal types.
Tasks: t-1a1Eiay
|
|
- Add global IORef for currentLineManager and namespaceLines
mapping - Update logs function to use LogC.updateCurrentLine -
Add updateCurrentLine and releaseCurrentLine helpers - Fallback to
normal printing when no LineManager active - Simplify buildTarget to
use global helpers instead of threading
Tasks: t-1a1EaJy
|
|
- Reserve line for each concurrent build - Release line on completion
with success/failed state - Fix hlint eta reduce warning
Task: t-1a1E3j1
|
|
- Implement LineManager abstraction with IORef state - Line
reservation/update/release functions - ANSI cursor positioning for
concurrent updates - Terminal capability detection (ANSI vs dumb) -
Graceful fallback for non-ANSI terminals
Tasks: t-1a1DzES, t-1a1DGY0, t-1a1DOev, t-1a1DVM5
|
|
- Add mapConcurrentlyBounded using QSemN for bounded parallelism -
Refactor build function to extract buildTarget worker - Build up to
--jobs targets concurrently - Preserves all existing functionality -
Output will be interleaved (will fix with LineManager next)
Related tasks: - t-1a0OVBs: mapConcurrentlyBounded helper - t-1a16ame:
refactor build function - t-1a1DdSB: replace forM with concurrent map
|
|
Replaced the old slow depth-first search with a breadth-first search
for detecting imports. This should be way faster when building a
single namespace because it doesn't have to visit the same file
multiple times.
The ghc-pkg caching means we only have to run ghc-pkg once per bild
invocation.
|
|
|
|
detectPythonImports now recursively analyzes imported modules to find
transitive dependencies, matching the behavior of detectHaskellImports.
Previously it only detected direct imports, which caused build failures
when Python modules had nested dependencies.
- Changed signature from [Text] -> IO (Set FilePath)
to Analysis -> [Text] -> IO (Set FilePath)
- Added filepaths, findDeps, and onlyPython helper functions -
Recursively calls analyze() on imported modules to find transitive
deps - Updated tests to pass empty Analysis map
|
|
- Created reusable PageLayout component in UI.py with consistent
header/navbar - Added Home link and Admin dropdown menu (Queue
Status, Manage Users) - Updated all pages to use PageLayout: home,
account, admin queue, admin users - Added demo@example.com to admin
whitelist for testing - Added dark mode styling for table headers -
Fixed component children syntax for Ludic framework - Proper type
annotations instead of type: ignore comments
|
|
Change 'Manage Subscription' from HTMX link to regular form POST.
HTMX AJAX requests can't follow redirects to external domains like
Stripe.
|
|
- Remove user-facing error messages for portal configuration -
Just log the error server-side and return 500 status - Use Bootstrap
card-header class for proper padding on section headers - This fixes
icons touching the card borders
Portal errors will now be logged but won't break the UI.
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Catch Stripe exceptions when portal not configured - Redirect to
account page with user-friendly error message - Display error alert
on account page when present - Change portal return URL to /account
instead of /
Fixes issue when Stripe billing portal settings haven't been configured
in test/production dashboard.
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Display account information (email, creation date) - Show
subscription details (plan, status, features) - Display cancellation
warning if subscription ending - Add Upgrade button for free users -
Add Manage Subscription button for paid users (goes to Stripe portal) -
Add logout button in Actions section - Replace Coming Soon placeholder
with functional UI
Addresses account management epic tasks.
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Add 'Remove' button for non-pending queue items (error, processing,
cancelled) - Keep 'Cancel' button for pending items - Update
delete_queue_item to check referer and return appropriate response -
Use HX-Trigger for queue updates from user page, HX-Redirect for admin
- Add confirmation dialog for remove action - Allow admins to delete
any job, users can only delete their own
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Remove 'Logged in as' user email display - Convert button-styled
links to standard Bootstrap nav-link style - Left-align navigation
links (removed me-auto from wrapper) - Remove logout button from navbar
(will be in account page) - Use proper ul/li structure for navbar items
The navbar is now cleaner and follows Bootstrap navbar conventions.
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Add mx-1 (horizontal margin) to code element for better text spacing
- Pre-fill email input with demo@example.com in dev/test mode -
Makes it faster to test login flow in development
Amp-Thread-ID:
https://ampcode.com/threads/T-8edacbeb-b343-49ca-b524-1c999272acb6
Co-authored-by: Amp <amp@ampcode.com>
|
|
|
