diff options
| author | Ben Sima <ben@bsima.me> | 2025-11-14 20:54:16 -0500 |
|---|---|---|
| committer | Ben Sima <ben@bsima.me> | 2025-11-14 20:54:16 -0500 |
| commit | 8baf00d3f7e87b24d735f57fd489ab7210f1b7a9 (patch) | |
| tree | 325cbba4b41b52a6002bc53f47985d52b653c21e /Omni/Bild.hs | |
| parent | 05e9c79a6a7b8f13835398342b6f2d26417da946 (diff) | |
Implement per-module Nix derivations for incremental Haskell builds
This is the core architecture transformation from Phase 3 of the
performance plan. Each Haskell module is now built as a separate
Nix derivation, enabling true incremental builds where only changed
modules and their dependents are rebuilt.
Implementation: - buildHsModuleGraph: Analyzes transitive module
dependencies and builds DAG - TH detection: Falls back to monolithic
build if Template Haskell detected - SCC cycle detection: Falls
back if import cycles found - Per-module Nix builder: Each module ->
separate derivation with .hi and .o - Module dependencies: Copy .hi
files to build dir, use -i flags for imports - Final link: Use ghc
--make with entry point source + -i paths to .hi files - Entry point
fix: Explicitly analyze entry point module separately from deps
Architecture: - Module compilation: ghc -c with -i paths to dependency
.hi files - Source filtering: Each module derivation includes only
its source file - Dependency DAG: Expressed as recursive Nix attrset
with lib.fix - Link phase: ghc --make with entry source file + all .hi
search paths - Fallback: Monolithic ghc --make when hsGraph is null
(TH/cycles)
Performance characteristics: - Change one module -> rebuild only
that + dependents + relink - Nix handles DAG scheduling and caching
automatically - Parallel module compilation (Nix orchestrates) -
Content-addressed caching across machines
Testing: - Added test_buildHsModuleGraph unit test - Verified with
Omni/Bild/Example.hs (4 modules) - Tested incremental rebuild triggers
correct subset
This completes Phase 2 and Phase 3 core optimizations from the plan.
Diffstat (limited to 'Omni/Bild.hs')
| -rw-r--r-- | Omni/Bild.hs | 37 |
1 files changed, 29 insertions, 8 deletions
diff --git a/Omni/Bild.hs b/Omni/Bild.hs index e8c2f09..078aac1 100644 --- a/Omni/Bild.hs +++ b/Omni/Bild.hs @@ -178,7 +178,8 @@ main = Cli.Plan help move test_ pure |> Cli.main test_bildExamples, test_isGitIgnored, test_isGitHook, - test_detectPythonImports + test_detectPythonImports, + test_buildHsModuleGraph ] test_bildBild :: Test.Tree @@ -645,7 +646,7 @@ analyzeAll nss = do |> Meta.detectAll "--" |> \Meta.Parsed {..} -> detectHaskellImports mempty contentLines +> \(langdeps, srcs) -> do - graph <- buildHsModuleGraph namespace srcs + graph <- buildHsModuleGraph namespace quapath srcs pure <| Just Target @@ -921,6 +922,23 @@ test_detectPythonImports = Set.fromList ["Omni/Log.py"] @=? set ] +test_buildHsModuleGraph :: Test.Tree +test_buildHsModuleGraph = + Test.group + "buildHsModuleGraph" + [ Test.unit "includes entry point in graph" <| do + let ns = Namespace ["Omni", "Bild", "Example"] Namespace.Hs + let entryPoint = "Omni/Bild/Example.hs" + let deps = Set.fromList ["Alpha.hs", "Omni/Test.hs"] + + result <- buildHsModuleGraph ns entryPoint deps + case result of + Nothing -> Test.assertFailure "buildHsModuleGraph returned Nothing" + Just graph -> do + let modules = Map.keys (graphModules graph) + Text.pack "Omni.Bild.Example" `elem` modules @=? True + ] + type GhcPkgCacheMem = Map String (Set String) type GhcPkgCacheDisk = Map String [String] @@ -1013,12 +1031,15 @@ ghcPkgFindModule acc m = /> Set.union acc -- | Build module graph for Haskell targets, returns Nothing if TH or cycles detected -buildHsModuleGraph :: Namespace -> Set FilePath -> IO (Maybe HsModuleGraph) -buildHsModuleGraph namespace srcs = do +buildHsModuleGraph :: Namespace -> FilePath -> Set FilePath -> IO (Maybe HsModuleGraph) +buildHsModuleGraph namespace entryPoint deps = do root <- Env.getEnv "CODEROOT" - nodes <- foldM (analyzeModule root) Map.empty (Set.toList srcs) - let hasTH = any nodeHasTH (Map.elems nodes) - let hasCycles = detectCycles nodes + -- Analyze all dependencies first + depNodes <- foldM (analyzeModule root) Map.empty (Set.toList deps) + -- Then analyze the entry point itself + allNodes <- analyzeModule root depNodes entryPoint + let hasTH = any nodeHasTH (Map.elems allNodes) + let hasCycles = detectCycles allNodes if hasTH || hasCycles then pure Nothing else @@ -1026,7 +1047,7 @@ buildHsModuleGraph namespace srcs = do <| Just HsModuleGraph { graphEntry = Namespace.toHaskellModule namespace |> Text.pack, - graphModules = nodes + graphModules = allNodes } where analyzeModule :: FilePath -> Map ModuleName HsModuleNode -> FilePath -> IO (Map ModuleName HsModuleNode) |
