# Omni

The Omni project is to leverage automation and asymmetries to create wealth. The
target of the wealth is Bitcoin. The means: write everything down, first
in English, then in code.

This document describes how AI agents should interact with this repo, the "omnirepo".

## About Omnirepo

Resources defined in the repo can be used to quickly create and release
products. New technology shall be prototyped and developed as needed.

### Source Layout

The source tree maps to the module namespace, and roughly follows the Haskell
namespace hierarchy. This is true of all languages: Python, Scheme, Rust, C,
etc.

Namespaces are formatted either as file paths, like `Omni/Dev`, or
dot-separated, like `Omni.Dev`. Parts of the namespace should always be
capitalized.

The namespace for all products that we own is `Biz`, this includes proprietary
applications, products, and related infrastructure.

The `Omni` namespace is used for internal development tooling and infrastructure
that are shared between all other projects.

Stuff that can be open sourced or otherwise externalized should be outside of
`Biz` or `Omni`.

Related code should be kept close together. This means that you should start
with small namespaces: use `Omni/Thing.hs` before `Omni/Thing/Service.hs`. Try
to keep all related code in one spot for as long as possible.

Re-use code from the `Omni/` namespace as much as possible. For example, use
`Omni/Cli.hs` or `Omni/Test.py` instead of trying to roll your own code for cli
parsing or running test suites. If the the namespace doesn't have the feature
you need, then add the feature.

Boundaries and interfaces between namespaces should be singular and
well-defined. Likewise, the functionality and purpose of a particular
namespace should be singular and well-defined. Follow the unix principle
of "do one thing and do it well."

Namespaces are always capitalized. In Scheme and Python this actually translates
quite well and helps distinguish between types/classes/modules and values.

## Task Manager for AI Agents

The task manager is a dependency-aware issue tracker inspired by beads. It uses:
- **Storage**: Local JSONL file (`.tasks/tasks.jsonl`)
- **Sync**: Git-tracked (automatically synced across machines)
- **Dependencies**: Tasks can block other tasks
- **Ready work detection**: Automatically finds unblocked tasks

**IMPORTANT**: This project uses `task` for ALL issue tracking. Do NOT use markdown TODOs, task lists, or other tracking methods.

### Create a Task
```bash
task create "<title>" [--type=<type>] [--parent=<id>] [--deps=<ids>] [--dep-type=<type>] [--discovered-from=<id>] [--namespace=<ns>]
```

Examples:
```bash
# Create an epic (container for tasks)
task create "User Authentication System" --type=epic

# Create a task within an epic
task create "Design auth API" --parent=t-abc123

# Create a task with blocking dependency
task create "Write tests" --deps=t-a1b2c3 --dep-type=blocks

# Create work discovered during implementation (shortcut)
task create "Fix memory leak" --discovered-from=t-abc123

# Create related work (doesn't block)
task create "Update documentation" --deps=t-abc123 --dep-type=related

# Associate with a namespace
task create "Fix type errors" --namespace="Omni/Task"
```

**Task Types:**
- `epic` - Container for related tasks
- `task` - Individual work item (default)

**Dependency Types:**
- `blocks` - Hard dependency, blocks ready work queue (default)
- `discovered-from` - Work discovered during other work, doesn't block
- `parent-child` - Epic/subtask relationship, blocks ready work
- `related` - Soft relationship, doesn't block

The `--namespace` option associates the task with a specific namespace in the monorepo (e.g., `Omni/Task`, `Biz/Cloud`). This helps organize tasks by the code they relate to.

### List Tasks
```bash
task list [--type=<type>] [--parent=<id>] [--status=<status>] [--namespace=<ns>]
```

Examples:
```bash
task list                              # All tasks
task list --type=epic                  # All epics
task list --parent=t-abc123            # All tasks in an epic
task list --status=open                # All open tasks
task list --status=done                # All completed tasks
task list --namespace="Omni/Task"      # All tasks for a namespace
task list --parent=t-abc123 --status=open  # Combine filters: open tasks in epic
```

### Get Ready Work
```bash
task ready
```

Shows all tasks that are:
- Not closed
- Not blocked by incomplete dependencies

### Update Task Status
```bash
task update <id> <status>
```

Status values: `open`, `in-progress`, `done`

Examples:
```bash
task update t-20241108120000 in-progress
task update t-20241108120000 done
```

**Note**: Task updates modify `.tasks/tasks.jsonl` but don't auto-commit. The pre-commit hook will automatically export and stage task changes on your next `git commit`.

### View Dependencies
```bash
task deps <id>
```

Shows the dependency tree for a task.

### View Task Tree
```bash
task tree [<id>]
```

Shows task hierarchy with visual status indicators:
- `[ ]` - Open
- `[~]` - In Progress
- `[✓]` - Done

Examples:
```bash
task tree                 # Show all epics with their children
task tree t-abc123        # Show specific epic/task with its children
```

### Export Tasks
```bash
task export [--flush]
```

Consolidates and exports tasks to `.tasks/tasks.jsonl`, removing duplicates. The `--flush` flag forces immediate export (used by git hooks).

### Import Tasks
```bash
task import -i <file>
```

Imports tasks from a JSONL file, merging with existing tasks. Newer tasks (based on `updatedAt` timestamp) take precedence.

Examples:
```bash
task import -i .tasks/tasks.jsonl
task import -i /path/to/backup.jsonl
```

### Initialize (First Time)
```bash
task init
```

Creates `.tasks/` directory and `tasks.jsonl` file.

### Common Workflows

#### Starting New Work

1. **Find what's ready to work on:**
   ```bash
   task ready
   ```

2. **Pick a task and mark it in progress:**
   ```bash
   task update t-20241108120000 in-progress
   ```

3. **When done, mark it complete:**
   ```bash
   task update t-20241108120000 done
   ```

#### Creating Dependent Tasks

When you discover work that depends on other work:

```bash
# Create the blocking task first
task create "Design API" --type=task

# Note the ID (e.g., t-20241108120000)

# Create dependent task with blocking dependency
task create "Implement API client" --deps=t-20241108120000 --dep-type=blocks
```

The dependent task won't show up in `task ready` until the blocker is marked `done`.

#### Discovered Work Pattern

When you find work during implementation, use the `--discovered-from` flag:

```bash
# While working on t-abc123, you discover a bug
task create "Fix memory leak in parser" --discovered-from=t-abc123

# This is equivalent to:
task create "Fix memory leak in parser" --deps=t-abc123 --dep-type=discovered-from
```

The `discovered-from` dependency type maintains context but **doesn't block** the ready work queue. This allows AI agents to track what work was found during other work while still being able to work on it immediately.

#### Working with Epics

```bash
# Create an epic for a larger feature
task create "User Authentication System" --type=epic
# Note ID: t-abc123

# Create child tasks within the epic
task create "Design login flow" --parent=t-abc123
task create "Implement OAuth" --parent=t-abc123
task create "Add password reset" --parent=t-abc123

# List all tasks in an epic
task list --parent=t-abc123

# List all epics
task list --type=epic
```

### Agent Best Practices

#### 1. Always Check Ready Work First
Before asking what to do, check `task ready` to see unblocked tasks.

#### 2. Create Tasks for Discovered Work
When you encounter work during implementation:
```bash
task create "Fix type error in auth module" --discovered-from=t-abc123
task create "Add missing test coverage" --discovered-from=t-abc123
```

#### 3. Track Dependencies
If work depends on other work, use `--deps`:
```bash
# Can't write tests until implementation is done
task create "Test auth flow" --deps=t-20241108120000 --dep-type=blocks
```

#### 4. Use Descriptive Titles
Good: `"Add JWT token validation to auth middleware"`
Bad: `"Fix auth"`

#### 5. Use Epics for Organization
Organize related work using epics:
- Create an epic for larger features: `task create "Feature Name" --type=epic`
- Add tasks to the epic using `--parent=<epic-id>`
- Use `--discovered-from` to track work found during implementation

#### 6. Store AI planning docs in `_/llm` directory
AI assistants often create planning and design documents during development:
- PLAN.md, DESIGN.md, TESTING_GUIDE.md, tmp, and similar files
- **Best Practice: Use a dedicated directory for these ephemeral files.**
- Store ALL AI-generated planning/design docs in `_/llm`
- The `_` directory is ignored by git and all of our temporary files related to the omnirepo go there

### Dependency Rules

- A task is **blocked** if any of its dependencies are not `done`
- A task is **ready** if all its dependencies are `done` (or it has no dependencies)
- `task ready` only shows tasks with status `open` or `in-progress` that are not blocked

### File Structure

```
.tasks/
├── tasks.jsonl       # Git-tracked, production database
├── tasks-test.jsonl  # Test database (not tracked, auto-created)

Omni/Ide/hooks/
├── pre-commit        # Exports tasks before commit (auto-stages tasks.jsonl)
├── post-checkout     # Imports tasks after branch switch
└── ...               # Other git hooks
```

Each line in `tasks.jsonl` is a JSON object representing a task.

**Git Hooks**: This repository uses hooks from `Omni/Ide/hooks/` (configured via `core.hooksPath`). Do NOT add hooks to `.git/hooks/` - they won't be version controlled and may cause confusion.

### Testing and Development

**IMPORTANT**: When writing or testing code that modifies tasks, use the test database:

```bash
# Set test mode to protect production database
export TASK_TEST_MODE=1

# Now all task operations use .tasks/tasks-test.jsonl
task create "Test task" --type=task
task list

# Unset when done
unset TASK_TEST_MODE
```

**The test suite automatically uses test mode** - you don't need to set it manually when running `task test` or `bild --test Omni/Task.hs`.

**Never run destructive tests against the production database** (`.tasks/tasks.jsonl`) as this will delete real task data.

## Integration with Git

The `.tasks/tasks.jsonl` file is git-tracked. When you:
- Create/update tasks locally
- Commit and push
- Other machines/agents get the updates on `git pull`

**Important**: Add to `.gitignore`:
```
.tasks/*.db
.tasks/*.db-journal
.tasks/*.sock
```

But **do** track:
```
!.tasks/
!.tasks/tasks.jsonl
```

### Troubleshooting

#### "Task not found"
- Check the task ID is correct with `task list`
- Ensure you've run `task init`

#### "Database not initialized"
Run: `task init`

#### Dependencies not working
- Verify dependency IDs exist: `task list`
- Check dependency tree: `task deps <id>`

### Example Session

```bash
# First time setup
task init

# Create an epic for the work
task create "Task Manager Improvements" --type=epic
# Returns: t-abc123

# Create tasks within the epic
task create "Design task manager schema" --parent=t-abc123
task create "Implement JSONL storage" --parent=t-abc123
task create "Add dependency tracking" --parent=t-abc123

# See what's ready (all of them, no blockers yet)
task ready

# Start working
task update t-20241108120000 in-progress

# Discover work during implementation
task create "Fix edge case in ID generation" --discovered-from=t-20241108120000

# Discover dependent work with blocking
task create "Write storage tests" --deps=t-20241108120000 --dep-type=blocks

# Complete first task
task update t-20241108120000 done

# Now the test task is unblocked (discovered work was already unblocked)
task ready
# Shows: "Write storage tests" and "Fix edge case in ID generation"
```

### Important Rules

- Use `task` for ALL task tracking
- Link discovered work with `discovered-from` dependencies
- Check `task ready` before asking "what should I work on?"
- Store AI planning docs in `_/llm` directory
- Do NOT create markdown TODO lists
- Do NOT put TODOs or FIXMEs in code comments
- Do NOT use external issue trackers
- Do NOT duplicate tracking systems
- Do NOT clutter repo root with planning documents

## Development Guide and Tools

### bild

`bild` is the universal build tool. It can build and test everything in the repo.

Examples:
```bash
bild --test Omni/Bild.hs         # Build and test a namespace
bild --time 0 Omni/Cloud.nix     # Build with no timeout
bild --plan Omni/Test.hs         # Analyze build without building
```

When the executable is built, the output will go to `_/bin`. Example:

```bash
# build the example executable
bild Omni/Bild/Example.py
# run the executable
_/bin/example
```

### run.sh

`run.sh` is a convenience wrapper that builds (if needed) and runs a namespace.

Examples:
```bash
Omni/Ide/run.sh Omni/Task.hs     # Build and run task manager
Omni/Ide/run.sh Biz/PodcastItLater/Web.py  # Build and run web server
```

This script will:
1. Check if the binary exists in `_/bin/`
2. Build it if it doesn't exist (exits on build failure)
3. Execute the binary with any additional arguments

### lint

Universal lint and formatting tool. Errors if lints fail or code is not formatted properly.

Examples:
```bash
lint Omni/Cli.hs                 # Lint a namespace
lint --fix **/*.py               # Lint and fix all Python files
```

### repl.sh

Like `nix-shell` but specific to this repo. Analyzes the namespace, pulls dependencies, and starts a shell or repl.

Examples:
```bash
repl.sh Omni/Bild.hs            # Start Haskell repl with namespace loaded
repl.sh --bash Omni/Log.py      # Start bash shell for namespace
```

### typecheck.sh

Like `lint` but only runs type checkers. Currently just supports Python with `mypy`, but eventually will support everything that `bild` supports.

Examples:
```bash
typecheck.sh Omni/Bild/Example.py  # Run the typechecker and report any errors
```

### Test Commands

Run tests:
```bash
bild --test Omni/Task.hs         # Build and test a namespace
```

The convention for all programs in the omnirepo is to run their tests if the first argument is `test`. So for example:

```bash
# this will build a the latest executable and then run tests
bild --test Omni/Task.hs

# this will just run the tests from the existing executable
_/bin/task test
```

## Adding New Dependencies

### Python Packages

To add a new Python package as a dependency:

1. Add the package name to `Omni/Bild/Deps/Python.nix` (alphabetically sorted)
2. Use it in your Python file with `# : dep <package-name>` comment at the top
3. Run `bild <yourfile.py>` to build with the new dependency

Example:
```python
# : out myapp
# : dep stripe
# : dep pytest
import stripe
```

The package name must match the nixpkgs python package name (usually the PyPI name).
Check available packages: `nix-env -qaP -A nixpkgs.python3Packages | grep <name>`

## Coding Conventions

1. **Test interface**: Every program must accept `test` as a first argument to run its test suite
2. **Entrypoint naming**: The entrypoint for every program shall be called `main`
3. **Always include tests**: Every new feature and bug fix must include tests. No code should be committed without corresponding test coverage
4. **No TODO/FIXME comments**: Instead of leaving TODO or FIXME comments in code, create a task with `task create` to track the work properly
5. **Fast typechecking**: Use `Omni/Ide/typecheck.sh <file>` for quick Python typechecking instead of `bild --test` when you only need to check types

## Git Workflow

### Use git-branchless

This repository uses **git-branchless** for a patch-based workflow instead of traditional branch-based git.

Key concepts:
- Work with **patches** (commits) directly rather than branches
- Use **stacking** to organize related changes
- Leverage **smartlog** to visualize commit history

### Common git-branchless Commands

**View commit graph:**
```bash
git smartlog
```

**Create a new commit:**
```bash
# Make your changes
git add .
git commit -m "Your commit message"
```

**Amend the current commit:**
```bash
# Make additional changes
git add .
git amend
```

**Move/restack commits:**
```bash
git move -s <source> -d <destination>
git restack
```

### When to Record Changes in Git

**DO record in git:**
- Completed features or bug fixes
- Working code that passes tests and linting
- Significant milestones in task completion

**DO NOT record in git:**
- Work in progress (unless specifically requested)
- Broken or untested code
- Temporary debugging changes

### Workflow Best Practices

1. **Make small, focused commits** - Each commit should represent one logical change
2. **Write descriptive commit messages** - Explain what and why, not just what
3. **Rebase and clean up history** - Use `git commit --amend` and `git restack` to keep history clean
4. **Test before committing** - Run `bild --test` and `lint` on affected namespaces

### Required Checks Before Completing Tasks

After completing a task, **always** run these commands for the namespace(s) you modified:

```bash
# Run tests
bild --test Omni/YourNamespace.hs

# Run linter
lint Omni/YourNamespace.hs
```

**Fix all reported errors** related to your changes before marking the task as complete. This ensures code quality and prevents breaking the build for other contributors.