path: root/AGENTS.md

# Omni

The Omni project is to leverage automation and asymmetries to create wealth. The
target of the wealth is Bitcoin. The means: write everything down, first
in English, then in code.

This document describes how AI agents should interact with this repo, the "omnirepo".

## Important Rules for AI Agents

**CRITICAL**: This project uses `task` for ALL issue tracking. You MUST follow these rules:

- ✅ Use `task` for ALL task/TODO tracking
- ✅ Always use `--json` flag for programmatic operations
- ✅ Link discovered work with `--discovered-from=<parent-id>`
- ✅ File bugs IMMEDIATELY when you discover unexpected behavior
- ✅ Run `task ready` before asking "what should I work on?"
- ✅ Store AI planning docs in `_/llm` directory (NEVER in repo root)
- ✅ Run `task sync` at end of session to commit changes locally
- ❌ Do NOT use `todo_write` tool
- ❌ Do NOT create markdown TODO lists or task checklists
- ❌ Do NOT put TODO/FIXME comments in code
- ❌ Do NOT use external issue trackers
- ❌ Do NOT duplicate tracking systems
- ❌ Do NOT clutter repo root with planning documents

### Session Checklist

**First time in this repo?**
```bash
task init --quiet  # Non-interactive initialization
```

**Standard workflow:**
```bash
# 1. Find ready work
task ready --json

# 2. Claim a task
task update <id> in-progress --json

# 3. During work: create discovered issues
task create "Fix type error found" --discovered-from=<current-id> --json

# 4. Complete the task
task update <id> done --json

# 5. End of session: sync to git (local commit only)
task sync
```

### Bug Discovery Pattern

**When you discover a bug or unexpected behavior:**
```bash
# CORRECT: Immediately file a task
task create "Command X fails when Y" --discovered-from=<current-task-id> --json

# WRONG: Ignoring it and moving on
# WRONG: Leaving a TODO comment
# WRONG: Mentioning it but not filing a task
```

**Examples of bugs you MUST file:**
- "Expected `--flag value` to work but only `--flag=value` works"
- "Documentation says X but actual behavior is Y"
- "Combining two flags causes parsing error"
- "Feature is missing that would be useful"

### Forbidden Patterns

**Markdown checklist (NEVER do this):**
```markdown
❌ Wrong:
- [ ] Refactor auth module
- [ ] Add tests
- [ ] Update docs

✅ Correct:
task create "Refactor auth module" -p 2 --json
task create "Add tests for auth" -p 2 --json
task create "Update auth docs" -p 3 --json
```

**todo_write tool (NEVER do this):**
```
❌ Wrong: todo_write({todos: [{content: "Fix bug", ...}]})
✅ Correct: task create "Fix bug in parser" -p 1 --json
```

**Inline code comments (NEVER do this):**
```python
❌ Wrong:
# TODO: write tests for this function
# FIXME: handle edge case

✅ Correct:
# Create task instead:
task create "Write tests for parse_config" -p 2 --namespace="Omni/Config" --json
task create "Handle edge case in parser" -p 1 --discovered-from=<current-id> --json
```

## About Omnirepo

Resources defined in the repo can be used to quickly create and release
products. New technology shall be prototyped and developed as needed.

### Source Layout

The source tree maps to the module namespace, and roughly follows the Haskell
namespace hierarchy. This is true of all languages: Python, Scheme, Rust, C,
etc.

Namespaces are formatted either as file paths, like `Omni/Dev`, or
dot-separated, like `Omni.Dev`. Parts of the namespace should always be
capitalized.

The namespace for all products that we own is `Biz`, this includes proprietary
applications, products, and related infrastructure.

The `Omni` namespace is used for internal development tooling and infrastructure
that are shared between all other projects.

Stuff that can be open sourced or otherwise externalized should be outside of
`Biz` or `Omni`.

Related code should be kept close together. This means that you should start
with small namespaces: use `Omni/Thing.hs` before `Omni/Thing/Service.hs`. Try
to keep all related code in one spot for as long as possible.

Re-use code from the `Omni/` namespace as much as possible. For example, use
`Omni/Cli.hs` or `Omni/Test.py` instead of trying to roll your own code for cli
parsing or running test suites. If the the namespace doesn't have the feature
you need, then add the feature.

Boundaries and interfaces between namespaces should be singular and
well-defined. Likewise, the functionality and purpose of a particular
namespace should be singular and well-defined. Follow the unix principle
of "do one thing and do it well."

Namespaces are always capitalized. In Scheme and Python this actually translates
quite well and helps distinguish between types/classes/modules and values.

## Task Manager for AI Agents

The task manager is a dependency-aware issue tracker inspired by beads. It uses:
- **Storage**: Local JSONL file (`.tasks/tasks.jsonl`)
- **Sync**: Git-tracked (automatically synced across machines)
- **Dependencies**: Tasks can block other tasks
- **Ready work detection**: Automatically finds unblocked tasks

**IMPORTANT**: You MUST use `task` for ALL issue tracking. NEVER use markdown TODOs, todo_write, task lists, or any other tracking methods.

### Human Setup vs Agent Usage

**If you see "database not found" or similar errors:**
```bash
task init --quiet  # Non-interactive, auto-setup, no prompts
```

**Why `--quiet`?** The regular `task init` may have interactive prompts. The `--quiet` flag makes it fully non-interactive and safe for agent-driven setup.

**If `task init --quiet` fails:** Ask the human to run `task init` manually, then continue.

### Create a Task
```bash
task create "<title>" [--type=<type>] [--parent=<id>] [--deps=<ids>] [--dep-type=<type>] [--discovered-from=<id>] [--namespace=<ns>]
```

Examples:
```bash
# Create an epic (container for tasks)
task create "User Authentication System" --type=epic

# Create a task within an epic
task create "Design auth API" --parent=t-abc123

# Create a task with blocking dependency
task create "Write tests" --deps=t-a1b2c3 --dep-type=blocks

# Create work discovered during implementation (shortcut)
task create "Fix memory leak" --discovered-from=t-abc123

# Create related work (doesn't block)
task create "Update documentation" --deps=t-abc123 --dep-type=related

# Associate with a namespace
task create "Fix type errors" --namespace="Omni/Task"
```

**Task Types:**
- `epic` - Container for related tasks
- `task` - Individual work item (default)

**Dependency Types:**
- `blocks` - Hard dependency, blocks ready work queue (default)
- `discovered-from` - Work discovered during other work, doesn't block
- `parent-child` - Epic/subtask relationship, blocks ready work
- `related` - Soft relationship, doesn't block

The `--namespace` option associates the task with a specific namespace in the monorepo (e.g., `Omni/Task`, `Biz/Cloud`). This helps organize tasks by the code they relate to.

### List Tasks
```bash
task list [options]  # Flags can be in any order
```

Examples:
```bash
task list                              # All tasks
task list --type=epic                  # All epics
task list --parent=t-abc123            # All tasks in an epic
task list --status=open                # All open tasks
task list --status=done                # All completed tasks
task list --namespace="Omni/Task"      # All tasks for a namespace
task list --parent=t-abc123 --status=open  # Combine filters: open tasks in epic
```

### Get Ready Work
```bash
task ready
```

Shows all tasks that are:
- Not closed
- Not blocked by incomplete dependencies

### Update Task Status
```bash
task update <id> <status>
```

Status values: `open`, `in-progress`, `done`

Examples:
```bash
task update t-20241108120000 in-progress
task update t-20241108120000 done
```

**Note**: Task updates modify `.tasks/tasks.jsonl` but don't auto-commit. The pre-commit hook will automatically export and stage task changes on your next `git commit`.

### View Dependencies
```bash
task deps <id>
```

Shows the dependency tree for a task.

### View Task Tree
```bash
task tree [<id>]
```

Shows task hierarchy with visual status indicators:
- `[ ]` - Open
- `[~]` - In Progress
- `[✓]` - Done

Examples:
```bash
task tree                 # Show all epics with their children
task tree t-abc123        # Show specific epic/task with its children
```

### Export Tasks
```bash
task export [--flush]
```

Consolidates and exports tasks to `.tasks/tasks.jsonl`, removing duplicates. The `--flush` flag forces immediate export (used by git hooks).

### Import Tasks
```bash
task import -i <file>
```

Imports tasks from a JSONL file, merging with existing tasks. Newer tasks (based on `updatedAt` timestamp) take precedence.

Examples:
```bash
task import -i .tasks/tasks.jsonl
task import -i /path/to/backup.jsonl
```

### Initialize (First Time)
```bash
task init --quiet  # Non-interactive (recommended for agents)
# OR
task init          # Interactive (for humans)
```

Creates `.tasks/` directory and `tasks.jsonl` file.

**Agents MUST use `--quiet` flag** to avoid interactive prompts.

### Common Workflows

#### Starting New Work

1. **Find what's ready to work on:**
   ```bash
   task ready
   ```

2. **Pick a task and mark it in progress:**
   ```bash
   task update t-20241108120000 in-progress
   ```

3. **When done, mark it complete:**
   ```bash
   task update t-20241108120000 done
   ```

#### Creating Dependent Tasks

When you discover work that depends on other work:

```bash
# Create the blocking task first
task create "Design API" --type=task

# Note the ID (e.g., t-20241108120000)

# Create dependent task with blocking dependency
task create "Implement API client" --deps=t-20241108120000 --dep-type=blocks
```

The dependent task won't show up in `task ready` until the blocker is marked `done`.

#### Discovered Work Pattern

When you find work during implementation, use the `--discovered-from` flag:

```bash
# While working on t-abc123, you discover a bug
task create "Fix memory leak in parser" --discovered-from=t-abc123

# This is equivalent to:
task create "Fix memory leak in parser" --deps=t-abc123 --dep-type=discovered-from
```

The `discovered-from` dependency type maintains context but **doesn't block** the ready work queue. This allows AI agents to track what work was found during other work while still being able to work on it immediately.

#### Working with Epics

```bash
# Create an epic for a larger feature
task create "User Authentication System" --type=epic
# Note ID: t-abc123

# Create child tasks within the epic
task create "Design login flow" --parent=t-abc123
task create "Implement OAuth" --parent=t-abc123
task create "Add password reset" --parent=t-abc123

# List all tasks in an epic
task list --parent=t-abc123

# List all epics
task list --type=epic
```

### Agent Best Practices

#### 1. ALWAYS Check Ready Work First
Before asking what to do, you MUST check `task ready --json` to see unblocked tasks.

#### 2. ALWAYS Create Tasks for Discovered Work
When you encounter work during implementation, you MUST create linked tasks:
```bash
task create "Fix type error in auth module" --discovered-from=t-abc123 --json
task create "Add missing test coverage" --discovered-from=t-abc123 --json
```

**CRITICAL: File bugs immediately when you discover them:**
- If a command doesn't work as documented → create a task
- If a command doesn't work as you expected → create a task
- If behavior is inconsistent or confusing → create a task
- If documentation is wrong or misleading → create a task
- If you find yourself working around a limitation → create a task

**NEVER leave TODO comments in code.** Create a task instead.

**NEVER ignore bugs or unexpected behavior.** File a task for it immediately.

#### 3. Track Dependencies
If work depends on other work, use `--deps`:
```bash
# Can't write tests until implementation is done
task create "Test auth flow" --deps=t-20241108120000 --dep-type=blocks --json
```

#### 4. Use Descriptive Titles
Good: `"Add JWT token validation to auth middleware"`
Bad: `"Fix auth"`

#### 5. Use Epics for Organization
Organize related work using epics:
- Create an epic for larger features: `task create "Feature Name" --type=epic --json`
- Add tasks to the epic using `--parent=<epic-id>`
- Use `--discovered-from` to track work found during implementation

#### 6. ALWAYS Store AI Planning Docs in `_/llm` Directory
AI assistants often create planning and design documents during development:
- PLAN.md, DESIGN.md, TESTING_GUIDE.md, tmp, and similar files
- **You MUST use a dedicated directory for these ephemeral files**
- Store ALL AI-generated planning/design docs in `_/llm`
- The `_` directory is ignored by git and all of our temporary files related to the omnirepo go there
- NEVER commit planning docs to the repo root

### Dependency Rules

- A task is **blocked** if any of its dependencies are not `done`
- A task is **ready** if all its dependencies are `done` (or it has no dependencies)
- `task ready` only shows tasks with status `open` or `in-progress` that are not blocked

### File Structure

```
.tasks/
├── tasks.jsonl       # Git-tracked, production database
├── tasks-test.jsonl  # Test database (not tracked, auto-created)

Omni/Ide/hooks/
├── pre-commit        # Exports tasks before commit (auto-stages tasks.jsonl)
├── post-checkout     # Imports tasks after branch switch
└── ...               # Other git hooks
```

Each line in `tasks.jsonl` is a JSON object representing a task.

**Git Hooks**: This repository uses hooks from `Omni/Ide/hooks/` (configured via `core.hooksPath`). Do NOT add hooks to `.git/hooks/` - they won't be version controlled and may cause confusion.

### Testing and Development

**CRITICAL**: When manually testing task functionality (like tree visualization, flag ordering, etc.), you MUST use the test database:

```bash
# Set test mode to protect production database
export TASK_TEST_MODE=1

# Now all task operations use .tasks/tasks-test.jsonl
task create "Test task" --type=task
task list
task tree

# Unset when done
unset TASK_TEST_MODE
```

**The test suite automatically uses test mode** - you don't need to set it manually when running `task test` or `bild --test Omni/Task.hs`.

**NEVER run manual tests against the production database** (`.tasks/tasks.jsonl`). This pollutes it with test data that must be manually cleaned up. Always use `TASK_TEST_MODE=1` for experimentation.

## Integration with Git

The `.tasks/tasks.jsonl` file is git-tracked. When you:
- Create/update tasks locally
- Commit and push
- Other machines/agents get the updates on `git pull`

**Important**: Add to `.gitignore`:
```
.tasks/*.db
.tasks/*.db-journal
.tasks/*.sock
```

But **do** track:
```
!.tasks/
!.tasks/tasks.jsonl
```

### Troubleshooting

#### "Task not found"
- Check the task ID is correct with `task list`
- Ensure you've run `task init`

#### "Database not initialized"
Run: `task init`

#### Dependencies not working
- Verify dependency IDs exist: `task list`
- Check dependency tree: `task deps <id>`

### Example Session

```bash
# First time setup
task init

# Create an epic for the work
task create "Task Manager Improvements" --type=epic
# Returns: t-abc123

# Create tasks within the epic
task create "Design task manager schema" --parent=t-abc123
task create "Implement JSONL storage" --parent=t-abc123
task create "Add dependency tracking" --parent=t-abc123

# See what's ready (all of them, no blockers yet)
task ready

# Start working
task update t-20241108120000 in-progress

# Discover work during implementation
task create "Fix edge case in ID generation" --discovered-from=t-20241108120000

# Discover dependent work with blocking
task create "Write storage tests" --deps=t-20241108120000 --dep-type=blocks

# Complete first task
task update t-20241108120000 done

# Now the test task is unblocked (discovered work was already unblocked)
task ready
# Shows: "Write storage tests" and "Fix edge case in ID generation"
```

### Reinforcement: Critical Rules

Remember these non-negotiable rules:

- ✅ Use `task` for ALL task tracking (with `--json` flag)
- ✅ Link discovered work with `--discovered-from` dependencies
- ✅ File bugs IMMEDIATELY when you discover unexpected behavior
- ✅ Check `task ready --json` before asking "what should I work on?"
- ✅ Store AI planning docs in `_/llm` directory
- ✅ Run `task sync` at end of every session (commits locally, does NOT push)
- ❌ NEVER use `todo_write` tool
- ❌ NEVER create markdown TODO lists or task checklists
- ❌ NEVER put TODOs or FIXMEs in code comments
- ❌ NEVER use external issue trackers
- ❌ NEVER duplicate tracking systems
- ❌ NEVER clutter repo root with planning documents

**If you find yourself about to use todo_write or create a markdown checklist, STOP and use `task create` instead.**

## Development Guide and Tools

### bild

`bild` is the universal build tool. It can build and test everything in the repo.

Examples:
```bash
bild --test Omni/Bild.hs         # Build and test a namespace
bild --time 0 Omni/Cloud.nix     # Build with no timeout
bild --plan Omni/Test.hs         # Analyze build without building
```

When the executable is built, the output will go to `_/bin`. Example:

```bash
# build the example executable
bild Omni/Bild/Example.py
# run the executable
_/bin/example
```

### run.sh

`run.sh` is a convenience wrapper that builds (if needed) and runs a namespace.

Examples:
```bash
Omni/Ide/run.sh Omni/Task.hs     # Build and run task manager
Omni/Ide/run.sh Biz/PodcastItLater/Web.py  # Build and run web server
```

This script will:
1. Check if the binary exists in `_/bin/`
2. Build it if it doesn't exist (exits on build failure)
3. Execute the binary with any additional arguments

### lint

Universal lint and formatting tool. Errors if lints fail or code is not formatted properly.

Examples:
```bash
lint Omni/Cli.hs                 # Lint a namespace
lint --fix **/*.py               # Lint and fix all Python files
```

### repl.sh

Like `nix-shell` but specific to this repo. Analyzes the namespace, pulls dependencies, and starts a shell or repl.

Examples:
```bash
repl.sh Omni/Bild.hs            # Start Haskell repl with namespace loaded
repl.sh --bash Omni/Log.py      # Start bash shell for namespace
```

### typecheck.sh

Like `lint` but only runs type checkers. Currently just supports Python with `mypy`, but eventually will support everything that `bild` supports.

Examples:
```bash
typecheck.sh Omni/Bild/Example.py  # Run the typechecker and report any errors
```

### Test Commands

Run tests:
```bash
bild --test Omni/Task.hs         # Build and test a namespace
```

The convention for all programs in the omnirepo is to run their tests if the first argument is `test`. So for example:

```bash
# this will build a the latest executable and then run tests
bild --test Omni/Task.hs

# this will just run the tests from the existing executable
_/bin/task test
```

## Adding New Dependencies

### Python Packages

To add a new Python package as a dependency:

1. Add the package name to `Omni/Bild/Deps/Python.nix` (alphabetically sorted)
2. Use it in your Python file with `# : dep <package-name>` comment at the top
3. Run `bild <yourfile.py>` to build with the new dependency

Example:
```python
# : out myapp
# : dep stripe
# : dep pytest
import stripe
```

The package name must match the nixpkgs python package name (usually the PyPI name).
Check available packages: `nix-env -qaP -A nixpkgs.python3Packages | grep <name>`

## Coding Conventions

1. **Test interface**: Every program must accept `test` as a first argument to run its test suite
2. **Entrypoint naming**: The entrypoint for every program shall be called `main`
3. **Always include tests**: Every new feature and bug fix must include tests. No code should be committed without corresponding test coverage
4. **No TODO/FIXME comments**: Instead of leaving TODO or FIXME comments in code, create a task with `task create` to track the work properly
5. **Fast typechecking**: Use `Omni/Ide/typecheck.sh <file>` for quick Python typechecking instead of `bild --test` when you only need to check types

## Git Workflow

### Use git-branchless

This repository uses **git-branchless** for a patch-based workflow instead of traditional branch-based git.

Key concepts:
- Work with **patches** (commits) directly rather than branches
- Use **stacking** to organize related changes
- Leverage **smartlog** to visualize commit history

### Common git-branchless Commands

**View commit graph:**
```bash
git smartlog
```

**Create a new commit:**
```bash
# Make your changes
git add .
git commit -m "Your commit message"
```

**Amend the current commit:**
```bash
# Make additional changes
git add .
git amend
```

**Move/restack commits:**
```bash
git move -s <source> -d <destination>
git restack
```

### When to Record Changes in Git

**DO record in git:**
- Completed features or bug fixes
- Working code that passes tests and linting
- Significant milestones in task completion

**DO NOT record in git:**
- Work in progress (unless specifically requested)
- Broken or untested code
- Temporary debugging changes

**NEVER do these git operations without explicit user request:**
- ❌ `git push` - NEVER push to remote unless explicitly asked
- ❌ `git pull` - NEVER pull from remote unless explicitly asked
- ❌ Force pushes or destructive operations
- ❌ Branch deletion or remote branch operations

**Why:** The user maintains control over when code is shared with collaborators. Always ask before syncing with remote repositories.

### Workflow Best Practices

1. **Make small, focused commits** - Each commit should represent one logical change
2. **Write descriptive commit messages** - Explain what and why, not just what
3. **Rebase and clean up history** - Use `git commit --amend` and `git restack` to keep history clean
4. **Test before committing** - Run `bild --test` and `lint` on affected namespaces

### Required Checks Before Completing Tasks

After completing a task, **always** run these commands for the namespace(s) you modified:

```bash
# Run tests
bild --test Omni/YourNamespace.hs

# Run linter
lint Omni/YourNamespace.hs
```

**Fix all reported errors** related to your changes before marking the task as complete. This ensures code quality and prevents breaking the build for other contributors.