summaryrefslogtreecommitdiff
path: root/Omni
AgeCommit message (Collapse)Author
12 daysAdd Telegram button confirmation for subagent spawningBen Sima
Security improvement: subagents now require explicit user confirmation via Telegram inline buttons, preventing the agent from bypassing approval. Changes: - Add InlineKeyboardMarkup, InlineKeyboardButton, CallbackQuery types - Add parseCallbackQuery for handling button presses - Add sendMessageWithKeyboard and answerCallbackQuery API functions - Add PendingSpawn registry for tracking unconfirmed spawn requests - Add spawnSubagentToolWithApproval that sends approval buttons - Add handleCallbackQuery to process approve/reject button clicks - Add approveAndSpawnSubagent and rejectPendingSpawn functions Flow: 1. Agent calls spawn_subagent → creates pending request 2. User receives message with ✅ Approve / ❌ Reject buttons 3. Button click (outside agent loop) spawns or cancels 4. Pending requests expire after 10 minutes
12 daysMake subagents non-blocking with async spawningBen Sima
- Add global subagent registry to track running handles by ID - Modify executeSpawnSubagent to use spawnSubagentAsync and return immediately - Add check_subagent tool for querying status or getting results - Export subagentTools convenience function with both tools - Update Telegram.hs to use subagentTools instead of just spawnSubagentTool Ava can now spawn subagents in the background and continue conversations while checking on progress via check_subagent.
12 daysfeat(ava): subagent hardening with audit loggingBen Sima
Based on Anthropic's effective harnesses research. New modules: - Omni/Agent/AuditLog.hs: JSONL audit logging with SubagentId linking - Omni/Agent/Tools/AvaLogs.hs: Tool for Ava to query her own logs - Omni/Agent/Subagent/HARDENING.md: Design documentation Key features: - SubagentHandle with TVar status for async execution and polling - spawnSubagentAsync, querySubagentStatus, waitSubagent, cancelSubagent - User confirmation: spawn_subagent requires confirmed=true after approval - Audit logs stored in $AVA_DATA_ROOT/logs/{ava,subagents}/ - CLI: ava logs [--last=N] [<subagent_id>] - read_ava_logs tool for Ava self-diagnosis Tasks: t-267, t-268, t-269, t-270, t-271
12 daysava: add Python/CLI tools and local whisper fallbackBen Sima
- Wrap ava binary with Python (requests, httpx, pandas, etc.) - Add CLI tools: curl, pandoc, ffmpeg, imagemagick, csvkit - Add local whisper-cli fallback when OPENAI_API_KEY unavailable Amp-Thread-ID: https://ampcode.com/threads/T-019b2dc2-36e0-75e1-90c1-622901fc9847 Co-authored-by: Amp <amp@ampcode.com>
13 daysava: wrap binary with Python and CLI toolsBen Sima
13 daysFix Ava log buffering for journaldBen Sima
- Add LineBuffering to stdout/stderr in Ava.hs - Add StandardOutput/StandardError=journal to systemd service
13 daysMerge Ava deployment changesBen Sima
13 daysAdd Ava systemd deployment with dedicated user and workspaceBen Sima
- Add Omni.Agent.Paths module for configurable AVA_DATA_ROOT - Create ava Linux user in Users.nix with SSH key - Add systemd service in Beryllium/Ava.nix with graceful shutdown - Update Skills.hs and Outreach.hs to use configurable paths - Add startup logging of resolved paths in Telegram.hs - Create migration script for moving data from _/var/ava to /home/ava - Add deployment documentation in Beryllium/AVA.md In dev: AVA_DATA_ROOT unset uses _/var/ava/ In prod: AVA_DATA_ROOT=/home/ava via systemd Amp-Thread-ID: https://ampcode.com/threads/T-019b2d7e-bd88-7355-8133-275c65157aaf Co-authored-by: Amp <amp@ampcode.com>
13 daysAdd subagent system for AvaBen Sima
Enables orchestrator to spawn specialized subagents for focused tasks: - WebCrawler: web search + page reading (haiku, fast) - CodeReviewer: code analysis tools (sonnet, thorough) - DataExtractor: structured data extraction (haiku) - Researcher: combined web + codebase research (sonnet) Key features: - spawn_subagent tool with role-based tool selection - Per-subagent resource limits (timeout, cost, tokens) - Structured output with citations (claim, source_url, quote) - Separate API keys for OpenRouter vs Kagi - Efficiency-focused system prompts Defaults: 200k tokens, $1.00 cost cap, 600s timeout, 20 iterations
13 daysFix Manifest.hs FromJSON parsersBen Sima
Wrap parser chains in parentheses to fix compilation errors from incorrect use of </ operator.
13 daysFix deployer checking stale state instead of actual running servicesBen Sima
The deployer compared its in-memory stateServices map to decide if a service needed restarting. When the deployer restarted, this state was lost, causing it to think services were 'already at' the desired path when they were actually running old code. Changes: - Add getRunningStorePath to Systemd module to read actual store path - Update deployService to query systemd instead of stale in-memory state - Add DerivingStrategies extension to Deployer.hs
14 daysUpdate homepage with marketing copy and fix push.shBen Sima
14 daysFix hlint errors in Deploy modulesBen Sima
- Systemd: use list comprehension instead of if-then-else - Manifest: use </> operator, replace case with maybe - Deployer: use newtype, use flip removeService - Caddy: use newtype for single-field types
14 daysAdd skills system for avaBen Sima
- Create Omni/Agent/Skills.hs with skill loader and tools - Skills follow Claude Skills format (SKILL.md + scripts/references/assets) - Directory structure: _/var/ava/skills/{shared,<user>}/ - Three tools: skill, list_skills, publish_skill - Users can publish private skills to shared - Integrate skills tools into Telegram bot - Create skill-creator meta-skill at _/var/ava/skills/shared/skill-creator/
14 daysBatch web_reader tool, much fasterBen Sima
Added retry with backoff, parallel proccessing, editing pages down to main content, summarization with haiku. It's so much faster and more reliable now. Plus improved the logging system and distangled the status UI bar from the logging module.
14 daysrefactor: Rename Bot to Ava, remove cost guardrailBen Sima
- Rename Omni/Bot.hs to Omni/Ava.hs - Delete Omni/Bot.scm (unused Guile version) - Remove cost limit (was 10 cents, now 0) - Increase max iterations from 10 to 50
14 daysfeat(deploy): Complete mini-PaaS deployment system (t-266)Ben Sima
- Add Omni/Deploy/ with Manifest, Deployer, Systemd, Caddy modules - Manifest CLI: show, update, add-service, list, rollback commands - Deployer: polls S3 manifest, pulls closures, manages systemd units - Caddy integration for dynamic reverse proxy routes - bild: auto-cache to S3, outputs STORE_PATH for push.sh - push.sh: supports both NixOS and service deploys - Biz.nix: simplified to base OS + deployer only - Services (podcastitlater-web/worker) now deployer-managed - Documentation: README.md with operations guide
2025-12-15Restrict new tools to Ben onlyBen Sima
python_exec, http_get/post, outreach_*, feedback_list, and read_file now require isBenAuthorized check, matching email/hledger pattern.
2025-12-15t-265.6: Add feedback collection endpoint for PILBen Sima
- Add feedback table with migration in Core.py - Add FeedbackForm and FeedbackPage UI components - Add /feedback GET/POST routes and /api/feedback JSON endpoint - Add admin feedback view at /admin/feedback - Create Omni/Agent/Tools/Feedback.hs with feedback_list tool - Wire feedback tool into Telegram agent
2025-12-14t-265.5: Add SMTP email sending for Ava outreachBen Sima
- Add emailSendTool to Email.hs for sending approved drafts - Add sendApprovedEmail function that checks draft status - Use Network.Mail.Mime.simpleMail' with SMTP.sendMail - Integrate with Outreach module to verify approval and mark sent - Add tests for new tool
2025-12-14t-265.4: Add read_file tool and PIL codebase context to AvaBen Sima
- Import Omni.Agent.Tools in Telegram.hs - Add readFileTool to Ava's tool list - Add podcastitlater context section to system prompt with key file paths
2025-12-14Fix message splitting to not use LLMBen Sima
The haiku-based splitting was modifying message content. Replace with deterministic paragraph-based splitting that preserves the original text exactly.
2025-12-14Add outreach approval queue for Ava (t-265.3)Ben Sima
- Create Omni/Agent/Tools/Outreach.hs with tools - Drafts stored in _/var/ava/outreach/{pending,approved,...} - Add Telegram commands: /review, /approve, /reject, /queue - Integrate outreach tools into agent's tool list Amp-Thread-ID: https://ampcode.com/threads/T-019b202c-2156-74db-aa4a-e0a2f4397fbb Co-authored-by: Amp <amp@ampcode.com>
2025-12-14Add HTTP request tools for agent API interactionsBen Sima
- Create Omni/Agent/Tools/Http.hs with http_get and http_post tools - Support headers, query params, JSON body, 30s timeout - Return structured JSON with status, headers, body - Add 9 unit tests including real HTTP calls to httpbin.org - Wire tools into Telegram agent's tool list Completes t-265.2
2025-12-14Add python_exec tool for agent Python executionBen Sima
- Create Omni/Agent/Tools/Python.hs with python_exec tool - Execute Python snippets via subprocess with 30s default timeout - Return structured JSON with stdout, stderr, exit_code - Add 8 unit tests covering print, imports, errors, timeout - Wire tool into Telegram agent's tool list Completes t-265.1
2025-12-14telegram: switch to HaskellNet for IMAP, fix message delivery bugsBen Sima
- Replace openssl s_client with HaskellNet/HaskellNet-SSL for proper IMAP client support (better protocol handling, no manual parsing) - Add HaskellNet deps to Haskell.nix with doJailbreak for version bounds - Fix lost messages: sendMessageReturningId now throws on API errors instead of returning Nothing (which was incorrectly treated as success) - Auto-retry markdown parse errors as plain text - Hardcode benChatId for reliable email check loop startup
2025-12-13Add hledger tools to Telegram botBen Sima
- New Omni/Agent/Tools/Hledger.hs with 5 tools: - hledger_balance: query account balances - hledger_register: show transaction history - hledger_add: create new transactions - hledger_income_statement: income vs expenses - hledger_balance_sheet: net worth view - All tools support currency parameter (default: USD) - Balance, register, income_statement support period parameter - Period uses hledger syntax (thismonth, 2024, from X to Y) - Shell escaping fixed for multi-word period strings - Authorization: only Ben and Kate get hledger tools - Max iterations increased from 5 to 10 - Transactions written to ~/fund/telegram-transactions.journal
2025-12-13telegram: round cost to 2 decimal places in logsBen Sima
2025-12-13telegram: per-user memory in groups, continuous typingBen Sima
Memory changes: - Add thread_id column to conversation_messages for topic support - Add saveGroupMessage/getGroupConversationContext for shared history - Add storeGroupMemory/recallGroupMemories with 'group:<chat_id>' user - Fix SQLite busy error: set busy_timeout before journal_mode Telegram changes: - Group chats now use shared conversation context (chat_id, thread_id) - Personal memories stay with user, group memories shared across group - Memory context shows [Personal] and [Group] prefixes - Add withTypingIndicator: refreshes typing every 4s while agent thinks - Fix typing UX: indicator now shows continuously until response sent
2025-12-13feat: add Telegram topic (message_thread_id) supportBen Sima
- Parse message_thread_id from incoming messages - Include thread_id in sendMessage API calls - Pass thread_id through message queue system - Replies now go to the correct topic in supergroups
2025-12-13fix: use OpenAI Whisper for voice transcriptionBen Sima
OpenRouter's chat completion API doesn't properly pass audio to models. Switched to calling OpenAI's /v1/audio/transcriptions endpoint directly with the whisper-1 model. Requires OPENAI_API_KEY environment variable.
2025-12-13Enable WAL mode and busy timeout for SQLite concurrencyBen Sima
Fixes 'database is locked' errors when multiple threads access the memory database simultaneously (incoming batcher, message dispatch, reminder loop, main handler).
2025-12-13Add incoming message queue for Telegram botBen Sima
Batches incoming messages by chat_id with a 3-second sliding window before processing. This prevents confusion when messages arrive simultaneously from different chats. - New IncomingQueue module with STM-based in-memory queue - Messages enqueued immediately, offset acked on enqueue - 200ms tick loop flushes batches past deadline - Batch formatting: numbered messages, sender attribution for groups, media stubs, reply context - Media from first message in batch still gets full processing
2025-12-13telegram: fix audio transcription model and prompt orderBen Sima
- Switch from gemini-2.0-flash-001 to gemini-2.5-flash - Put audio content before text prompt (model was ignoring audio) - Strengthen prompt to return only transcription
2025-12-13telegram: unified message queue with async/scheduled sendsBen Sima
- Add Messages.hs with scheduled_messages table and dispatcher loop - All outbound messages now go through the queue (1s polling) - Disable streaming responses, use runAgentWithProvider instead - Add send_message tool for delayed messages (up to 30 days) - Add list_pending_messages and cancel_message tools - Reminders now queue messages instead of sending directly - Exponential backoff retry (max 5 attempts) for failed sends
2025-12-13Fix Telegram streaming markdown parse errorsBen Sima
Amp-Thread-ID: https://ampcode.com/threads/T-019b1894-b431-777d-aba3-65a51e720ef2 Co-authored-by: Amp <amp@ampcode.com>
2025-12-13Add ISO 8601 timestamps to conversation context messagesBen Sima
2025-12-13Add knowledge graph with typed relations to Memory moduleBen Sima
- Add RelationType with 6 relation types - Add MemoryLink type and memory_links table - Add graph functions: linkMemories, getMemoryLinks, queryGraph - Add link_memories and query_graph agent tools - Wire up graph tools to Telegram bot - Include memory ID in recall results for linking - Fix streaming usage parsing for cost tracking Closes t-255 Amp-Thread-ID: https://ampcode.com/threads/T-019b181f-d6cd-70de-8857-c445baef7508 Co-authored-by: Amp <amp@ampcode.com>
2025-12-13feat: only allow whitelisted users to add bot to groupsBen Sima
When the bot is added to a group, check if the user who added it is in the whitelist. If not, send a message explaining and leave the group immediately. This prevents unauthorized users from bypassing DM access controls by adding the bot to a group.
2025-12-13feat: allow all users in group chats, whitelist only for DMsBen Sima
2025-12-13feat: enable Markdown rendering in Telegram messagesBen Sima
Add parse_mode=Markdown to sendMessage and editMessage API calls
2025-12-13fix: accumulate streaming tool call arguments across SSE chunksBen Sima
OpenAI's SSE streaming sends tool calls incrementally - the first chunk has the id and function name, subsequent chunks contain argument fragments. Previously each chunk was treated as a complete tool call, causing invalid JSON arguments. - Add ToolCallDelta type with index for partial tool call data - Add StreamToolCallDelta chunk type - Track tool calls by index in IntMap accumulator - Merge argument fragments across chunks via mergeToolCallDelta - Build final ToolCall objects from accumulator when stream ends - Handle new StreamToolCallDelta in Engine.hs pattern match
2025-12-13telegram: add conversation context to group pre-filterBen Sima
Pre-filter now sees last 5 messages so it can detect when user is continuing a conversation with Ava, even without explicit mention. - Fetch recent messages before shouldEngageInGroup - Update classifier prompt to understand Ava context - Handle follow-up messages to bot's previous responses
2025-12-13fix: correct cost estimation formulasBen Sima
- Update to Dec 2024 OpenRouter pricing - Use blended input/output rates - Add gemini-flash, claude-sonnet-4.5 specific rates - Fix math: was off by ~30x for Claude models
2025-12-13telegram: add cheap pre-filter for group messagesBen Sima
Use Gemini Flash to classify group messages before running the full Sonnet agent. Skips casual banter to save tokens/cost. - shouldEngageInGroup: yes/no classifier using gemini-2.0-flash - Only runs for group chats, private chats skip the filter - On classifier failure, defaults to engaging (fail-open)
2025-12-13telegram: intelligent group response (LLM decides when to speak)Ben Sima
- Remove mention-based filtering, bot sees all group messages - Add response rules to system prompt for group chats: - tool invocation = always respond - direct question = respond - factual correction = maybe respond - casual banter = stay silent - Empty response in group = intentional silence (no fallback msg) - Add chat type context to system prompt
2025-12-13telegram: add group chat supportBen Sima
- Only respond in groups when @mentioned or replied to - Add ChatType to TelegramMessage (private/group/supergroup/channel) - Add getMe API call to fetch bot username on startup - Add shouldRespondInGroup helper function
2025-12-13telegram: fix parsing, add webpage reader, use geminiBen Sima
- Fix Provider.hs to strip leading whitespace from OpenRouter responses - Fix FunctionCall parser to handle missing 'arguments' field - Use eitherDecode for better error messages on parse failures - Switch to claude-sonnet-4.5 for main agent - Use gemini-2.0-flash for conversation summarization (cheaper) - Add read_webpage tool for fetching and summarizing URLs - Add tagsoup to Haskell deps (unused, kept for future)
2025-12-12telegram bot: refactor + multimedia + reply supportBen Sima
Refactor Telegram.hs into submodules to reduce file size: - Types.hs: data types, JSON parsing - Media.hs: file downloads, image/voice analysis - Reminders.hs: reminder loop, user chat persistence Multimedia improvements: - Vision uses third-person to avoid LLM confusion - Better message framing for embedded descriptions - Size validation (10MB images, 20MB voice) - MIME type validation for voice messages New features: - Reply support: bot sees context when users reply - Web search: default 5->10, max 10->20 results - Guardrails: duplicate tool limit 3->10 for research - Timezone: todos parse/display in Eastern time (ET)
2025-12-12feat: add image and voice message support for Telegram botBen Sima
- Add TelegramPhoto and TelegramVoice types - Parse photo and voice fields from Telegram updates - Download photos/voice via Telegram API - Analyze images using Claude vision via OpenRouter - Transcribe voice messages using Gemini audio via OpenRouter - Wire multimedia processing into handleAuthorizedMessage Photos are analyzed with user's caption as context. Voice messages are transcribed and treated as text input.