# Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation This site has an agentic ask interface you may use to query the documentation dynamically by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://dotagent.avelino.run/index.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. --- # Introduction > Write your agent in **any language**. dotagent schedules it, supervises it, and tells you when it breaks. dotagent is a scheduler and supervisor for the small jobs you run every day. It is **not** an agent runtime, not an SDK, and not an AI product. Your agent stays a script you own; dotagent handles the parts that are boring to write and expensive to get wrong. ## How this got here I'm a CTO. I have a handful of small tasks I run every day: pulling metrics, drafting reports, triaging the inbox, briefing myself before the week starts. So I did what every engineer does, I automated them. **Step 1: an LLM ran my schedule.** I wrote the agenda in plain markdown and let the model do the rest. Worked for a week. Then I noticed the bill: every run, the model was *generating a fresh script* to execute the same piece of the schedule. I was burning tokens writing throwaway code the model had already written yesterday, last week, and the week before. Orchestration cost more than the actual work. **Step 2: I wrote the scripts myself.** I'm a fish user, so they became fish. I kept `claude -p` only where I actually needed judgment: drafting a sentence, classifying ambiguous input, summarizing a thread. Everything else was deterministic shell. Tokens dropped, runs got faster, the work itself stayed the same. **Step 3: a framework appeared.** As agents multiplied, the same boilerplate showed up everywhere: load config, write heartbeat, retry, notify on failure. I extracted it into `lib/agent.fish`, then a tiny orchestrator on top, then wired the whole thing into launchd so the laptop wakes each agent at the right time. **Step 4: dotagent.** Same architecture, rewritten in Rust, shipped as a single binary. No fish dependency, no shell-specific assumptions, OS-native scheduling on macOS *and* Linux, plugins in any language. > If you're at step 1 (an LLM orchestrating shell scripts) or step 2 (hand-rolled scripts), this is the road you were already going to walk. dotagent is what you'd build if you had the patience to refactor it. ## Why You have a handful of jobs that run on a schedule: * A daily report that pulls metrics from N APIs * A 90-min poll that classifies your inbox * A weekly snapshot that publishes to your knowledge base * A morning briefing that pings you on iMessage You started in `cron`. Then you needed retries. Then notifications. Then preflight checks ("only if VPN is up"). Then health visibility. Pretty soon your "tiny shell script" is 800 lines and the orchestration competes with the actual work. **dotagent extracts the orchestration.** Your agent stays small (Fish, Python, Go, Rust, anything that reads env vars and exits). dotagent handles the load-bearing parts: * **OS-native scheduling** — generates a launchd plist / systemd unit. No polling, no `sleep` loops. * **Adaptive supervisor** — one daemon sleeps until the next event, wakes, dispatches, sleeps again. * **Retries + backoff per schedule** — missed windows get detected and retried, with an out-of-band notification when they give up. * **Notifications built in** — desktop, iMessage, Slack, ntfy and Pushover ship inside the daemon. No extra binaries on `$PATH`. * **Pluggable I/O** — preflight checks and output sinks are external binaries speaking JSON over stdio. Write one in any language. * **Local client transport** — `dotagent api` bridges raw JSONL over a user-local Unix socket, so scripts and TUI backends can stream assistant output without turning the daemon into a conversation runtime. It is not a TUI itself. * **No SDK** — your agent reads env vars and exits with a code. That's the entire API. ## What it looks like ```mermaid flowchart LR M[agent.toml] --> D[dotagent daemon] D -->|spawn| A[your agent script] A -->|stdout| S[sink plugin] A -->|fail| N[built-in notifier] D --> H[(heartbeat + audit log)] ``` A manifest declares who the agent is, when it runs, and what happens on success or failure: ```toml [agent] name = "morning-briefing" timeout_seconds = 300 [run] command = "python3" args = ["./brief.py"] [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] hours = [8] minute = 30 [[notifiers]] driver = "imessage" to = "+5511999999999" events = ["given_up"] ``` Every weekday at 08:30 the daemon wakes, runs `brief.py`, records the heartbeat, retries with backoff on failure, and pings your phone once it gives up. ## Where to go next * [Installation](/getting-started/installation) — every install path, with verify steps * [Your first agent](/getting-started/first-agent) — zero to daemon-managed in 15 minutes * [Architecture](/concepts/architecture) — daemon, runner, plugins, state * [Agents](/concepts/agents) — the patterns worth copying * [Local client API](/reference/local-api) — Unix-socket JSON-lines contract * [LLM agents](/guides/llm-agents) — calling `claude -p` from an agent, and the headless gotchas * [FAQ](/misc/faq) — quick answers **Writing an agent with an LLM?** Point it at [`llms.txt`](https://github.com/avelino/dotagent/tree/main/docs/llms.txt), a single-fetch digest with the full manifest schema, env vars, every notifier driver and plugin, exit code semantics, and worked examples. Most models can write a working `agent.toml` zero-shot after reading it. Canonical raw URL for `WebFetch`: `https://raw.githubusercontent.com/avelino/dotagent/main/docs/llms.txt` ## Status Pre-release. The manifest schema, plugin protocol and heartbeat shape are stable. Follow [issues](https://github.com/avelino/dotagent/issues) for milestones. The project lives at [github.com/avelino/dotagent](https://github.com/avelino/dotagent) under the MIT license. # Installation > Pick one path. All four end with `dotagent --version` working and `dotagent doctor` reporting "no agents discovered" (which is the healthy state before you write your first agent). dotagent ships as a single Rust binary plus five first-party plugin binaries. There's nothing to compile at runtime, nothing to install in a language ecosystem (no `node_modules`, no `pip`, no `pyenv`). Pick: | Path | When to use | Platforms | | ----------------------------------------------------- | --------------------------------------------------------------- | ---------------------------------------- | | [Homebrew](#1-homebrew-recommended) | macOS / Linux with Homebrew. The repo doubles as its own tap. | macOS, Linux (homebrew on linux) | | [GitHub Release binaries](#2-github-release-binaries) | You want a prebuilt binary today and don't have Rust toolchain. | macOS arm64/x86\_64, Linux arm64/x86\_64 | | [`cargo install`](#3-cargo-install) | You have Rust stable and want the latest from `main`. | macOS, Linux | | [Build from source](#4-build-from-source) | You want to hack on dotagent itself. | macOS, Linux | > **Windows is not supported.** dotagent uses `kill(2)`, `osascript`, launchd, systemd — all Unix-shaped. Linux + WSL works; native Windows doesn't. After whichever path you pick, jump to [Verify the install](#verify-the-install). *** ## 1. Homebrew (recommended) The repo doubles as its own Homebrew tap — the formula lives at [`Formula/dotagent.rb`](https://github.com/avelino/dotagent/blob/main/Formula/dotagent.rb) in the default branch, and the release workflow rewrites it with fresh `sha256` values on every tagged release. No separate `homebrew-dotagent` repo to chase. ```bash brew tap avelino/dotagent https://github.com/avelino/dotagent brew install dotagent brew services start dotagent # runs `dotagent daemon` via launchd / systemd ``` > The URL is required because the tap repo isn't named `homebrew-dotagent`. Run `brew tap` once — afterwards `brew install dotagent` and `brew upgrade dotagent` Just Work. `brew install dotagent` drops `dotagent` plus every first-party plugin (`dotagent-plugin-preflight-warp`, `dotagent-plugin-sink-roam`, etc.) into the same `bin/`, so plugin discovery via `$PATH` works with zero config. `brew services start` registers the daemon via `launchctl bootstrap` (macOS) / `systemctl --user enable --now` (Linux). Skip this step if you'd rather manage the daemon yourself — see [`guides/daemon-lifecycle.md`](/guides/daemon-lifecycle). ### 1a. Beta channel (`dotagent@beta`) Every push to `main` republishes a rolling `beta` GitHub release and rewrites [`Formula/dotagent@beta.rb`](https://github.com/avelino/dotagent/blob/main/Formula/dotagent@beta.rb) with fresh `sha256` values. The formula version is `0.0.1-beta.`, monotonic so `brew upgrade dotagent@beta` detects every new build. ```bash brew tap avelino/dotagent https://github.com/avelino/dotagent brew install dotagent@beta brew link --overwrite dotagent@beta # `dotagent@beta` is keg_only brew services start dotagent@beta ``` `dotagent@beta` is `keg_only` (Homebrew convention for versioned formulae), so `brew link --overwrite` is required to put `dotagent` on `$PATH`. To switch back to stable: ```bash brew unlink dotagent@beta brew link dotagent ``` > **Beta caveats**: builds may break — the `beta` channel tracks the tip of `main`, including in-flight refactors. For production use, stick with stable. The `beta` tag on GitHub Releases is **rolling**: it's deleted and recreated on every push, so the URL `releases/download/beta/...` always serves the latest build. Use stable (`v0.0.1`, `v0.0.2`, …) when you need a pinned version. *** ## 2. GitHub Release binaries Each tagged release publishes signed archives for **macOS arm64**, **macOS x86\_64**, **Linux arm64**, and **Linux x86\_64**. ```bash # Pick your platform (run `uname -ms` if unsure): # Darwin arm64 → aarch64-darwin # Darwin x86_64 → x86_64-darwin # Linux aarch64 → aarch64-linux # Linux x86_64 → x86_64-linux VERSION="0.0.1" # check https://github.com/avelino/dotagent/releases ARCH="aarch64-darwin" # adjust NAME="dotagent-${VERSION}-${ARCH}" curl -L -o "${NAME}.tar.gz" "https://github.com/avelino/dotagent/releases/download/v${VERSION}/${NAME}.tar.gz" curl -L -o "${NAME}.tar.gz.sha256" "https://github.com/avelino/dotagent/releases/download/v${VERSION}/${NAME}.tar.gz.sha256" # Verify (the sha256 file ships next to the archive). shasum -a 256 -c "${NAME}.tar.gz.sha256" # Extract. tar -xzf "${NAME}.tar.gz" # → bin/dotagent bin/dotagent-plugin-* LICENSE README.md ``` Install the binaries somewhere on `$PATH`: ```bash # Option A: user-local (no sudo) mkdir -p ~/.local/bin cp bin/dotagent bin/dotagent-plugin-* ~/.local/bin/ export PATH="$HOME/.local/bin:$PATH" # add to ~/.zshrc / ~/.bashrc # Option B: system-wide (sudo) sudo cp bin/dotagent bin/dotagent-plugin-* /usr/local/bin/ ``` > **Plugins must live next to `dotagent`** or somewhere on `$PATH` — the CLI resolves them by name (`dotagent-plugin-sink-roam` etc.) via standard `$PATH` lookup. See [`reference/paths.md`](/reference/paths) for the full discovery order. *** ## 3. `cargo install` If you already have a Rust toolchain (stable, 1.75+ recommended): ```bash git clone https://github.com/avelino/dotagent cd dotagent # Install the CLI. cargo install --path crates/dotagent # Install each plugin you want. cargo install --path plugins/preflight-warp cargo install --path plugins/preflight-cmd cargo install --path plugins/sink-roam cargo install --path plugins/sink-outl cargo install --path plugins/sink-file ``` The binaries land in `~/.cargo/bin/`, which most installers add to `$PATH` automatically. Verify: ```bash which dotagent # → ~/.cargo/bin/dotagent which dotagent-plugin-preflight-warp # → ~/.cargo/bin/... ``` Don't forget the plugins — `cargo install --path crates/dotagent` alone does NOT pull them in. `dotagent doctor` will tell you which ones are referenced by your manifests but missing on `$PATH`. *** ## 4. Build from source For contributors and anyone wanting to vendor dotagent into a custom distribution. ```bash git clone https://github.com/avelino/dotagent cd dotagent # Build the whole workspace (CLI + every plugin). cargo build --release --workspace # Binaries end up in target/release/. Drop them on $PATH manually: cp target/release/dotagent ~/.local/bin/ cp target/release/dotagent-plugin-* ~/.local/bin/ ``` Self-verification (matches the CI matrix — never report "done" without these three passing): ```bash cargo fmt --all -- --check cargo clippy --workspace --all-targets -- -D warnings cargo test --workspace --all-targets ``` A faster sanity check that exercises the discovery + run path: ```bash DOTAGENT_ROOT=$PWD/examples ./target/release/dotagent doctor DOTAGENT_ROOT=$PWD/examples ./target/release/dotagent run hello-fish --schedule manual --dry-run ``` If both succeed, the workspace is healthy. *** ## Verify the install Whichever path you picked, run: ```bash dotagent --version # → dotagent 0.0.1 (or whichever) dotagent --help # → polyglot agent orchestrator. Usage: dotagent dotagent doctor # → no agents discovered ← expected, you haven't written one yet ``` If all three work, you're done. Skip ahead to [`first-agent.md`](/getting-started/first-agent). If `dotagent --help` works but `dotagent doctor` fails, see the [CLI reference](/reference/cli#doctor) and the [troubleshooting guide](/guides/troubleshooting). *** ## Shell completion (optional) Tab-completion of subcommands, flags, and **agent names** (dynamic — runs against your live manifests, not a baked-in list). ```bash # fish dotagent completions fish | source # or persistent: dotagent completions fish > ~/.config/fish/completions/dotagent.fish # zsh dotagent completions zsh > ~/.zfunc/_dotagent # ensure ~/.zfunc is in $fpath # bash dotagent completions bash > ~/.local/share/bash-completion/completions/dotagent ``` After install, `dotagent run hel` expands to `hello-fish hello-go hello-python hello-rust`. See [`reference/cli.md#completions`](/reference/cli#completions). *** ## What got installed where After install you should find: | What | Path | | ------------------------------- | ------------------------------------------------------------------------- | | `dotagent` binary | depends on path — `~/.cargo/bin/`, `~/.local/bin/`, `/opt/homebrew/bin/`… | | Plugin binaries | same directory as `dotagent` (or anywhere on `$PATH`) | | Default home | `~/.config/dotagent/` (created lazily on first run) | | Default agents directory | `~/.config/dotagent/agents/` | | launchd plist (after `install`) | `~/Library/LaunchAgents/run.avelino.dotagent.plist` (macOS) | | systemd unit (after `install`) | `~/.config/systemd/user/run.avelino.dotagent.service` (Linux) | The full disk layout is documented in [`reference/paths.md`](/reference/paths). *** ## Upgrading | Path | How to upgrade | | ------------------ | -------------------------------------------------------------------------------------- | | Homebrew (stable) | `brew update && brew upgrade dotagent` | | Homebrew (`@beta`) | `brew update && brew upgrade dotagent@beta` (new build on every push to `main`) | | GitHub Release | Download the new archive, overwrite binaries in `~/.local/bin/` (or `/usr/local/bin/`) | | `cargo install` | `cargo install --path crates/dotagent --force` (and same for each plugin) | | Source build | `git pull && cargo build --release --workspace` then re-copy | After upgrading: ```bash dotagent reload # daemon picks up new manifests + plugin set on next tick ``` `reload` sends SIGHUP — it doesn't restart the binary. If you swapped the `dotagent` binary itself, restart the daemon instead: ```bash # macOS (launchd) launchctl kickstart -k "gui/$(id -u)/run.avelino.dotagent" # Linux (systemd) systemctl --user restart run.avelino.dotagent ``` *** ## Uninstall ```bash # 1. Stop & remove the daemon unit. dotagent uninstall # → removes ~/Library/LaunchAgents/run.avelino.dotagent.plist (or systemd unit) # 2. Remove the binaries (path varies). brew uninstall dotagent # if homebrew rm ~/.cargo/bin/dotagent ~/.cargo/bin/dotagent-plugin-* # if cargo rm ~/.local/bin/dotagent ~/.local/bin/dotagent-plugin-* # if release / source ``` dotagent **does not delete** `~/.config/dotagent/` on uninstall — your manifests, heartbeats, audit log, and config stay put. ```bash # Nuke everything (irreversible). rm -rf ~/.config/dotagent ``` *** ## Next * [Write your first agent →](/getting-started/first-agent) * Already have an agent? [CLI reference](/reference/cli) and [daemon lifecycle](/guides/daemon-lifecycle). # Your first agent > Goal: in 15 minutes you'll have an agent that prints "hello" every few minutes under the dotagent daemon, with logs you can tail and a desktop notification when it fails. Prerequisites: `dotagent --version` works. If not, go back to [`installation.md`](/getting-started/installation). The walkthrough has six steps. Each step ends with a "**Verify**" line — if that command doesn't behave as described, jump to [Troubleshooting](#troubleshooting) at the bottom. *** ## Step 1 — Pick the language Every example below uses **Fish** for brevity, but dotagent runs any executable. If you'd rather use Python, Go, or Rust, copy [`examples/hello-python/`](https://github.com/avelino/dotagent/tree/main/examples/hello-python/README.md), [`examples/hello-go/`](https://github.com/avelino/dotagent/tree/main/examples/hello-go/README.md), or [`examples/hello-rust/`](https://github.com/avelino/dotagent/tree/main/examples/hello-rust/README.md) instead. The flow is identical — only `[run].command` and the script body change. ```bash # Verify your shell of choice exists. which fish # or: which python3 / which go / which cargo ``` *** ## Step 2 — Create the agent directory dotagent discovers agents by scanning `~/.config/dotagent/agents/` (and a few other roots — see [`reference/paths.md`](/reference/paths)). Each agent is a single directory. ```bash mkdir -p ~/.config/dotagent/agents/hello cd ~/.config/dotagent/agents/hello ``` **Verify:** ```bash ls -ld ~/.config/dotagent/agents/hello # → drwxr-xr-x ... hello ``` *** ## Step 3 — Write the script This script just echoes the env vars dotagent injects. It exits 0 most of the time and exits 1 every 5th minute so we can see the failure notification later. ```fish # ~/.config/dotagent/agents/hello/agent.fish #!/usr/bin/env fish # dotagent injects: # $AGENT_NAME — manifest's agent.name # $AGENT_TMPDIR — scratch dir, auto-cleaned after the run # $AGENT_SCHEDULE_ID — which schedule fired # $AGENT_START_EPOCH — unix epoch of the start echo "=== hello from $AGENT_NAME ===" echo "schedule: $AGENT_SCHEDULE_ID" echo "tmp: $AGENT_TMPDIR" echo "epoch: $AGENT_START_EPOCH" # Every 5 minutes (00, 05, 10...) fail on purpose so we can see the # notifier kick in. Comment this out once you've seen one alert. set -l minute (date +%M | string trim --left --chars=0) if test -z "$minute"; set minute 0; end if test (math "$minute % 5") -eq 0 echo "intentional failure for demo" >&2 exit 1 end exit 0 ``` Make it executable: ```bash chmod +x ~/.config/dotagent/agents/hello/agent.fish ``` **Verify** (running the script directly should work — env vars will be empty, that's fine): ```bash ./agent.fish # → === hello from === # → schedule: # → ... # → exit 0 (or 1 if you're on a minute divisible by 5) ``` *** ## Step 4 — Write the manifest `agent.toml` is the contract. It declares identity, how to run the script, when to run it, what to do on failure. ```toml # ~/.config/dotagent/agents/hello/agent.toml [agent] name = "hello" description = "Tutorial agent — prints hello every 2 minutes." timeout_seconds = 30 [run] command = "fish" args = ["./agent.fish"] # Run every 2 minutes so you don't have to wait long. [[schedules]] id = "every-2min" type = "interval" interval_minutes = 2 # Retry policy: 2 retries, 1 minute between attempts. # Used so we can see `attempt_failed` AND `given_up` events fire. [defaults] max_retries = 2 retry_backoff_minutes = [1] # Desktop banner whenever a run fails (every attempt). # Built-in driver — no plugin subprocess. [[notifiers]] driver = "desktop" title = "hello" sound = true events = ["attempt_failed", "given_up"] ``` That's the whole manifest. No SDK. No imports. No registration. **Verify** — dotagent should now discover the agent and validate the manifest end-to-end: ```bash dotagent doctor # → ✓ hello: manifest ok # → notifier driver=desktop (built-in) # → ⚠ hello: no [security] section — blast radius is unbounded. # → summary: 1 agent(s), 0 error(s), 1 warning(s) ``` The `[security]` warning is harmless for now — v0 is schema-only. Read [`security/threat-model.md`](/security/threat-model) when you're ready. *** ## Step 5 — Smoke-test (foreground) Before handing the agent to the daemon, run it once in the foreground. This is what `dotagent run` does — useful for development too. ```bash dotagent run hello --schedule every-2min # === hello from hello === # schedule: every-2min # tmp: /tmp/.tmpXXXXXX # epoch: 1700000000 ``` The agent ran in the foreground, dotagent injected env vars, captured stdout, wrote the heartbeat, and exited. **Verify**: ```bash ls ~/.config/dotagent/state/agents/hello/ # → default.heartbeat.json cat ~/.config/dotagent/state/agents/hello/default.heartbeat.json | jq . # → { "name": "hello", "slug": "default", "started_at": ..., "exit_code": 0, ... } ``` If `exit_code = 1` (because you hit a minute divisible by 5), that's the demo failure firing — try again a minute later. *** ## Step 6 — Hand it to the daemon Now the real thing: the daemon will fire the agent every 2 minutes on its own, retry on failure, and ping you on the desktop when it gives up. ### 6a. Install the daemon unit ```bash dotagent install # wrote ~/Library/LaunchAgents/run.avelino.dotagent.plist (macOS) # OR # wrote ~/.config/systemd/user/run.avelino.dotagent.service (Linux) # # Next steps: # launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` This writes **one** unit file. dotagent does NOT install one unit per agent — the daemon manages every schedule internally. ### 6b. Start the daemon **macOS (launchd):** ```bash launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` **Linux (systemd):** ```bash systemctl --user daemon-reload systemctl --user enable --now run.avelino.dotagent ``` The daemon process is now alive. macOS launchd / Linux systemd will restart it if it crashes. **Verify** (the daemon should be running and have written a PID file): ```bash cat ~/.config/dotagent/state/daemon.pid # → 12345 ps -p $(cat ~/.config/dotagent/state/daemon.pid) -o command= # → dotagent daemon ``` ### 6c. Watch it work Tail the agent's log — every 2 minutes a new "=== hello ===" header appears: ```bash dotagent logs hello --follow ``` In another terminal, see the dashboard: ```bash dotagent status ``` ``` ═══ Agent Health · 2026-08-06 14:32 ═══ ✅ ok 1/1 ⚠️ degraded 0 ❌ failing 0 🕑 stale 0 AGENT/SCHEDULE STATE LAST RUN REASON ──────────────────────────────────────────────────────────────────────────────────────────────────── hello/every-2min ✅ ok 2026-08-06T14:30:01-0300 ok ``` One row per `(agent, schedule)`, and `REASON` says which piece of state decided the verdict — see [`reference/cli.md`](/reference/cli#reading-the-reason-column). Wait for a "demo failure" minute (`:00`, `:05`, `:10`...) and you'll see: * A **desktop banner** ("hello — free space low" style) pop up * The dashboard shifts the agent into `failing`, with the attempts burned so far in `REASON` (`2 attempts, will retry`) * After the retry budget runs out it transitions to `given_up`, and `REASON` switches to `gave up after 3 attempts` * The audit log records every step ```bash tail ~/.config/dotagent/audit.log | jq -c . # → {"event": {"event_type":"agent_run","exit_code":1,...}} # → {"event": {"event_type":"plugin_invoked","plugin":"notifier:desktop",...}} # → {"event": {"event_type":"agent_given_up", ...}} ``` *** ## Where files live now After Step 6, `~/.config/dotagent/` looks like: ``` ~/.config/dotagent/ ├── agents/hello/ │ ├── agent.toml │ └── agent.fish ├── state/ │ ├── agents/hello/default.heartbeat.json │ ├── windows/hello-default-.json │ ├── known_manifests.json │ └── daemon.pid ├── logs/ │ ├── daemon/dotagent.log │ └── agents/hello/hello.log └── audit.log ``` Full reference: [`reference/paths.md`](/reference/paths). *** ## What just happened (mental model) ```mermaid sequenceDiagram participant launchd participant daemon as dotagent daemon participant fish as agent.fish participant desktop as desktop notifier launchd->>daemon: KeepAlive=true (at login) daemon->>daemon: sleep until next event Note over daemon: every-2min schedule fires daemon->>fish: spawn with AGENT_* env vars fish-->>daemon: stdout + exit code Note over daemon: exit=1 → attempt_failed daemon->>desktop: send banner daemon->>daemon: retry 1min later Note over daemon: 2nd retry → given_up daemon->>desktop: send "given_up" banner daemon->>daemon: back to sleep ``` The OS keeps the daemon alive. The daemon keeps everything else alive. *** ## Clean up If this was just a tutorial: ```bash # Stop the daemon. launchctl bootout "gui/$(id -u)/run.avelino.dotagent" # macOS systemctl --user disable --now run.avelino.dotagent # Linux # Remove the daemon unit. dotagent uninstall # Remove the agent (your manifests/state stay otherwise untouched). rm -rf ~/.config/dotagent/agents/hello rm -rf ~/.config/dotagent/state/agents/hello rm -rf ~/.config/dotagent/logs/agents/hello ``` *** ## Troubleshooting ### `dotagent doctor` says "no agents discovered" ```bash ls ~/.config/dotagent/agents/hello/agent.toml # → file must exist ``` Discovery scans `~/.config/dotagent/agents/*/agent.toml`. Make sure the directory name is `hello` and the file is named exactly `agent.toml` (not `hello.toml` or `manifest.toml`). ### `dotagent run` says "agent not found" The agent name in `agent.toml` (`[agent].name = "hello"`) must match the argument you pass to `dotagent run`. The directory name is **advisory** — what dotagent indexes is `agent.name`. ### `dotagent run` works but the daemon never fires it Check the daemon is alive: ```bash ps -p $(cat ~/.config/dotagent/state/daemon.pid 2>/dev/null) -o command= 2>/dev/null # Should print "dotagent daemon" ``` If empty, the daemon isn't running. Re-do Step 6b. If alive, tail the daemon log: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq . ``` You should see a `tick` line every 30 minutes at most, plus a "dispatching run" line every 2 minutes. If you see `tick` but no "dispatching", the heartbeat says the previous window already succeeded — which is correct behavior. Wait 2 minutes. ### Desktop notification never fires ```bash # macOS — check Settings → Notifications → dotagent or your terminal app # is allowed to send notifications. # Linux — check `notify-send` works: notify-send "test" "hello" ``` If `notify-send` doesn't show anything, the `desktop` driver won't either — it's the same D-Bus pathway. See [`guides/troubleshooting.md`](/guides/troubleshooting) for the full sintoma → diagnostics map. *** ## Next You have: * A working agent * The daemon running * Logs you can tail * A notification that fires on failure **Where to next?** [`next-steps.md`](/getting-started/next-steps) maps the rest of the docs to your next questions. # Next steps > You have a working agent under the daemon. Pick what's next based on what you want to do. ```mermaid flowchart TD A[Working agent + daemon] --> B{What now?} B --> N["Notify when things go wrong
→ notifications"] B --> M["Add more agents
→ agents concept"] B --> P["Persist output somewhere
→ plugins / sinks"] B --> S["Schedule by cron-style
→ agent spec"] B --> O["See into the daemon
→ observability"] B --> F["Migrate from Fish/cron
→ migration guide"] B --> D["Operate the daemon
→ daemon lifecycle"] B --> T["Something broke
→ troubleshooting"] ``` *** ## Make failures loud The example in [`first-agent.md`](/getting-started/first-agent) already wired a **desktop banner**. The rest of the built-in drivers (`slack`, `ntfy`, `pushover`, `telegram`, `imessage`) follow the same shape — a `[[notifiers]]` block, no plugin install, no subprocess fork: ```toml [[notifiers]] driver = "slack" webhook_url = "${SLACK_WEBHOOK_URL}" events = ["given_up", "recovered"] ``` Note the `${VAR}`. A Slack webhook URL *is* the credential, and an `agent.toml` lives in a repo — put the value in `~/.config/dotagent/secrets.env` (mode `0600`) and dotagent resolves it at send time. The same holds for `ntfy.token`/`base_url`/`topic`, `pushover.token`/`user`, and `telegram.bot_token`. → **Read**: [`concepts/notifications.md`](/concepts/notifications) for every driver, per-driver message size caps, the tiered-notify pattern (desktop on every fail, phone only on `given_up`), and rate-limit semantics. [`concepts/secrets.md`](/concepts/secrets) covers the secrets file. *** ## Make output go somewhere A *sink plugin* persists the agent's stdout. Three come pre-installed: * **`sink-file`** — write to a file (overwrite or append). * **`sink-roam`** — publish hierarchical content to Roam Research, idempotent via `marker_regex`. * **`sink-outl`** — same shape as `sink-roam` but targets Outl, with a single batched delete + write call. ```toml [[on_success]] plugin = "sink-file" config = { path = "/Users/me/reports/daily.md", mode = "overwrite" } ``` → **Read**: [`docs/plugins/sink-file.md`](/built-in-plugins/sink/sink-file), [`docs/plugins/sink-roam.md`](/built-in-plugins/sink/sink-roam), [`docs/plugins/sink-outl.md`](/built-in-plugins/sink/sink-outl), and the overview at [`concepts/plugins.md`](/concepts/plugins). *** ## Gate the run on something external A *preflight plugin* fails the run before spawning the agent. Built-ins: * **`preflight-warp`** — abort if Cloudflare WARP isn't connected. * **`preflight-cmd`** — generic: run any command, gate on exit code + stdout match. ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "aws", args = ["sts", "get-caller-identity"], expect_exit = 0 } ``` → **Read**: [`docs/plugins/preflight-cmd.md`](/built-in-plugins/preflight/preflight-cmd), [`docs/plugins/preflight-warp.md`](/built-in-plugins/preflight/preflight-warp). *** ## Run it on demand, not only on schedule Every agent you install is already a callable tool: ```bash dotagent mcp # JSON-RPC over stdio, one tool per agent ``` Point Claude Code or Claude Desktop at it and you can ask for any agent by name: ```json { "mcpServers": { "dotagent": { "command": "dotagent", "args": ["mcp"] } } } ``` For a chat front end, inbound Telegram hands messages to a dispatcher agent whose stdout goes back to the conversation. Off until you configure it, and it changes the threat model — read [V8](/security/threat-model) first. → **Read**: [`docs/reference/mcp.md`](/reference/mcp), [`docs/concepts/telegram.md`](/concepts/telegram), [`docs/concepts/triggers.md`](/concepts/triggers). *** ## Schedule like a grown-up The tutorial used `type = "interval"` for speed. Production agents usually want **cron-style** schedules with weekdays + hours + minute: ```toml [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] # Mon-Fri (0 = Sun, 6 = Sat — matches launchd) hours = [8] minute = 30 ``` Plus retry policy: ```toml [defaults] max_retries = 3 retry_backoff_minutes = [5, 15, 30] stale_after_minutes = 120 # don't bother retrying a stale window ``` → **Read**: [`reference/agent-spec.md`](/reference/agent-spec) for the full manifest schema. [`concepts/agents.md`](/concepts/agents) for the patterns (digest / triage / generator / watchdog). *** ## See into the daemon Out of the box you get structured JSON logs, daily rotation, and a sharded log per agent — zero config. To export traces to Honeycomb / Grafana Tempo / Jaeger / Datadog, two lines of `config.toml`: ```toml [telemetry] otlp_endpoint = "https://api.honeycomb.io:443" ``` ```bash export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_KEY" dotagent reload ``` → **Read**: [`guides/observability.md`](/guides/observability) for log schema, vendor recipes, jq queries. → **Or just tweak retention**: [`guides/config-reference.md`](/guides/config-reference). *** ## Add many agents `agents//` is one directory per agent. Add a second: ```bash mkdir -p ~/.config/dotagent/agents/hn-morning # … drop your agent.toml + entry script … dotagent doctor # validates every agent at once dotagent reload # daemon picks up the new one on next tick ``` For inspiration: * **`examples/disk-alert/`** — pure-shell agent with tiered notifications * **`examples/hn-digest/`** — bash agent that calls `claude -p`, with the guards an LLM step needs (see [`guides/llm-agents.md`](/guides/llm-agents)) * **`examples/hello-{fish,python,go,rust}/`** — minimal "hello" variants per language * **The fish framework's gallery** — 9 production agents listed in [`concepts/agents.md#examples-gallery`](/concepts/agents#examples-gallery) → **Read**: [`concepts/agents.md`](/concepts/agents) — patterns, extending, connecting agents. *** ## Migrate from Fish / cron If you came from the `lib/agent.fish` framework, every concept maps 1-to-1: → **Read**: [`guides/migrating-from-fish.md`](/guides/migrating-from-fish). If you came from `cron` (no framework), the pattern is: 1. Move the script body into `agents//agent.sh`. 2. Write an `agent.toml` that recreates your cron line in the `[[schedules]]` block. 3. Remove the cron entry; let the daemon take over. There's no separate cron migration guide today — the Fish guide is the closest thing, and `cron` ↔ launchd weekday differences are flagged in [Troubleshooting](/guides/troubleshooting#symptom-cron-style-schedule-never-matches). *** ## Operate the daemon How to install/start/stop/reload across macOS launchd + Linux systemd, with diagnostics for "is this thing on?": → **Read**: [`guides/daemon-lifecycle.md`](/guides/daemon-lifecycle). *** ## Something broke A sintoma → diagnostic decision tree covering: * Daemon won't start * `doctor` errors * Agent never runs * Agent runs but fails * Notifier / sink not working * Logs / audit / plugin issues * Performance → **Read**: [`guides/troubleshooting.md`](/guides/troubleshooting). *** ## Read the source When the docs disagree with the code, **the code wins**. Worth bookmarking: | Question | Crate / module | | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | "What does the daemon actually do?" | [`crates/dotagent/src/commands/daemon.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent/src/commands/daemon.rs) | | "How does the runner spawn agents?" | [`crates/dotagent-runner/src/lib.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-runner/src/lib.rs) | | "How are heartbeats serialized?" | [`crates/dotagent-core/src/heartbeat.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-core/src/heartbeat.rs) | | "How does scheduling math work?" (pure functions, no IO) | [`crates/dotagent-scheduler/src/lib.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-scheduler/src/lib.rs) | | "What audit events exist?" | [`crates/dotagent-core/src/audit.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-core/src/audit.rs) | | "What's in `agent.toml`?" | [`crates/dotagent-core/src/manifest.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-core/src/manifest.rs) | For contributor onboarding: [`CLAUDE.md`](https://github.com/avelino/dotagent/tree/main/CLAUDE.md) at the repo root. *** ## Read the FAQ Quick answers to the recurring questions ([`docs/faq.md`](/misc/faq)): * Windows? root? cron → dotagent? debug without the daemon? multiple users? sandbox? when is 1.0? why not just cron? *** That's the menu. Each section above is an entry point into a deeper guide — pick the one closest to what you actually need, and ignore the rest until later. # Architecture > What the daemon does, what it doesn't, and which crate owns each decision. dotagent is built like a coffee shop, not a kitchen brigade. **One person behind the counter takes every order**. The "daemon" is that one person — it watches the schedules, decides what to fire next, supervises each agent, and goes back to waiting. There is no per-agent process running in the background. There is no scheduler poll loop. Read this if you want to: * Picture what happens between `dotagent install` and `your agent runs`. * Know which crate to touch when changing behavior. * Understand why a panicking plugin doesn't take down everything. For the user-facing concepts see [`agents.md`](/concepts/agents) and [`plugins.md`](/concepts/plugins). For the schema details see the [reference docs](/reference/agent-spec). *** ## The 30-second mental model ```mermaid flowchart LR OS["OS scheduler
(launchd / systemd)"] -->|spawn at boot| D[dotagent daemon] D -->|reads| M["agent.toml × N"] D -->|sleeps until
next event| D D -->|spawn| A[your agent script] A -->|stdout / exit code| D IN["Telegram / local UDS"] -->|trigger| G["gateway
admission + FIFO + delivery"] G -->|supervised run| A G -->|reply events| IN D -->|fire| P["plugins
(preflight, sink)"] D -->|fire| N["notifiers
(in-process)"] D -->|append| AU[(audit log)] D -->|write| ST[(state: heartbeat, window)] ``` Three things to internalize: 1. **The OS owns when the daemon runs.** launchd / systemd keep `dotagent daemon` alive. dotagent does not install itself as a service-aware program — `dotagent install` writes a unit file, and the OS does the rest. 2. **The daemon owns when each agent runs.** Not the OS. There is **one** unit file (`run.avelino.dotagent`), not one per agent. The daemon's adaptive scheduler computes the next event across every schedule of every agent and sleeps until that exact moment. 3. **Your agent is a one-shot subprocess.** It reads env vars, does its thing, writes stdout, exits. No SDK, no IPC, no long-running state. The one exception is opt-in: an agent declaring `[lifecycle] mode = "persistent"` is kept alive between runs and handed requests over JSON lines. Same supervisor, same deadline, same reaping — see [`lifecycle.md`](/concepts/lifecycle). 4. **The trigger gateway is transport and policy, not a conversation runtime.** It admits and orders Telegram/local triggers, caps concurrent conversations, and delivers output through the submitting transport. Session ids are opaque; transcripts, model state, and conversation persistence belong to the agent. Everything else in this doc is detail. *** ## Crate layout The workspace has eleven crates. Each has one responsibility and a single direction of "depends on" — no cycles. ```mermaid flowchart TD cli[dotagent
CLI binary] --> runner[dotagent-runner
spawn + hooks] cli --> scheduler[dotagent-scheduler
pure functions] cli --> mcp[dotagent-mcp
JSON-RPC + MCP types] cli --> unitgen[dotagent-unit-gen
plist / service] runner --> core[dotagent-core
types] runner --> state[dotagent-state
filesystem state] runner --> plugin[dotagent-plugin
subprocess JSON] runner --> telemetry[dotagent-telemetry
tracing + OTel] runner --> supervisor[dotagent-supervisor
process lifecycle] scheduler --> core state --> core telemetry --> core telemetry --> state plugin --> supervisor core --> notify[dotagent-notify
notifiers + ingress] notify --> secrets[dotagent-secrets
secrets.env loader] ``` Two edges surprise people: **`core → notify`, not the other way.** The manifest re-exports `NotifierEntry`, so the type crate depends on the driver crate. It also means `cargo publish` has to publish `notify` *before* `core`. **`notify → secrets`, and nothing depends on `secrets`.** It exists as its own crate precisely to avoid a `core → notify → core` cycle: the Telegram driver needs `${VAR}` resolution, and putting that loader in `core` would close the loop. | Crate | Owns | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `dotagent-core` | Shared types: `AgentManifest`, `Schedule`, `Heartbeat`, `WindowState`, `Config`, `AuditEvent`, `TriggerRequest`. | | `dotagent-scheduler` | **Pure** scheduling math. `compute_next_event`, `expected_at`, `should_retry`, `health_state`. No IO. | | `dotagent-runner` | Spawn the agent subprocess. Timeout, env injection, stdio capture, heartbeat lifecycle, hook firing. Also the pool of live processes for `[lifecycle] mode = "persistent"` agents. | | `dotagent-state` | Filesystem state: heartbeats, window state, audit log, plugin state, manifest cache. | | `dotagent-notify` | Built-in notifier drivers (`desktop`, `slack`, `ntfy`, `pushover`, `telegram`, `imessage`) **and** inbound Telegram transport. | | `dotagent-secrets` | Loader for `secrets.env`. Separate crate to keep `core → notify` acyclic. | | `dotagent-plugin` | `PluginClient` — discover + spawn + JSON-stdio for preflight / sink / third-party notify. | | `dotagent-mcp` | JSON-RPC 2.0 + Model Context Protocol wire types. No IO, no agent knowledge. | | `dotagent-supervisor` | Subprocess lifecycle: deadlines, kill-tree via POSIX process groups, live registry. | | `dotagent-telemetry` | `tracing` setup, JSON file logging, daily rotation, retention sweep, optional OTLP export. | | `dotagent-unit-gen` | Render `daemon.plist` (macOS) / `daemon.service` (Linux) from templates. | | `dotagent` (binary) | CLI subcommands plus daemon wiring for scheduling, Telegram ingress, the local UDS API, and the trigger gateway. | **Rule of thumb**: if you can write it as `fn f(now: DateTime, ...) -> X` with no `std::fs`, it goes in `dotagent-scheduler`. Anything that touches disk or processes goes in `dotagent-state`, `dotagent-runner`, or `dotagent-plugin`. *** ## Lifecycle of a single run The full path from "the daemon wakes up" to "your agent has run and the audit is written" — step by step. ```mermaid sequenceDiagram autonumber participant OS as OS scheduler participant D as daemon (tick loop) participant SCH as scheduler participant ST as state store participant PF as preflight plugin participant A as your agent participant SK as sink plugin participant N as notifier (in-process) participant AU as audit log OS->>D: spawn at boot (KeepAlive=true) D->>D: tick() D->>SCH: compute_next_event() SCH-->>D: ts = 2026-05-19T08:30 D->>D: sleep until min(ts, +30min) Note over D: SIGHUP → wake early & reload D->>SCH: expected_at(sched, now) - any window missed? SCH-->>D: yes — fire schedule X D->>PF: invoke (subprocess + JSON stdin) PF-->>D: {"ok": true} D->>ST: write heartbeat (started_at) D->>A: spawn with AGENT_* env vars + tmpdir A-->>D: stdout / stderr / exit code D->>ST: update heartbeat (finished_at, exit_code) D->>SK: invoke with stdout tail SK-->>D: {"ok": true} D->>N: fire (rate-limit check → POST to slack/etc.) D->>AU: append AgentRun, PluginInvoked × N D->>D: back to tick() ``` What's worth pointing out: * **Step 4 (sleep).** No polling. The daemon computes `ts` once and `tokio::time::sleep` until then. `MAX_SLEEP_MINUTES = 30` is the safety cap so a new manifest dropped into `agents/` is picked up within that window even if no scheduled event fires. The [daily summary](/guides/config-reference#daily_summary) is a third input to that `min()`: it is the one scheduled thing the daemon does that no agent schedule accounts for, so it schedules its own wake-up rather than relying on the cap to land inside its window. * **Step 7-9 (preflight).** If any preflight returns `ok=false`, the agent is **never spawned**. dotagent emits `PreflightFailed` to audit and fires the matching notifier. * **Step 10 (heartbeat start).** Written before the spawn. Crash detection: if `started_at` exists but `finished_at` doesn't, the previous run died. * **Step 11 (env injection).** dotagent sets nine `AGENT_*` env vars plus whatever `[env].extra` declares. See [`reference/env-vars.md`](/reference/env-vars). * **Step 14-15 (sink).** Fires only when the agent exited zero. `stdout_tail` (last 500 lines) is the payload. * **Step 16 (notifier).** Built-in drivers run in-process — no fork. Rate-limit state is read from `state/notify//.json`. *** ## Where each decision lives When debugging "why did X happen", you need to know which crate to grep. | Question | Where to look | | ------------------------------------------------------ | --------------------------------------------------------------------------------- | | When does a schedule fire next? | `dotagent-scheduler` — `compute_next_event` | | Is this window expected to have run already? | `dotagent-scheduler` — `expected_at` | | Should this failed window be retried? | `dotagent-scheduler` — `should_retry` | | Is this agent `ok` / `degraded` / `failing` / `stale`? | `dotagent-scheduler` — `health_state` | | How does the agent receive env vars? | `dotagent-runner` — `apply_env` | | When does the agent get killed for timeout? | `dotagent-runner` — `tokio::time::timeout` then `kill` + `SIGKILL` after 5s grace | | Where is the heartbeat written? | `dotagent-state` — `StateStore::write_heartbeat` | | How is `marker_regex` resolved? | The sink plugin itself (e.g., `plugins/sink-roam/src/main.rs`) | | How are plugins discovered? | `dotagent-plugin` — `PluginClient::resolve` | | Where does `dotagent install` write the unit? | `dotagent-unit-gen` — `launchd::generate_daemon` / `systemd::generate_daemon` | | What's logged where? | `dotagent-telemetry` — `init_from_default_config` | *** ## Crash isolation A single panicking plugin must not take down the daemon. Same for a runaway agent or a misconfigured notifier. The boundaries: ```mermaid flowchart LR subgraph daemon_proc["daemon process"] D[adaptive scheduler] N[notifiers] end subgraph agent_proc["agent process (one-shot subprocess)"] A[your script] end subgraph plugin_proc["plugin process (one-shot subprocess)"] P[plugin binary] end D --> A D --> P ``` | Failure | Containment | | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Agent script panics / segfaults | Subprocess dies. `exit_code` captured. Daemon writes the heartbeat and moves on. Retry policy kicks in. | | Persistent instance dies | The next request finds the dead pipe, discards the instance and spawns a replacement. A death in the window between setting the deadline and writing the request costs one retry, not the message. | | Agent hangs | `agent.timeout_seconds` triggers SIGTERM, then SIGKILL after 5s. `exit_code = 124`. | | Plugin panics | Subprocess dies. dotagent records `PluginInvoked { ok: false }` and continues. No retry of the plugin. | | Notifier driver fails (e.g., Slack 503) | The notifier call returns `Err` but the **run already happened**. Audit records the failure. The run is still considered successful. | | Daemon crashes | launchd `KeepAlive=true` / systemd `Restart=always` brings it back. Audit reconstructs state on startup. | | Audit log tampered | `verify_chain()` on startup catches it. Emits `AuditChainBroken` (which is itself a chained audit entry) and fires the configured `Critical`-severity notifier. A log that merely **rotated** (32MB → `audit.log.` + a hash seam) verifies clean, and a segment the operator deleted reads as "intact since ``" rather than broken — see [`security/threat-model.md`](/security/threat-model#what-the-hash-chain-guarantees-and-what-it-does-not). | The trade-off: **fork+exec per plugin (\~5-10ms)**. Plugins fire on discrete events (preflight, sink-on-success, third-party notify), never in hot loops, so the cost is invisible. The built-in notifiers were promoted out of the plugin protocol precisely because they fire *often* (every failure attempt) — see [`notifications.md`](/concepts/notifications) for the trade-off. *** ## State on disk dotagent is "stateless" in the sense that the daemon can be killed and restarted without losing any decision context — every consequential write is committed to disk first. The full layout is at [`reference/paths.md`](/reference/paths); the high-points: ``` ~/.config/dotagent/ # $DOTAGENT_HOME ├── agents/ # YOUR manifests (or symlinks) ├── plugins/ # YOUR custom plugin binaries ├── config.toml # optional global config ├── state/ │ ├── agents//.heartbeat.json # per-run lifecycle │ ├── windows/--.json # expected-vs-actual ledger │ ├── plugins//.json # plugin-owned state │ ├── notify//.json # built-in notifier rate-limit │ ├── known_manifests.json # sha256 cache for drift detection │ └── daemon.pid # for `dotagent reload` ├── logs/ │ ├── daemon/dotagent.log # structured JSON, daily rotation │ ├── daemon/run.avelino.dotagent.log # launchd/systemd stdout │ └── agents//.log # raw agent stdout+stderr ├── audit.log # append-only, hash-chained (live) └── audit.log.20260806T101500 # sealed segment, never deleted ``` Two important properties: 1. **`audit.log` is the source of truth for "did this happen".** It is append-only and hash-chained (`prev_hash` field). Past 32MB it rotates into a sealed `audit.log.` segment, stitched to the new file by a hash **seam** so the chain never has a gap — and no segment is ever deleted automatically. Operational logs are dense (debug-grade) and disposable; the audit log is sparse and forever. 2. **`known_manifests.json` is how drift / phantom agents are detected.** sha256 of every loaded manifest is cached. On the next load, mismatch → `ManifestDriftDetected`; new agent not in the cache → `PhantomAgentDetected`. Both are `Critical` severity (out-of-band notify). *** ## What the daemon does NOT do * **Run business logic.** Every domain concern (APIs to hit, prompts to draft, files to write) belongs in your agent script. dotagent's surface ends at "spawn the process, watch what came out." * **Sleep to wait for time.** Scheduling math is `compute_next_event` + `tokio::time::sleep`. No `loop { sleep(1s); check() }`. This is why a hundred schedules don't burn CPU. * **Embed AI / call LLMs.** dotagent has zero LLM dependencies. Your agent decides whether and when to invoke `claude -p`, `openai`, the `mcp` CLI, or nothing at all. * **Own conversation state.** The daemon routes and supervises a trigger, but `session_id`, transcripts, and LLM state belong to the agent process. The local API is a Unix socket, not a public HTTP/TCP conversation service. * **Replace the `mcp` CLI.** dotagent and `mcp` are independent projects. Agents that use Roam / Sentry / Grafana / etc. call `mcp` directly the same way they did before dotagent existed. dotagent *serves* MCP via `dotagent mcp` (agents as tools) but has no MCP client of its own — see [`../reference/mcp.md`](/reference/mcp). * **Sandbox the agent.** The `[security]` block in `agent.toml` is v0 schema-only — `doctor` reports inconsistency, but the runner doesn't yet enforce `allowed_commands` / `filesystem_writable` / network policy. Enforcement (sandbox-exec / bwrap / firejail) lands as a follow-up; see [`threat-model.md`](/security/threat-model). *** ## Related * [`agents.md`](/concepts/agents) — what an agent looks like from the author's side * [`plugins.md`](/concepts/plugins) — how the plugin protocol works * [`notifications.md`](/concepts/notifications) — why notifiers are NOT plugins * [`lifecycle.md`](/concepts/lifecycle) — agents kept alive between runs * [`triggers.md`](/concepts/triggers) — runs caused by an event, not the clock * [`../reference/local-api.md`](/reference/local-api) — local UDS transport and wire contract * [`telegram.md`](/concepts/telegram) — inbound chat * [`../reference/mcp.md`](/reference/mcp) — agents as MCP tools * [`../reference/cli.md`](/reference/cli) — every CLI subcommand * [`../reference/agent-spec.md`](/reference/agent-spec) — full manifest schema * [`../reference/paths.md`](/reference/paths) — filesystem layout * [`../security/threat-model.md`](/security/threat-model) — security posture # Agents > An **agent** is a small program that does, on a schedule, something a person would otherwise do — and uses AI when (and only when) judgment is actually required. This guide covers: 1. [What is an agent](#what-is-an-agent) — the idea, why it matters 2. [The dotagent definition](#the-dotagent-definition) — script controls, AI analyzes 3. [Anatomy of an agent](#anatomy-of-an-agent) — directory layout, files 4. [Writing your first agent](#writing-your-first-agent) — end-to-end walkthrough 5. [Patterns](#patterns) — common agent shapes 6. [Extending agents](#extending-agents) — env vars, plugins, lifecycle hooks 7. [Connecting agents](#connecting-agents) — chaining, shared state, handoff 8. [When NOT to write an agent](#when-not-to-write-an-agent) 9. [Examples gallery](#examples-gallery) — real-world inspiration For the formal `agent.toml` schema see [`agent-spec.md`](/reference/agent-spec). For plugins see [`plugins.md`](/concepts/plugins). For built-in notifiers see [`notifications.md`](/concepts/notifications). For the local assistant transport see [`local-api.md`](/reference/local-api). *** ## What is an agent The word *agent* got crowded in 2024–2025. It came to mean "an LLM that plans and executes." That definition is useful, but it's not what dotagent runs. **Here, an agent is something that automates a discrete piece of your day** — pulling a report, triaging an inbox, drafting a post, briefing you before standup. It might use an LLM. It might not. Either way, the agent is **the script that knows what to do and when**; the LLM is a component the script calls when judgment is needed. A useful working definition: > An agent is *the smallest unit of work you'd otherwise do yourself*, wrapped in a script, scheduled by a machine, and delivered to wherever you actually look (iMessage, Roam, email, file). Three properties make a piece of code "an agent" in this sense: 1. **Recurring.** It runs on a schedule (daily 08:30, every 90min, whenever inbox grows by N). One-shot scripts aren't agents. 2. **Bounded.** It does one thing well. "Daily DORA metrics for the eng team" is an agent. "Run my whole business" is not. 3. **Acting on your behalf.** The output is something you'd otherwise produce — a message, a document, a label, a decision. Not just data. dotagent is the substrate: it schedules, supervises, retries, notifies. **Writing the agent is your job.** dotagent makes that job small. *** ## The dotagent definition dotagent agents follow a pattern the legacy Fish framework distilled over a year of production use: > **The script controls the flow. AI is a tool the script calls.** A typical agent looks like: ```mermaid flowchart LR collect["collect
(CLIs, APIs)"] --> transform["format /
merge / sanitize"] transform --> ai["AI
(when needed)"] ai --> deliver["deliver
(notify, sink)"] ``` Concretely, the 5 phases (from the legacy `lib/agent.fish` framework): | Phase | What happens | | ------- | ---------------------------------------------------------- | | Init | Read env vars dotagent injected, create tmpdir | | Collect | Hit APIs/CLIs in parallel, dump results to tmpdir | | Prompt | Assemble prompt from collected data + static system prompt | | Run | Call `claude -p` (or omit if no AI needed) | | Output | Send to iMessage / Roam / file / stdout | **Why this and not "100% AI agent":** * **Cost.** When the LLM regenerates the same orchestration code every run, you pay tokens for work that's already deterministic. * **Speed.** A `gh api` + `jq` pipeline finishes in 200ms. An LLM doing the same finishes in 8s. * **Reliability.** Code is deterministic. LLM output isn't. * **Debuggability.** You can step through 80 lines of shell. You can't step through a model. So: AI when there's actual judgment ("which of these 30 stories is the most polemic?"). Code when there isn't ("fetch the top 30 stories from HN's Firebase API"). The skill is knowing the difference. *** ## Anatomy of an agent An agent is a directory. Everything it owns lives there. ``` my-agent/ agent.toml # required — manifest (schema in docs/reference/agent-spec.md) agent.fish # required — the entry point (or .py, .go, a binary, etc.) prompt.md # optional — system prompt for the LLM (if using one) config.json # optional — static data (mappings, allow-lists) CLAUDE.md # optional — doc-for-LLMs explaining the agent README.md # optional — doc-for-humans ``` The entry point is what `[run].command` points to in the manifest. It can be any executable: ```toml # Fish [run] command = "fish" args = ["./agent.fish"] # Python (uv-managed) [run] command = "uv" args = ["run", "./agent.py"] # Go binary [run] command = "./agent" # Rust binary (built separately) [run] command = "./target/release/my-agent" # Bash one-liner directly [run] command = "/bin/sh" args = ["-c", "curl -sf --max-time 20 https://example.com/data | jq . > $AGENT_TMPDIR/data.json"] ``` ### Discovery dotagent finds agents by scanning, in order: 1. Every directory in `$DOTAGENT_ROOT` (colon-separated) 2. `~/.config/dotagent/agents/` 3. `$CWD/agents/` 4. `$CWD` Each direct subdirectory that contains an `agent.toml` becomes an agent. The most common production setup is to keep agents in their dev repo (say `~/dotfiles/agents//`) and symlink each one into `~/.config/dotagent/agents/`: ```bash ln -s ~/dotfiles/agents/finops-weekly ~/.config/dotagent/agents/finops-weekly ``` That way `git` versions the agent and dotagent picks it up. *** ## Writing your first agent Goal: a 10-line agent that pulls the top 5 Hacker News stories every morning and writes them to a file. No LLM yet — just to see the moving parts. ### 1. Create the directory ```bash mkdir -p ~/.config/dotagent/agents/hn-morning cd ~/.config/dotagent/agents/hn-morning ``` ### 2. Write the script ```bash cat > agent.fish <<'EOF' #!/usr/bin/env fish # dotagent injects: # $AGENT_TMPDIR — scratch dir, auto-cleaned # $AGENT_HOME — this directory # $AGENT_DRY_RUN — "true" / "false" # --max-time on every call: `timeout_seconds` in the manifest is the # backstop that kills a wedged run, not the thing that keeps one call # from eating the whole budget. set -l stories (curl -sf --max-time 20 'https://hacker-news.firebaseio.com/v0/topstories.json' \ | jq -r '.[:5][]') for id in $stories curl -sf --max-time 20 "https://hacker-news.firebaseio.com/v0/item/$id.json" \ | jq -r '"- \(.title) (\(.score) points)\n \(.url // "n/a")"' end EOF chmod +x agent.fish ``` ### 3. Write the manifest ```bash cat > agent.toml <<'EOF' [agent] name = "hn-morning" description = "Top 5 HN stories at 08:00" timeout_seconds = 60 [run] command = "fish" args = ["./agent.fish"] [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] hours = [8] minute = 0 # Persist the agent's stdout to a file we can grep later. [[on_success]] plugin = "sink-file" config = { path = "/Users/me/reports/hn-today.md", mode = "overwrite" } EOF ``` ### 4. Smoke-test ```bash dotagent doctor # validate dotagent run hn-morning --schedule daily # run once, foreground cat /Users/me/reports/hn-today.md # see the output ``` If the manifest is right and `agent.fish` is executable, the file shows up populated. ### 5. Install the daemon ```bash dotagent install # writes the single plist launchctl bootstrap "gui/$(id -u)" \ ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` From now on, every weekday at 08:00, dotagent wakes, fires `agent.fish`, captures stdout, hands it to `sink-file`, writes the heartbeat, and goes back to sleep. **The whole orchestration is in `agent.toml`. The agent is 6 lines of fish.** *** ## Patterns Most useful agents fit one of these shapes. Pick the one closest to your need and copy. ### Pattern 1 — Pure data collector No LLM. Just normalize and persist. ```mermaid flowchart LR apis[APIs] --> jq["jq filter"] --> sink["sink-file / sink-roam"] ``` **Examples:** notes-to-blog, calendar-to-webhook. **When to pick this:** the output is structured data, not prose. You know exactly what fields you need. ### Pattern 2 — Brief / digest Collector + LLM summary + push to humans. ```mermaid flowchart LR sources["Sentry + Slack"] --> compact --> claude["claude haiku"] --> notify["imessage notifier"] ``` **Examples:** hourly-briefing (hourly CTO briefing), team-standup. Runnable in this repo: [`examples/hn-digest/`](https://github.com/avelino/dotagent/tree/main/examples/hn-digest/README.md). **When to pick this:** the input is noisy (50 issues, 10 Slack channels), you want 3 bullets. Cheap model (haiku) handles it. **Before you write one:** [`guides/llm-agents.md`](/guides/llm-agents) covers the headless gotchas (`claude -p` doesn't inherit your MCP config, there's no human to approve a tool call) and the failure modes that don't look like failures (empty output with a zero exit). ### Pattern 3 — Triage / classifier Collector + heuristic (no LLM) + LLM for residue + apply labels/actions. ```mermaid flowchart LR inbox["Gmail inbox"] --> heuristic["heuristic:
header / sender match"] inbox --> residue["residue"] heuristic --> apply1["batchModify labels"] residue --> haiku["claude haiku
(JSON)"] haiku --> apply2["batchModify labels"] ``` **Examples:** inbox-triage. **When to pick this:** 60-80% of cases are deterministic. Pay tokens only for the hard 20%. ### Pattern 4 — Generator with idempotency Collector + LLM generation + idempotent replace in target. ```mermaid flowchart LR sources["HN + Reddit + Lobsters"] --> claude --> roam["sink-roam
(replace block)"] ``` **Examples:** post-draft, meeting-notes-sync. **When to pick this:** output is drafted prose. Re-running shouldn't duplicate — use a sink that supports `marker_regex` (sink-roam) or overwrite mode (sink-file). ### Pattern 5 — Watchdog / preflight chain Mostly preflight plugins. Agent itself is a thin shell. ```mermaid flowchart LR warp[preflight-warp] --> auth["preflight-cmd
(auth check)"] --> collect --> rest["…"] ``` **Examples:** any agent that depends on a flaky connection (WARP, VPN, authenticated CLI). **When to pick this:** the failure mode is "external dep down" more than "my logic broke." Let preflight gate the run and the daemon retry until WARP comes back. *** ## Extending agents ### Environment variables dotagent injects When dotagent invokes your agent, these are set in addition to the inherited environment (unless `env.inherit = false` in the manifest): | Variable | Value | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AGENT_NAME` | manifest `agent.name` | | `AGENT_HOME` | absolute path to the manifest directory | | `AGENT_TMPDIR` | freshly created tempdir, auto-cleaned | | `AGENT_DRY_RUN` | `"true"` / `"false"` | | `AGENT_SCHEDULE_ID` | which schedule is firing | | `AGENT_SLUG` | derived slug for the heartbeat (from args) | | `AGENT_START_EPOCH` | epoch seconds of `started_at` | | `AGENT_ARGV` | JSON array of the schedule's `args` | | `AGENT_TRIGGER_*` | only on one-shot [triggered](/concepts/triggers) runs — source, actor, reply handle, payload. **Not set** for a [persistent](/concepts/lifecycle) agent. | | `AGENT_SESSION_ID` | opaque conversation id for a one-shot triggered run, when present. Persistent agents receive it as `trigger.session_id` per request instead. | | `AGENT_ASSISTANT_*` | only on triggered runs of an agent with `[assistant]`: session pointer, toolkit config + hash, memory recall block, plus a one-shot automatic-retirement marker — see [agent-spec](/reference/agent-spec#assistant-conversational-harness-opt-in) | | `AGENT_LIFECYCLE` | `persistent` when the agent stays alive between runs; absent otherwise | | `AGENT_PERSIST_KEY` | which slice a persistent instance answers for | | `AGENT_HEARTBEAT_FILE` | path to the heartbeat file (empty if `dry_run`) | | `LANG` | a UTF-8 locale, **only** when neither `LANG` nor `LC_ALL` was inherited — see [agent-spec](/reference/agent-spec#lang) | Use them. Don't reinvent (no need for your own tempdir, no need to write your own state). ### Triggered assistant runs Telegram and the local Unix-socket API can send a message to the configured dispatcher agent. The daemon is the harness at this boundary: it admits and orders the trigger, supervises the process, and delivers stdout. It does not hold the conversation transcript or an LLM session. For one-shot triggered runs, `AGENT_SESSION_ID` is an opaque key passed to the agent; persistent agents receive the per-request key as `trigger.session_id`. The agent owns any transcript or durable conversation state — unless the manifest opts into the `[assistant]` harness, which moves the *pointers* (not the transcripts) into the daemon: recorded model session ids, toolkit hashes and transcript sizes, reinjected as `AGENT_ASSISTANT_*` on the next trigger, with `MEMO:` capture lines stripped from replies into the memory workspace. Schema in [agent-spec](/reference/agent-spec#assistant-conversational-harness-opt-in). When the daemon retires a transcript that passed the configured ceiling, the first next trigger also receives `AGENT_ASSISTANT_CONTEXT_RETIRED=true`. It is absent on later triggers and after a manual `/novo` reset, so agents can avoid claiming continuity they no longer have. An agent that wants structured streaming declares `[run] protocol = "assistant-v1"` and emits JSON Lines `delta`, `reply`, and optional `session` frames. Local clients receive raw stdout lines as `reply.delta` followed by the shaped final `reply`; Telegram delivers only the final reply. The `claude --include-partial-messages` flag is an example detail, not a dotagent requirement. See the [Local Client API](/reference/local-api). ### Extra env vars Need a constant in scope of your agent? ```toml [env] inherit = true # default true; keep parent env extra = { LOG_LEVEL = "info", PYTHONUNBUFFERED = "1" } ``` ### Lifecycle hooks via plugins and notifiers ```toml [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#DAILY.*2026-05-19" } # Notifications are built into the daemon — declare them under [[notifiers]]. # See docs/concepts/notifications.md for every driver. [[notifiers]] driver = "imessage" to = "+55..." rate_limit_minutes = 60 events = ["given_up"] # only fire after retries exhausted ``` The agent's exit code drives this: * exit 0 → `on_success` plugins + matching `[[notifiers]]` fire * exit ≠ 0 → matching `[[notifiers]]` + any legacy `on_failure` plugins fire This is the right place to put cross-cutting concerns — when every agent notifies the same way, that boilerplate doesn't belong in each `agent.fish`. ### Retry policy Set per-agent or per-schedule: ```toml [defaults] max_retries = 3 retry_backoff_minutes = [5, 15, 30] stale_after_minutes = 120 ``` `stale_after_minutes` matters: if the window passed N minutes ago and the data isn't useful anymore (a 9am briefing at 2pm isn't a briefing), the daemon skips retrying silently. ### When your agent legitimately should "do everything" Sometimes you want one agent that's actually a small pipeline: collect, transform, run model, write output. That's fine — `agent.fish` can be 200 lines. The dotagent boundary is at the *orchestration* edge: scheduling, supervising, notifying, sinking. Below that is your shell / Python / Go problem. *** ## Connecting agents There are four ways to make two agents talk to each other. Pick the simplest one that works. ### 1. Filesystem handoff (preferred) Agent A writes a file. Agent B reads it on its next run. ```toml # Agent A — writes [[on_success]] plugin = "sink-file" config = { path = "/tmp/dotagent/handoff/a.json" } # Agent B — reads [run] command = "fish" args = ["./agent.fish"] # inside agent.fish: cat /tmp/dotagent/handoff/a.json | jq … ``` **Why preferred:** zero coupling. A doesn't know about B. B is robust to A not having run yet (empty file or missing file). ### 2. Shared Roam page Both agents write to / read from the same Roam page using `sink-roam` + the Roam MCP tools. Block markers (`marker_regex`) keep them from clobbering each other. ```toml # Agent A [[on_success]] plugin = "sink-roam" config = { page = "acme/tech/daily", marker_regex = "#a-output" } # Agent B [[on_success]] plugin = "sink-roam" config = { page = "acme/tech/daily", marker_regex = "#b-output" } ``` Good when the page is the "shared dashboard" the human reads. Roam becomes the bus, not the storage. ### 3. Direct invocation (rare) Agent A's script literally calls `dotagent run-now agent-b`. ```fish # Inside agent-a/agent.fish if test "$something" = condition dotagent run-now agent-b --schedule daily end ``` **Use sparingly.** It couples A to B and bypasses B's own schedule logic. Right when the trigger is genuinely event-driven, not time-driven — "after every inbox-triage success, run the unsubscribe sweep if it flagged anything." ### 4. Schedule alignment Two agents that should be coherent in time share schedules: ```toml # agent-a/agent.toml [[schedules]] id = "morning" type = "cron" hours = [8] minute = 0 # agent-b/agent.toml — runs 5 minutes after a, so a's output is fresh [[schedules]] id = "morning" type = "cron" hours = [8] minute = 5 ``` Loose coupling. If A fails its 08:00 window, B still runs at 08:05 with yesterday's data. Often that's what you want — "give me *something* at 08:05" is better than "give me nothing because A broke." ### Anti-pattern — chain of LLMs Don't build a sequence of 4 agents that each prompt Claude with the previous one's output. You're paying tokens to deserialize and re-serialize the same information. Either: * Collapse into one agent with a multi-step prompt. * Or share intermediate state via filesystem (option 1 above). *** ## When NOT to write an agent dotagent is a hammer. Not everything is a nail. * **One-off scripts** → just run them. `cron` or a shell alias is fine. * **Long-running services** → use a real service manager. dotagent agents are spawn-and-die. * **Interactive workflows** → if you need to *answer* a prompt, that's not an agent. That's a CLI you run. * **Sub-second latency** → dotagent's daemon wakes adaptively; spawn overhead is real. Don't schedule something that needs to fire 100x/s. * **Public event-driven work** → if the trigger is "a webhook hits an HTTP endpoint," you want a real service. dotagent's event transports are the user-local Unix socket and the configured Telegram poller; it does not expose a public HTTP/TCP endpoint. A good rule of thumb: if the task naturally fits in a sentence like "every weekday at 08:30, …" or "every 90 minutes, …", it's an agent. Otherwise it's something else. *** ## Examples gallery A realistic set, once you have a handful of agents running side by side. Every row is one of the five patterns above, wired to a different source: | Agent | Pattern | Notes | | --------------------- | ---------------------- | ----------------------------------------------------------------------- | | `hourly-briefing` | Brief / digest | Sentry + Slack → haiku → iMessage. 14 runs/day weekday. | | `team-standup` | Brief / digest | GitHub (30+ repos) + Sentry → sonnet → Roam. WARP preflight, retry 20x. | | `finops-weekly` | Pure collector | AWS multi-account → sonnet → file. Weekly. | | `calendar-to-webhook` | Connector | gcal → HTTP webhook. No LLM. | | `meeting-notes-sync` | Generator + idempotent | Mailbox (meeting recaps) → sonnet → Roam. Every 4h. | | `inbox-triage` | Triage / classifier | Gmail → heuristic + haiku → labels. Every 90 min. | | `post-draft` | Generator + idempotent | HN + Reddit + Lobsters → claude → Roam draft. Weekday 7am. | | `notes-to-blog` | Pure collector | Roam → static-site blog. 10am + 22h. | | `backup-offsite` | Watchdog / preflight | VPN preflight, then rsync. Nightly, retry until the link is up. | The point of the table is the middle column: pick the pattern first, then decide which source and sink it hangs off. For the **simplest possible agent**, see [`examples/hello-fish/`](https://github.com/avelino/dotagent/tree/main/examples/hello-fish/README.md) in this repo (and the parallel `hello-python/`, `hello-go/`, `hello-rust/`). For a **complete agent that calls a model**, see [`examples/hn-digest/`](https://github.com/avelino/dotagent/tree/main/examples/hn-digest/README.md) — collect with `curl`, one `claude -p` call for the judgment, deterministic render into a sink. For an agent that **stays alive between runs**, see [`examples/hello-persistent/`](https://github.com/avelino/dotagent/tree/main/examples/hello-persistent/README.md) — the whole JSON-lines protocol in about twenty lines of bash. Why you would want that is in [`lifecycle.md`](/concepts/lifecycle). *** ## Related docs * [`agent-spec.md`](/reference/agent-spec) — formal `agent.toml` schema * [`plugins.md`](/concepts/plugins) — what plugins are and how to use/build them * [`notifications.md`](/concepts/notifications) — built-in notifier drivers * [`plugin-protocol.md`](/reference/plugin-protocol) — protocol spec for plugins * [`lifecycle.md`](/concepts/lifecycle) — agents kept alive between runs * [`persistent-protocol.md`](/reference/persistent-protocol) — their wire format * [`migrating-from-fish.md`](/guides/migrating-from-fish) — for users of the legacy framework * [`threat-model.md`](/security/threat-model) — security considerations # Notifications dotagent ships with **built-in notification drivers** baked into the daemon. No plugin protocol, no subprocess fork, no extra binary on `$PATH`. The most common path (notify on failure) is the cheapest. > **What changed.** Earlier versions shipped five `dotagent-plugin-notify-*` binaries (desktop, imessage, slack, ntfy, pushover). Each notification forked a process and spoke JSON over stdio. That worked but cost \~5-20ms per fire and forced users to keep five extra binaries on `$PATH`. Notifications are now in-process. The plugin protocol stays alive for `sink` / `preflight` and third-party notifiers (`driver = "plugin"`). ## Shape Declare notifiers as a top-level array on the manifest: ```toml [[notifiers]] driver = "desktop" title = "dotagent" sound = true events = ["attempt_failed", "given_up"] [[notifiers]] driver = "slack" webhook_url = "${SLACK_WEBHOOK_URL}" events = ["given_up", "recovered"] ``` `events` is optional. Empty (or absent) means "all events". The same entry shape is reused at daemon level by `[[daily_summary.notifiers]]` in `config.toml`, which decides where the end-of-day health roll-up goes — see [`[daily_summary]`](/guides/config-reference#daily_summary). The one difference: `events` is ignored there, since that list is already scoped to a single event. > **Credentials go in `${VAR}`, not in the manifest.** An `agent.toml` lives in a versioned repo; a webhook URL written literally into one is a webhook URL in git history forever. Every credential-bearing field on every HTTP driver accepts `${VAR}`, resolved at send time from `~/.config/dotagent/secrets.env` — see [Secrets and credentials](#secrets-and-credentials) below and [`secrets.md`](/concepts/secrets). | Event | When it fires | | ---------------- | ---------------------------------------------------- | | `attempt_failed` | The agent exited non-zero (a retry may still happen) | | `timed_out` | The agent exceeded `agent.timeout_seconds` | | `given_up` | All retries exhausted — operator action expected | | `stale` | The schedule stopped running at all (see below) | | `recovered` | A previously-failing schedule passed | | `success` | Every successful run (use sparingly) | | `preflight` | A preflight plugin blocked the run | ### `stale`, and why alerts repeat Every other event fires from something that *happened*. An agent that quietly stops being scheduled does nothing at all: no run, no failure, no event. Its window ages past `stale_after_minutes`, the daemon stops even attempting it, and the last thing you heard was a success weeks ago. Silence reads as fine. `stale` fires from the *condition* instead of an event, on every daemon tick while it holds. The same applies to `given_up`, which used to be said once and never again — an agent broken for a week keeps asking. Because a condition is true continuously, dotagent spaces re-notifications on a rising ladder: **on entry, then after 1h, 6h, and once a day** for as long as it lasts. The state survives daemon restarts (`state/notify/alerts.json`) and is forgotten the moment the schedule succeeds again — so the next failure is loud from its first second instead of inheriting yesterday's silence. You do **not** have to add `"stale"` to an existing `events` list. A notifier that asked for `given_up` asked to be told the agent is broken, and stale is the same news only worse, so it is delivered on the `given_up` channel when nothing lists `stale` explicitly. Listing it makes the routing explicit. ## Drivers | `driver` | Transport | Subprocess? | | ---------- | ----------------------------------------------------- | -------------------------- | | `desktop` | `NSUserNotification` (macOS) / D-Bus (Linux) | No (native FFI) | | `slack` | HTTPS POST to Slack Incoming Webhooks | No (in-process reqwest) | | `ntfy` | HTTPS POST to ntfy.sh (or self-hosted) | No (in-process reqwest) | | `pushover` | HTTPS POST to api.pushover.net | No (in-process reqwest) | | `telegram` | HTTPS POST to api.telegram.org (Bot API) | No (in-process reqwest) | | `imessage` | `osascript` Messages.app automation | **Yes** — Apple has no API | | `plugin` | Falls back to the plugin protocol (`kind = "notify"`) | Yes (legacy escape hatch) | ### `desktop` ```toml [[notifiers]] driver = "desktop" title = "dotagent" # default: agent name subtitle = "free space low" # macOS only sound = true # macOS only urgency = "critical" # Linux only: low | normal | critical icon = "dialog-warning" # Linux only: icon name or absolute path expire_ms = 5000 # Linux only: 0 = persistent ``` ### `slack` ```toml [[notifiers]] driver = "slack" webhook_url = "${SLACK_WEBHOOK_URL}" # the URL *is* the credential channel = "#alerts" # optional username = "dotagent" # optional icon_emoji = ":robot_face:" # optional ``` ### `ntfy` ```toml [[notifiers]] driver = "ntfy" topic = "${NTFY_TOPIC}" # on public ntfy.sh the topic name is the secret base_url = "https://ntfy.sh" # default; set to self-hosted URL if needed token = "${NTFY_TOKEN}" # optional bearer auth priority = 4 # 1..5 title = "disk-alert" # default: agent name tags = ["warning", "skull"] ``` `title` and `tags` travel as the `X-Title` / `X-Tags` **HTTP headers**, and a header value is bytes, not text. Raw UTF-8 there is `obs-text` — RFC 9110 says a sender should not generate it, and ntfy's own docs warn it can arrive as `?`. So anything that is not plain visible ASCII is emitted as an [RFC 2047](https://datatracker.ietf.org/doc/html/rfc2047) `=?UTF-8?Q?…?=` encoded word, which the ntfy server decodes before reading the header. You write `title = "Falha na execução 🚨"`; it survives the wire intact. Control characters are flattened to spaces first — a newline is not a header value at all, and `HeaderValue` would reject the whole request over it. ### `pushover` ```toml [[notifiers]] driver = "pushover" token = "${PUSHOVER_TOKEN}" user = "${PUSHOVER_USER}" priority = 1 title = "disk-alert" # default: agent name ``` ### `telegram` ```toml [[notifiers]] driver = "telegram" bot_token = "${TELEGRAM_BOT_TOKEN}" # env interpolation; failsafe if unset chat_id = "-1001234567890" # or "@my_channel" parse_mode = "MarkdownV2" # optional: MarkdownV2 | HTML | Markdown disable_notification = false # optional: silent send ``` * `bot_token` accepts `${VAR}` references — resolution happens at send time against the daemon-loaded secrets file (`~/.config/dotagent/secrets.env`), falling back to the process env. `Debug` redacts the token explicitly. A literal token also works but committing it to the manifest is **not** recommended. See [secrets concept](/concepts/secrets) for the loader's posture (0600-enforced, never echoed, audit-logged by key count only). * `chat_id` is **not** `${VAR}`-expanded. It is an address, not a credential — write it literally. * When `parse_mode = "MarkdownV2"`, dotagent escapes the **19** characters Telegram reserves (``\_*[]()~`>#+-=|{}.!``) — the backslash included, since an unescaped one is itself an escape opener that desynchronizes everything after it. The body **always** goes through the escaper, so pre-escaping it produces doubled backslashes rather than formatting. If you want live markup, build the message with `parse_mode = "HTML"` instead, or leave `parse_mode` unset for plain text. * Escaping runs **before** the length cut, because escaping grows the text: a body that fits under 4096 characters plain can exceed it once every `.` and `-` has a backslash in front. If the cut lands between a backslash and the character it escapes, the orphaned backslash is dropped — Telegram rejects the entire message over one dangling escape. * When the Bot API refuses a send, the log line carries the API's own `description` (`"Bad Request: message is too long"`) rather than a bare status code, with the bot token scrubbed out of it first. * Outbound only, by default. Receiving Telegram updates is a separate, explicitly-enabled ingress — see [`telegram.md`](/concepts/telegram). ### `imessage` (macOS only) ```toml [[notifiers]] driver = "imessage" to = "+5511999999999" # phone or email iMessage handle rate_limit_minutes = 60 # skip-if-recent; 0 disables ``` > Apple does not expose any public API to send iMessages. This driver spawns `osascript` per send — it is the **only** built-in driver that forks. Rate-limit state lives at `$DOTAGENT_HOME/state/notify/imessage/.json`. ### `plugin` (escape hatch) For third-party notifiers (Discord, Teams, custom relays), use the legacy plugin protocol: ```toml [[notifiers]] driver = "plugin" name = "notify-discord" events = ["given_up"] [notifiers.config] webhook_url = "https://discord.com/api/webhooks/..." ``` The binary `dotagent-plugin-notify-discord` is resolved via `$DOTAGENT_PLUGIN_PATH` and the standard discovery order (see [`docs/reference/plugin-protocol.md`](/reference/plugin-protocol)). ## Message size limits Every backend rejects the **whole request** when the body is over its limit, so an alert that is too long is an alert that never arrives. dotagent trims instead, turning "silently undelivered" into "delivered, minus the tail". The cut is marked: `[truncated]` on bodies, `…` on titles. | Driver | Body limit | Title limit | | ---------- | ----------------- | -------------- | | `slack` | 40,000 characters | — | | `telegram` | 4,096 characters | — | | `pushover` | 1,024 characters | 250 characters | | `ntfy` | 4,096 **bytes** | 250 **bytes** | | `desktop` | (OS-dependent) | (OS-dependent) | | `imessage` | (none applied) | — | Characters versus bytes is not a pedantic distinction here. ntfy counts bytes, and pt-BR alert text is not ASCII: `ç` and `ã` cost 2 bytes each, an emoji costs 4. A character-based trim can hand ntfy four times its limit while believing it stayed under. The byte cutter also walks the cut back to a UTF-8 character boundary, so a truncated message may land a few bytes below the cap — under is correct, over is a rejected request. ntfy's title cap applies to the **raw** text, before RFC 2047 encoding, because encoding multiplies: a 4-byte emoji becomes 12 characters of `=XX`, and the cap has to bite before the encoder runs rather than after. ## Empty bodies The mirror image of the size limits: too *little* body is also a rejected request. Telegram answers an empty `sendMessage` with `400 Bad Request: message text is empty`, and the drivers that accept it deliver a blank line. What dotagent does depends on why the body is empty, and the two cases pull in opposite directions: | Event | Empty body | Why | | --------------- | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `success` | nothing is sent | The agent had nothing to report. A sweeper that finds no follow-ups is *working* — the run is already in `dotagent status` and in the agent's log. | | everything else | body synthesized as `agent/schedule: event (no output)` | The state change **is** the news. Losing a `given_up` because the process died too fast to print anything is the worst outcome available. | A skipped `success` is logged at debug level, not warn — it is the expected outcome for an agent that reports by exception. The decision happens before any driver runs, so there is no `plugin_invoked` audit entry either: nothing was invoked. This is a floor, not a substitute for a good message. An agent that always prints something useful never reaches either branch. ## Secrets and credentials Every credential-bearing field accepts `${VAR}`, resolved at send time against `~/.config/dotagent/secrets.env` first and the process environment second: | Driver | `${VAR}`-expanded fields | Left literal | | ---------- | ---------------------------- | ------------ | | `slack` | `webhook_url` | — | | `ntfy` | `token`, `base_url`, `topic` | — | | `pushover` | `token`, `user` | — | | `telegram` | `bot_token` | `chat_id` | | `imessage` | — | `to` | `chat_id` and `to` are addresses. Routing them through a secrets resolver would turn a typo into "env var unset" instead of a message delivered to the wrong place, which is the failure you actually want to see. An unresolved reference **fails the send** rather than falling back to the literal `"${SLACK_WEBHOOK_URL}"` — sending that string is a request authenticated as a placeholder, which answers `404` and looks like an outage rather than a typo. The error names the field and the variable and nothing else. Credentials also never reach `tracing`. `reqwest::Error`'s `Display` appends `for url (…)`, and for Slack and Telegram that URL *is* the secret — a single `?` on an HTTP call used to write a live webhook into `~/.config/dotagent/logs/daemon/dotagent.log` from a failure as mundane as a DNS blip. `NotifyError` has no `From` conversion at all, so that `?` no longer compiles: transport failures are reduced to a kind plus a status (`slack transport error (timeout)`) at the call site, and the leak is unrepresentable rather than merely unwritten. See [`threat-model.md`](/security/threat-model). > **Network allow-list.** If you declare `[security] network = [...]`, include `"api.telegram.org"` so the (future) sandbox lets the bot reach the API. v0 is schema-only, so it's a no-op today, but the declaration documents intent. ## Tiered notifications pattern Combine drivers + `events` filters to keep noisy channels cheap and pager channels rare: ```toml # Cheap: desktop banner on every failure [[notifiers]] driver = "desktop" title = "disk-alert" events = ["attempt_failed", "given_up"] # Loud: iMessage only when retries are exhausted [[notifiers]] driver = "imessage" to = "+5511999999999" rate_limit_minutes = 60 events = ["given_up"] # Audit: Slack thread when something recovered after pain [[notifiers]] driver = "slack" webhook_url = "${SLACK_WEBHOOK_URL}" events = ["recovered"] ``` ## Failure semantics A notifier failing is **logged but does not fail the run** — the run already happened. Each invocation lands in the audit log (`$DOTAGENT_HOME/audit.log`) as a `plugin_invoked` event with `plugin = "notifier:"`. If a notifier rate-limits or dedups, it returns `Skipped { reason }` which is treated as a success outcome. ## Legacy `[[on_failure]]` / `[[on_success]]` The legacy plugin-style hooks still work — they always meant "fire these plugins on these events". They are now reserved for **sink-style** hooks (persist output, publish to Roam, etc.). For notifications, prefer `[[notifiers]]` — it's faster, has fewer moving parts, and ships with the daemon. Migration is a 1-to-1 rename: ```toml # Before [[on_failure]] plugin = "notify-desktop" config = { title = "x", sound = true } events = ["given_up"] # After [[notifiers]] driver = "desktop" title = "x" sound = true events = ["given_up"] ``` # Secrets dotagent reads a single secrets file at daemon startup so notifier configs (and, eventually, agent env injection) can reference `${VAR}` without the operator having to wire env vars into the launchd plist or systemd unit. > If you're here because a notifier config says `bot_token = "${TELEGRAM_BOT_TOKEN}"` and you're wondering where that variable should live — this is the page. ## Where the file lives ``` ~/.config/dotagent/secrets.env # default ``` Override (in order of precedence): 1. `[secrets] file = "..."` in `~/.config/dotagent/config.toml` 2. `DOTAGENT_SECRETS_FILE` env var (absolute path) 3. The default above The override is useful when the file is mounted from a secret manager (Vault, AWS Secrets Manager, sops-nix, kubernetes secret volume) into somewhere like `/run/secrets/dotagent.env`. ## File format ```env # ~/.config/dotagent/secrets.env # Comments start with `#`. Blank lines are fine. TELEGRAM_BOT_TOKEN=1234:abcdef... SLACK_WEBHOOK_URL="https://hooks.slack.com/services/..." PUSHOVER_TOKEN='literal token with $no expansion' export DIRENV_STYLE=ok # 1Password CLI reference — resolved at daemon startup TELEGRAM_BOT_TOKEN_2=op://Personal/dotagent/telegram-token ``` Roughly dotenv-compatible: * One `KEY=VALUE` per line. The first `=` is the separator; subsequent `=` characters are part of the value. * Lines may start with `export` — the prefix is stripped (handy for copy-paste from a `.envrc`). * **Double-quoted** values strip the quotes and process the escapes `\n`, `\r`, `\t`, `\\`, `\"`. Anything else after `\` is an error, on purpose, so typos don't pass silently. * **Single-quoted** values strip the quotes and are otherwise literal (no escapes, no `${VAR}`) — useful for tokens with backslashes or dollar signs. * **Unquoted** values: leading/trailing whitespace trimmed, interior preserved. No shell expansion: `$HOME` stays the literal string. * Keys must match `[A-Za-z_][A-Za-z0-9_]*`. ### Secret references (1Password) A value matching `op://vault/item[/section]/field` is recognised as a **1Password CLI reference**. At daemon startup the loader shells out to `op read --no-newline ` and stores the plaintext result in memory. The literal `op://…` string is **never** kept — if `op` fails, the key is removed from the store so any notifier needing it fails loud (`env var ${…} is unset`) instead of sending the placeholder over the wire. Requirements: * `op` (the 1Password CLI) on `PATH`. * An active 1Password session — desktop app biometric unlock or `op signin` before starting the daemon. The loader does not prompt. ```env TELEGRAM_BOT_TOKEN=op://Personal/dotagent/telegram-token SLACK_WEBHOOK_URL=op://Work/dotagent/slack-webhook-url ``` > **Why CLI, not the SDK?** Issue #34 explicitly framed remote secret managers as out-of-scope; the CLI sits in the same posture as populating the file from `vault read`, just with the lookup happening at startup instead of being a separate pre-step. The 1Password Service Account SDK could replace the fork later (issue not yet filed) but it requires a Business-plan service account token, which most laptop-dev users don't have. ### Other secret managers For anything that isn't `op`, **populate the file before starting the daemon**. The file is the single source of truth at startup time. ```bash # AWS Secrets Manager aws secretsmanager get-secret-value \ --secret-id dotagent/prod \ --query SecretString --output text \ > ~/.config/dotagent/secrets.env chmod 600 ~/.config/dotagent/secrets.env # HashiCorp Vault vault kv get -format=json secret/dotagent \ | jq -r '.data.data | to_entries[] | "\(.key)=\(.value)"' \ > ~/.config/dotagent/secrets.env chmod 600 ~/.config/dotagent/secrets.env # sops + age (decrypt at boot) sops -d secrets.enc.env > ~/.config/dotagent/secrets.env chmod 600 ~/.config/dotagent/secrets.env ``` Wire it into your launchd plist / systemd unit's `ExecStartPre` and you get the same posture as Kubernetes secret volumes: a normal file on disk, refreshed by an out-of-band tool. ## Permission posture The file **must** be mode `0600` (read/write owner-only). Any group or world bits and the daemon refuses to load it — same posture as `ssh` refusing to use a world-readable private key. ```bash chmod 600 ~/.config/dotagent/secrets.env ``` Refusal is non-fatal: the daemon still starts so scheduled runs keep firing. But notifiers that depended on the rejected file will fail loudly (`env var ${TELEGRAM_BOT_TOKEN} is unset`) — there is no silent fall-through to "send the literal placeholder". Run `dotagent doctor` to see the file's state in one place. ## How values get used At send time, every notifier config string that contains `${VAR}` walks through this resolver: 1. **Secrets store** — the in-memory copy of the file loaded at daemon startup. 2. **`std::env::var`** — fallback for operators who already wired the variable into the plist / systemd unit. If both miss, the notifier fails with a clear error rather than sending a request authenticated as the literal string `${…}`. ```mermaid flowchart LR A["slack webhook_url = \"${SLACK_WEBHOOK_URL}\""] --> B{secrets store has key?} B -- yes --> C[use value from secrets.env] B -- no --> D{std::env::var has key?} D -- yes --> E[use value from process env] D -- no --> F[fail: env var unset] ``` ### Which fields honor `${VAR}` Every credential-bearing field on every built-in HTTP driver: | Field | Why it is a credential | | -------------------- | ------------------------------------------------------------------------------------------------------------------ | | `slack.webhook_url` | The URL **is** the credential — anyone holding it can post. | | `ntfy.token` | Bearer token for authenticated topics. | | `ntfy.base_url` | A self-hosted URL can legitimately embed HTTP basic credentials. | | `ntfy.topic` | On public `ntfy.sh` the topic name is the only thing standing between your alert stream and anyone who guesses it. | | `pushover.token` | Application token. | | `pushover.user` | User/group key. | | `telegram.bot_token` | Full control of the bot. | Two fields are deliberately **not** expanded: `telegram.chat_id` and `imessage.to`. They are addresses, not secrets — a phone number or a channel id identifies *where* the alert goes, and putting them through a resolver would turn a typo into a confusing "env var unset" instead of a message that lands in the wrong place. Keep them literal. Expansion happens at **send time**, not at load time, so rotating `secrets.env` and sending `SIGHUP` is enough — no manifest edit. ### What an unresolved reference does It fails the send. There is no fallback to the literal `"${SLACK_WEBHOOK_URL}"`: that string is not a degraded credential, it is a request authenticated as a placeholder, which answers `404` and reads like an outage rather than a typo. The error names the **field** and the **variable** and nothing else — `slack.webhook_url: env var ${SLACK_WEBHOOK_URL} is unset`. It never carries a resolved value, because the input is a credential template and the store holds sibling credentials; echoing either would turn a config typo into a disclosure in `~/.config/dotagent/logs/`. ## Reload The file is read once at daemon startup. To pick up changes: * `dotagent reload` (sends `SIGHUP` — preferred), or * restart the unit (`launchctl kickstart` / `systemctl --user restart`). There is no automatic file-watch reload. The reasoning is the same as for `sshd`: secrets rotation is rare, surprises are expensive, and a deliberate signal is a feature. ### What happens when reload fails If the file becomes unreadable between startup and SIGHUP (insecure mode, parse error, file deleted), the daemon **drops the previously loaded store** rather than keeping it as a fallback. Rationale: an operator who just ran `op signin && op item rotate ` would be worse off if dotagent kept serving the old token. Once dropped: * Notifier `${VAR}` lookups fall through to `std::env::var`. * Anything that has no env fallback fails loud with `env var ${…} is unset` rather than silently using a revoked credential. * The audit log records `secrets_refused` with reason ending in `; previous store dropped`, so the chain explains itself. Restore the file (fix permissions, re-decrypt, etc) and SIGHUP again to re-populate the store. ## What the audit log shows dotagent's append-only `audit.log` records the load outcome — never the values themselves. | Event | Severity | Payload | | ----------------- | -------- | ---------------------------------------------------------------------------------- | | `secrets_loaded` | Notice | `path`, `key_count`, `unresolved_references` | | `secrets_refused` | Critical | `path`, `reason` (e.g., "insecure permissions … mode 640; previous store dropped") | ### What never leaks vs. what may appear * **Values never appear** anywhere — not in `audit.log`, not in `tracing` output, not in `Debug` impls (`SecretsStore`'s `Debug` redacts to `len` + `source`). * **Key names may appear in operational `tracing` warnings** — duplicate-key warnings and notifier `env var ${KEY} is unset` errors both include the key name on purpose, because they're unactionable without it. Key names also appear in failed `op://...` warnings (along with the reference path), since the vault/item identifiers are operator-visible metadata, not the secret itself. * **The audit log itself never includes key names** — only counts. If you treat key names as sensitive, route the daemon's `tracing` output to a sink you trust (the daemon's `logs/daemon/*.log` inherits the umask of whoever started it — set `umask 077` in the launchd / systemd unit's environment if you want owner-only). ## Agent subprocess isolation The secrets file is read by the daemon **only**. Agent subprocesses do not get the whole file dumped into their env — they only see what their own manifest explicitly opts into. > v0 ships with notifier-side resolution working. The opt-in mechanism for `agent.toml` (`[env].from_secrets = ["KEY1", ...]`) will land in a follow-up issue. ## Non-goals * **GPG / age decryption.** Tracked separately. * **Remote secret managers** (Vault, AWS Secrets Manager, Doppler). Out of scope. Tools like [`vault read -format=table`](https://developer.hashicorp.com/vault/docs/commands/read) or [`aws secretsmanager get-secret-value`](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/get-secret-value.html) can populate the file from upstream. * **Automatic file-watch reload.** SIGHUP / restart only. ## Troubleshooting | Symptom | Fix | | ---------------------------------------------- | ---------------------------------------------------------------------------- | | `env var ${TELEGRAM_BOT_TOKEN} is unset` | Add the key to `secrets.env`, then `dotagent reload`. | | `dotagent doctor` flags `insecure permissions` | `chmod 600 ~/.config/dotagent/secrets.env`. | | `dotagent doctor` says `(not present)` | Create the file. Missing is OK — only an issue if a notifier needs `${VAR}`. | | Edited file, change didn't take effect | `dotagent reload`. The daemon caches the file in memory. | ## See also * [`docs/concepts/notifications.md`](/concepts/notifications) — notifier drivers and where `${VAR}` matters today. * [`docs/reference/agent-spec.md`](/reference/agent-spec) — manifest schema. * [`docs/security/threat-model.md`](/security/threat-model) — overall posture. # Plugins > Plugins are how dotagent talks to the outside world for **preflight checks**, **output sinks**, and **third-party notifiers**. Anything that isn't *scheduling the agent* and isn't a built-in notifier lives in a plugin. > **Looking for notifications?** The common notifiers (`desktop`, `imessage`, `slack`, `ntfy`, `pushover`, `telegram`) are now **built into the daemon** — they are not plugins. See [`notifications.md`](/concepts/notifications) for the `[[notifiers]]` shape. The plugin protocol below is still how you wire a *custom* notifier (Discord, Teams, etc.) via `driver = "plugin"`. This guide covers: 1. [What is a plugin](#what-is-a-plugin) 2. [Using a plugin](#using-a-plugin) — declaring, configuring, debugging 3. [Built-in plugins](#built-in-plugins) — first-party preflights and sinks 4. [Creating a plugin](#creating-a-plugin) — protocol, languages, examples 5. [Publishing a plugin](#publishing-a-plugin) — distribution, signing, brew 6. [Best practices](#best-practices) — single-purpose, secrets, idempotency 7. [Debugging](#debugging) — stderr, `dotagent plugin invoke`, audit log For the formal protocol spec (verbs, payload shapes, exit codes) see [`plugin-protocol.md`](/reference/plugin-protocol). *** ## What is a plugin A **dotagent plugin** is a separate binary that follows a tiny CLI protocol over JSON stdio. dotagent invokes it as a subprocess at specific lifecycle moments and acts on what comes back. ```mermaid sequenceDiagram participant M as agent.toml participant D as dotagent daemon participant P as dotagent-plugin-sink-roam M->>D: [[on_success]] plugin = "sink-roam" Note over D: agent exits 0
event = success D->>P: spawn subprocess
+ JSON payload on stdin Note over P: read stdin
do the thing (mcp roam_publish)
write stdout P-->>D: {"ok": true} on stdout
exit 0 Note over D: audit log:
plugin_invoked ``` ### Why subprocess + JSON instead of an SDK / dylib / WASM * **Any language.** Rust, Go, Python, Bash, Node — if it can read stdin and write JSON to stdout, it can be a plugin. * **Crash isolation.** A panicking plugin doesn't take down the daemon. * **Independent versioning.** Plugins ship and update on their own cadence. * **No ABI.** No `cdylib`, no `extern "C"`, no symbol versioning headaches. Trade-off: every invocation pays `fork+exec` (\~5–10ms). Plugins fire on discrete events (notification, preflight, sink-on-success), not in hot loops, so the cost is invisible. ### Three kinds of plugins | Kind | When dotagent invokes it | Decides what? | | ----------- | ------------------------------------------------------------- | ---------------------------------------- | | `preflight` | BEFORE spawning the agent | Can the agent run? (`ok=true` = proceed) | | `notify` | On failure events (`attempt_failed`, `given_up`, `recovered`) | Sends a human-facing message | | `sink` | On successful run | Persists the agent's output | A plugin can declare multiple `kinds` in its `info` response if it makes sense — but most plugins are single-kind for clarity. ### What a plugin is NOT * **Not a place for agent business logic.** Your agent already does that. * **Not a way to schedule other work.** Scheduling lives in `agent.toml`. * **Not a service.** No long-running process, no IPC, no daemon-to-daemon. Each invocation is a one-shot. *** ## Using a plugin ### 1. Declare it in your `agent.toml` ```toml [agent] name = "my-agent" [run] command = "fish" args = ["./agent.fish"] [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] hours = [8] minute = 30 # Preflight: abort the run if Cloudflare WARP isn't connected. [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } # On success: persist the output to Roam. [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#DORA.*2026-05-19" } # Notifications are built-in — declare them via [[notifiers]], not [[on_failure]]. # See docs/concepts/notifications.md. [[notifiers]] driver = "imessage" to = "+5511999999999" rate_limit_minutes = 60 events = ["given_up"] ``` **Top-level keys**: * `plugin` — the short name. dotagent resolves it to a binary called `dotagent-plugin-` (see [Discovery](#discovery-order) below). * `config` — opaque JSON forwarded to the plugin's `invoke` verb. The plugin's `info` response describes the schema. * `events` (optional, only on `on_success` / `on_failure`) — restrict firing to specific events. Empty / omitted = all events. Valid values: `attempt_failed`, `given_up`, `stale`, `recovered`, `timed_out`, `preflight`, `success`, `daily_summary`. ### 2. Validate at install time ```bash dotagent doctor ``` This iterates every manifest and: * Resolves every plugin reference to a binary path. * Calls `info` to confirm the plugin actually responds. * Reports `✗ plugin not found` if discovery failed. ### 3. List discovered plugins ```bash dotagent plugin list ``` Outputs a table of every plugin referenced by any manifest, with its version, kinds, and resolved path. ### 4. Invoke manually (debug) When a plugin misbehaves and you want to see exactly what's happening: ```bash echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "hello world", "config": { "page": "today", "marker_regex": "#test" } }' | dotagent-plugin-sink-roam invoke ``` Stdout = the JSON response. Stderr = the plugin's human-readable log. ### Discovery order When you write `plugin = "sink-roam"` in a manifest, dotagent looks for a binary called `dotagent-plugin-sink-roam` in this order: 1. Every directory in `$DOTAGENT_PLUGIN_PATH` (colon-separated) 2. `~/.config/dotagent/plugins/` 3. `/usr/local/lib/dotagent/plugins/` 4. Every directory in `$PATH` First match wins. The Homebrew formula drops every first-party plugin into the same `bin/` as `dotagent`, which means `$PATH` resolution covers the default install with zero config. To verify which binary will be used: ```bash dotagent plugin list # output: sink-roam 0.0.1 sink /opt/homebrew/bin/dotagent-plugin-sink-roam ``` ### The `events` filter `on_success` / `on_failure` entries can filter on event name. Events dotagent emits: | Event | When | | ---------------- | -------------------------------------------------------------------------------------------------- | | `success` | agent exit code 0 | | `attempt_failed` | agent exit ≠ 0 but more retries available | | `timed_out` | agent killed for exceeding `timeout_seconds` | | `given_up` | retries exhausted (`max_retries` reached), and repeated while it holds | | `stale` | the schedule stopped running at all — window aged past `stale_after_minutes` | | `recovered` | success on a window that had ≥1 previous failed attempt | | `preflight` | preflight plugin returned `ok=false` and the run was aborted | | `daily_summary` | the daemon's health summary at `[daily_summary].time` (default 22:45), or `dotagent daily-summary` | `daily_summary` is the one event a manifest cannot subscribe to. It belongs to the daemon, not to any agent, so it is delivered only to `[[daily_summary.notifiers]]` in [`config.toml`](/guides/config-reference#daily_summary) — and those entries ignore `events` entirely, since a list already scoped to one event can only be subtracted from. Writing `events = ["daily_summary"]` on a manifest entry matches nothing. **Example — fire a sink on every successful run** (no filter): ```toml [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#daily" } # `events` omitted → fires on every success ``` > Notifications follow the same event semantics but are configured under `[[notifiers]]` — see [`notifications.md`](/concepts/notifications). *** ## Built-in plugins These ship with the Homebrew install and live under [`plugins/`](/built-in-plugins/plugins) in this repo. | Plugin | Kind | Purpose | | ---------------- | --------- | -------------------------------------------------------------- | | `preflight-warp` | preflight | Checks `warp-cli status` reports "Connected". | | `preflight-cmd` | preflight | Generic: runs an arbitrary command, checks exit code + stdout. | | `sink-roam` | sink | Publishes hierarchical content to Roam Research via `mcp` CLI. | | `sink-outl` | sink | Publishes hierarchical content to Outl via `mcp outl_batch`. | | `sink-file` | sink | Writes the message to a file (overwrite or append). | > Notifications (`desktop`, `imessage`, `slack`, `ntfy`, `pushover`, `telegram`) are **not** plugins anymore — they ship as in-process drivers inside the daemon. See [`notifications.md`](/concepts/notifications). For per-plugin details (config schema, examples, troubleshooting) see [**`docs/plugins/`**](/built-in-plugins/plugins). ### Quick config reference ```toml # preflight-warp config = { connect_command = "warp-cli connect" } # preflight-cmd config = { command = "gh", args = ["auth", "status"], expect_exit = 0 } # sink-roam config = { page = "today", marker_regex = "#DORA.*2026-05-19" } # sink-outl config = { page = "today", marker_regex = "#DORA.*2026-05-19" } # sink-file config = { path = "/Users/me/reports/today.md", mode = "overwrite" } ``` Run `dotagent-plugin- info | jq .schema` for the full JSON schema of each plugin's `config`. *** ## Creating a plugin The full protocol spec is in [`plugin-protocol.md`](/reference/plugin-protocol). This section is the friendly quickstart. ### Anatomy of a plugin A plugin is **any executable** named `dotagent-plugin--` (e.g., `dotagent-plugin-notify-discord`) that accepts one of three verbs as its first positional argument: ``` dotagent-plugin- info # describe yourself dotagent-plugin- validate # check the config (stdin JSON) dotagent-plugin- invoke # do the thing (stdin JSON) ``` ### `info` — no stdin, prints metadata ```jsonc { "name": "notify-discord", "version": "0.1.0", "kinds": ["notify"], // "notify" | "preflight" | "sink" "platforms": ["darwin", "linux"], "schema": { // JSON Schema for `config` "type": "object", "required": ["webhook_url"], "properties": { "webhook_url": { "type": "string" }, "username": { "type": "string" } } } } ``` ### `validate` — stdin is the `config` object ```jsonc // stdin { "webhook_url": "https://discord.com/api/webhooks/..." } // stdout { "ok": true } // or on failure { "ok": false, "error": "webhook_url must start with https://" } ``` dotagent calls `validate` when loading manifests. It's how `dotagent doctor` catches typos before runtime. ### `invoke` — stdin is the full payload ```jsonc // stdin { "kind": "notify", // mirrors what the plugin advertised "agent": "finops-weekly", "schedule": "weekly", "event": "given_up", "message": "🚨 5-line tail of stderr ...", "config": { "webhook_url": "..." } } // stdout { "ok": true } ``` Stdout is REQUIRED to be valid JSON with at least `{"ok": bool}`. Extra fields are forwarded into dotagent's audit log and `--verbose` output. Stderr is for human-readable logs and is captured by the daemon (visible via `dotagent logs dotagent`). ### Minimal plugins, three languages #### Rust (single-file) ```rust // plugins/notify-discord/src/main.rs use std::io::Read; use serde_json::{json, Value}; fn main() -> anyhow::Result<()> { let verb = std::env::args().nth(1).ok_or_else(|| anyhow::anyhow!("missing verb"))?; match verb.as_str() { "info" => { println!(r#"{{"name":"notify-discord","version":"0.1.0","kinds":["notify"], "platforms":["darwin","linux"], "schema":{{"type":"object","required":["webhook_url"], "properties":{{"webhook_url":{{"type":"string"}}}}}}}}"#); } "validate" => { let v: Value = serde_json::from_reader(std::io::stdin())?; let ok = v.get("webhook_url").and_then(|s| s.as_str()).is_some_and(|s| s.starts_with("https://")); println!("{}", json!({"ok": ok})); } "invoke" => { let mut raw = String::new(); std::io::stdin().read_to_string(&mut raw)?; let v: Value = serde_json::from_str(&raw)?; let url = v.pointer("/config/webhook_url").and_then(|s| s.as_str()).unwrap_or(""); let msg = v.get("message").and_then(|s| s.as_str()).unwrap_or(""); let resp = ureq::post(url).send_json(json!({"content": msg})); println!("{}", json!({"ok": resp.is_ok()})); } _ => anyhow::bail!("unknown verb"), } Ok(()) } ``` #### Python ```python #!/usr/bin/env python3 """dotagent-plugin-notify-discord — Python implementation.""" import json import sys import urllib.request verb = sys.argv[1] if len(sys.argv) > 1 else "" if verb == "info": print( json.dumps( { "name": "notify-discord", "version": "0.1.0", "kinds": ["notify"], "platforms": ["darwin", "linux"], "schema": { "type": "object", "required": ["webhook_url"], "properties": {"webhook_url": {"type": "string"}}, }, } ) ) elif verb == "validate": cfg = json.load(sys.stdin) ok = cfg.get("webhook_url", "").startswith("https://") print(json.dumps({"ok": ok})) elif verb == "invoke": payload = json.load(sys.stdin) url = payload["config"]["webhook_url"] msg = payload.get("message", "") req = urllib.request.Request( url, data=json.dumps({"content": msg}).encode(), headers={"Content-Type": "application/json"}, ) try: urllib.request.urlopen(req, timeout=10) print(json.dumps({"ok": True})) except Exception as e: print(json.dumps({"ok": False, "error": str(e)})) sys.exit(1) else: sys.exit(2) ``` Drop this in `~/.config/dotagent/plugins/dotagent-plugin-notify-discord`, `chmod +x`, and dotagent finds it on the next `doctor`. #### Bash ```bash #!/usr/bin/env bash # dotagent-plugin-sink-tee — append message to a file AND echo to stdout set -euo pipefail case "${1:-}" in info) cat <<'JSON' { "name": "sink-tee", "version": "0.1.0", "kinds": ["sink"], "platforms": ["darwin", "linux"], "schema": { "type": "object", "required": ["path"], "properties": { "path": { "type": "string" } } } } JSON ;; validate) cfg="$(cat)" path=$(echo "$cfg" | jq -r '.path // empty') if [[ -z "$path" ]]; then echo '{"ok": false, "error": "path is required"}'; exit 0 fi echo '{"ok": true}' ;; invoke) payload="$(cat)" path=$(echo "$payload" | jq -r '.config.path') msg=$(echo "$payload" | jq -r '.message // ""') mkdir -p "$(dirname "$path")" printf '%s\n' "$msg" | tee -a "$path" >/dev/null echo '{"ok": true}' ;; *) echo "{\"ok\":false,\"error\":\"unknown verb\"}"; exit 2 ;; esac ``` ### Local testing ```bash # 1) info ./my-plugin info | jq . # 2) validate echo '{"webhook_url":"https://..."}' | ./my-plugin validate # 3) invoke (mimic dotagent's payload) echo '{ "kind": "notify", "agent": "test", "schedule": "test", "event": "given_up", "message": "hi", "config": {"webhook_url":"https://..."} }' | ./my-plugin invoke ``` If all three return valid JSON and exit 0 on the happy path, the plugin integrates. ### Walking through a build — Discord notify plugin (third-party) > Discord, Teams, and other custom notifiers don't ship as built-in drivers — wire them as plugins and reference them with `driver = "plugin"` under `[[notifiers]]`. The pattern below scaffolds a Discord notify plugin from scratch. ```bash # 1) Create a new directory under your dotagent checkout's plugins/. mkdir -p plugins/notify-discord/src cd plugins/notify-discord # 2) Cargo.toml — start from a sink plugin that already POSTs JSON over # HTTPS. `sink-roam` and `sink-file` are the in-tree references; copy # whichever is closest in shape. cp ../sink-file/Cargo.toml ./Cargo.toml sed -i '' 's/sink-file/notify-discord/g' Cargo.toml # 3) Write src/main.rs — implement info/validate/invoke, POST to the # Discord webhook URL on `invoke`. # 4) Register the new crate in the workspace. # Append `plugins/notify-discord` to members in /Cargo.toml. # 5) Build + smoke. cargo build --release -p dotagent-plugin-notify-discord ./target/release/dotagent-plugin-notify-discord info | jq . # 6) Wire it into an agent and run doctor. # In some agent's agent.toml: # [[notifiers]] # driver = "plugin" # name = "notify-discord" # events = ["given_up"] # [notifiers.config] # webhook_url = "https://discord.com/api/webhooks/..." DOTAGENT_PLUGIN_PATH=$PWD/target/release dotagent doctor ``` For the agent-side ergonomics see the [`new-plugin` skill](https://github.com/avelino/dotagent/tree/main/.claude/skills/new-plugin/SKILL.md) — it's the quickest scaffold. *** ## Publishing a plugin ### Option A — bundle with your dotfiles Put the binary or script in `~/.config/dotagent/plugins/` and check it into version control. No further work. This is the right move for plugins that wrap your private secrets. ### Option B — Homebrew (first-party or community) The `dotagent` Homebrew formula installs every plugin shipped with the core release. For a community plugin, create a separate formula: ```ruby # Formula/dotagent-plugin-pushover-pro.rb (your tap) class DotagentPluginPushoverPro < Formula desc "dotagent plugin: enhanced Pushover with retry + dedup" homepage "https://github.com/you/dotagent-plugin-pushover-pro" url ".../v0.1.0/binaries.tar.gz" sha256 "..." depends_on "dotagent" # optional, but signals intent def install bin.install "dotagent-plugin-pushover-pro" end test do assert_match "kinds", shell_output("#{bin}/dotagent-plugin-pushover-pro info") end end ``` Users tap your repo and install: ```bash brew tap you/dotagent-plugins brew install dotagent-plugin-pushover-pro ``` The plugin lands in `bin/` and dotagent finds it via `$PATH`. ### Option C — `cargo install`, `go install`, `pip install` ```bash cargo install --git https://github.com/you/dotagent-plugin-pushover-pro go install github.com/you/dotagent-plugin-foo@latest pipx install dotagent-plugin-bar ``` Whatever package manager — as long as the binary lands in a directory on `$PATH`, dotagent finds it. ### Versioning Plugins are versioned independently of dotagent. The `info` response reports the plugin's own `version` so the daemon can record it in audit events (which version actually ran). dotagent itself only commits to keeping the **protocol** stable — specifically the three verbs and the JSON shape of `invoke`'s payload. *** ## Best practices ### Single-purpose Each plugin does ONE thing. Don't combine notify + sink into one binary. Easier to test, easier for others to discover, easier to swap. ### Stdout = JSON, stderr = humans Never log info text to stdout. Stdout is parsed strictly. Use stderr for "connecting to webhook…", "retrying after 503…", etc. ### Don't leak secrets The agent's `config` is opaque to dotagent — but if your plugin logs the config to stderr, those secrets show up in the daemon's log file. Redact or omit. ### Be idempotent where it makes sense A `sink` plugin running twice on the same window with the same input should produce the same end state. `sink-roam` does this via `marker_regex` (delete the old block before writing the new one). ### Handle external failures gracefully If your plugin makes a network call, **don't panic**. Return `{"ok": false, "error": "503 from upstream"}` and exit non-zero. dotagent records this as a `plugin_invoked` audit event with `ok=false` but doesn't crash. ### Respect platform declarations If your plugin only works on macOS, declare `"platforms": ["darwin"]` in `info`. `dotagent doctor` warns when a manifest references a plugin that won't work on the current host. ### Rate-limit external services Plugins that send to humans (custom Discord/Teams notifiers, etc.) should support a `rate_limit_minutes` config so the user can throttle without modifying the plugin. See the built-in `imessage` driver (`dotagent-notify/src/drivers/imessage.rs`) for the canonical pattern — state lives under `$DOTAGENT_HOME/state/notify//.json` for built-ins; mirror that under `~/.config/dotagent/state/plugins//` for plugins. ### Validate early, fail clearly `validate` is your chance to reject bad config when the user runs `dotagent doctor` — long before a real event fires. Be strict. *** ## Debugging ### "Plugin not found" ``` ✗ plugin notify-discord not found ``` `dotagent doctor` couldn't resolve the binary. Check: ```bash which dotagent-plugin-notify-discord # is it on $PATH? echo $DOTAGENT_PLUGIN_PATH # custom dir? ls ~/.config/dotagent/plugins/ # local install? ``` ### "Plugin failed (exit N): …" Stderr is captured. Either run the plugin manually with the same payload or watch the daemon's structured log: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq -c 'select(.fields.plugin)' ``` (Not `run.avelino.dotagent-error.log` — under launchd / systemd that file only receives daemon crashes, not per-plugin output.) ### "Plugin returned ok=false" The plugin spoke the protocol but reported failure. The audit log has the plugin response: ```bash tail ~/.config/dotagent/audit.log | jq 'select(.event.event_type == "plugin_invoked")' ``` ### Manifest references plugin but it never fires Check the `events` filter. If you wrote: ```toml [[on_success]] plugin = "sink-roam" events = ["daily_summary"] ``` …that filter can never match: `daily_summary` is the daemon's own event and never reaches a manifest entry. The plugin stays silent on ordinary `success` too, because the filter excludes it. Remove the filter, or list the event you actually wanted. ### Plugin works manually but not from the daemon The daemon clears `$DOTAGENT_PLUGIN_PATH` if you start it from a fresh launchd session. Check `launchctl getenv DOTAGENT_PLUGIN_PATH` or move the plugin into `~/.config/dotagent/plugins/` (which is always searched). ### See every plugin invocation in real time ```bash RUST_LOG=dotagent_plugin=debug dotagent daemon ``` Each invocation logs `invoking plugin ` plus the resolved binary path. ### Run a plugin with the exact payload the daemon would send ```bash dotagent plugin invoke sink-roam '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "smoke test", "config": {"page": "today", "marker_regex": "#test"} }' ``` (`dotagent plugin invoke` accepts the payload as a positional arg or `-` to read from stdin.) *** ## Related docs * [`notifications.md`](/concepts/notifications) — built-in notifier drivers (replaces the old `notify-*` plugins) * [`plugin-protocol.md`](/reference/plugin-protocol) — formal protocol spec * [`agent-spec.md`](/reference/agent-spec) — manifest schema (where `[[preflight]]` / `[[on_*]]` / `[[notifiers]]` live) * [`threat-model.md`](/security/threat-model) — security considerations (especially around untrusted plugins) * [`.claude/skills/new-plugin/SKILL.md`](https://github.com/avelino/dotagent/tree/main/.claude/skills/new-plugin/SKILL.md) — scaffolding shortcut # Triggers > A run that no clock asked for. Every run used to come from a `[[schedules]]` window: cron, interval, or expression. A trigger is the other cause — a chat message arrived, a local client sent a message, an MCP client called a tool, or an agent finished and wants to start the next one. ```mermaid flowchart LR clock([clock]) --> daemon msg([a Telegram message]) --> daemon local([local UDS client]) --> daemon tool([an MCP tools/call]) --> daemon daemon --> run[run an agent] run --> back([the answer reaches whoever asked]) ``` ## What a triggered run keeps Everything a scheduled run has. Same supervisor, same heartbeat, same audit trail, same `[[on_success]]` / `[[on_failure]]` hooks, same built-in notifiers. A trigger changes *why* a run started, not *how* it is run. Two things are deliberately different. **No window state.** A triggered run has no attempts counter, never consumes a retry, and can never mark a cron window as given up. Retry policy exists to answer "did the 08:30 window succeed"; a message at 14:00 is not an answer to that question. **Its own slug.** State files are keyed by `trigger-` instead of the schedule's args, so `~/.config/dotagent/state/agents//trigger-telegram.heartbeat.json` sits beside the scheduled history rather than on top of it. When a trigger carries a session, the slug becomes `trigger--`, for example `trigger-local-chat-9_a`. Without this, asking an agent to run on demand would write `last_success_at` for a window that never fired, and the scheduler would stop retrying a genuinely missed run. The local API defaults an omitted `session_id` to `default`, so that transport uses `trigger-local-default`. A trigger with no session at all keeps the session-less `trigger-` form for backwards-compatible state paths. ## Sources | Source | Who produces it | Answers back? | | ---------- | --------------------------------------------------------------------------- | ------------------------ | | `telegram` | the daemon's inbound poller — see [Telegram](/concepts/telegram) | yes, in the same chat | | `local` | the daemon's Unix-socket API — see [Local Client API](/reference/local-api) | yes, as local API events | | `mcp` | `dotagent mcp`, one tool per agent — see [MCP server](/reference/mcp) | yes, as the tool result | | `cli` | reserved | no | ## Serialization Triggers enter the gateway through a bounded channel, separate from the daemon's tick loop. The gateway owns one FIFO worker per `(source, session_id/reply_to/default)` conversation. Request N+1 for the same conversation is not started until N has answered, which preserves per-conversation order by construction rather than by luck. Different conversations run concurrently up to four live gateway workers by default. A per-conversation queue holds up to 64 jobs; a full queue or a new conversation over the cap is rejected rather than blocking unrelated conversations. What is *no longer* serialized is a trigger against the tick. The channel used to be an arm of the daemon's main `select!`, and that arm is only reached **between** ticks — a tick awaits every scheduled run inline. A single scheduled run with a 20-minute deadline therefore held every queued message for its whole duration: the sender saw silence, resent, and got both answers at once when the run finally ended. A triggered run now proceeds beside the tick. This bounded trigger concurrency is safe on everything the gateway and tick paths touch in common: * **Heartbeat.** A triggered run's slug is namespaced by source and, when present, session (`trigger-local-chat-9_a`), so it never shares a file with the scheduled history of the same agent. The state store takes a per-file `flock` regardless. * **Window state.** Triggered runs never write it, so retry accounting stays single-writer. * **Audit log.** Every append takes an exclusive `flock` on the log. * **Persistent instances.** The pool keys a mutex per `(agent, key)`, so a scheduled and a triggered request for the same instance queue instead of interleaving. Scheduled runs remain serialized against each other — the tick still awaits each one inline. Triggered conversations are concurrent with each other up to the gateway cap and with the scheduled tick. The practical consequence: a long *trigger* still delays the next trigger for the same conversation. A ten-minute answer means a message arriving at minute two waits, while a different conversation can use another worker. A long *scheduled run* no longer blocks triggers. If the remaining queueing matters for your setup, keep the dispatcher fast and let it hand slow work to something else. The gateway submit channel holds 64 pending requests, and each conversation has a separate 64-job queue. These bounds are deep enough for a burst to survive one slow answer, and shallow enough that a wedged daemon applies backpressure instead of growing without bound. `[lifecycle] mode = "persistent"` does not change this. It removes the startup cost of each run, not the queue. One consequence worth knowing before you `dotagent reload`: retiring persistent instances waits for whatever they are answering. A SIGHUP that arrives while a triggered request is in flight is applied after it finishes, bounded by that agent's `timeout_seconds`. See [Daemon lifecycle](/guides/daemon-lifecycle#reload-sighup). ## What the agent receives For one-shot runs, trigger context arrives as environment variables, alongside the usual `AGENT_*` block: | Variable | Meaning | | ------------------------ | --------------------------------------------------------------------- | | `AGENT_TRIGGER_SOURCE` | `telegram`, `local`, `mcp`, `cli` | | `AGENT_TRIGGER_ACTOR` | who asked, as the source can attest it | | `AGENT_TRIGGER_REPLY_TO` | opaque handle for the conversation to answer | | `AGENT_TRIGGER_PAYLOAD` | source-specific JSON body | | `AGENT_SESSION_ID` | opaque conversation id for the one-shot run, when the trigger has one | `AGENT_TRIGGER_PAYLOAD` carries the message text, plus a `command` object when the sender invoked one — see [Commands](/concepts/commands#the-payload). It rides in the environment rather than argv on purpose: a body that reached argv would be one quoting bug away from a shell problem, and every current producer is bounded well under `ARG_MAX` (a Telegram message caps at 4096 characters). A source with unbounded payloads should write a file into `AGENT_TMPDIR` rather than grow this variable. These variables are applied *before* the `AGENT_*` block, so a payload can never redefine `AGENT_NAME` or `AGENT_HEARTBEAT_FILE`. A [persistent](/concepts/lifecycle) agent gets none of these per-message environment variables: neither `AGENT_TRIGGER_*` nor `AGENT_SESSION_ID` is defined or fixed in its process environment. An environment is fixed at spawn and that process answers many different messages, so the same context arrives in the `trigger` field of each request frame instead. The frame carries `source`, optional `session_id`, `actor`, `reply_to`, and `payload` — see [the persistent protocol](/reference/persistent-protocol). An agent with no `[[schedules]]` at all is legal and gets the synthetic schedule id `trigger`. That is the natural shape for an agent that only ever runs because someone asked. ## Answering Whatever the agent prints on stdout goes back to whoever asked, when the source supports it. For a local client, raw stdout lines are streamed as `reply.delta` and the gateway sends a final shaped `reply`. Telegram ignores deltas and sends only the final reply. Log to stderr — it is captured for `dotagent logs` and stays out of the reply. The local API and Telegram dispatcher can use `[run] protocol = "assistant-v1"`. The agent emits `delta`, `reply`, and optional `session` JSON frames on stdout; the daemon uses only `reply` for final delivery shaping and does not own the transcript. See [Local Client API](/reference/local-api). The agent that a dispatcher *calls* never learns a chat exists. Only a one-shot triggered agent has `AGENT_TRIGGER_REPLY_TO`; a persistent agent reads `trigger.reply_to` from the current request frame. It is the agent whose stdout gets delivered. ## Security A trigger can originate from outside the machine. The full posture is in the [threat model](/security/threat-model); the short version: * `agent` in a trigger is a **selector, not a command**. It resolves against manifests already on disk, and anything else is refused. * Nothing in a trigger reaches a shell. * Every accepted trigger writes `trigger_received` and `agent_triggered` to the audit log; every refused gateway trigger writes `trigger_rejected` at `Critical`. * Message bodies are never written to the audit log. It records attribution, not transcripts. ## See also * [Lifecycle](/concepts/lifecycle) — keeping a dispatcher alive between messages * [Telegram](/concepts/telegram) — the inbound chat source * [MCP server](/reference/mcp) — agents as tools * [Local Client API](/reference/local-api) — Unix-socket assistant transport * [`examples/telegram-assistant`](https://github.com/avelino/dotagent/tree/main/examples/telegram-assistant/README.md) — the whole loop end to end # Lifecycle > How long one agent process lives. Every agent is a one-shot subprocess. dotagent spawns it, it reads its environment, writes stdout, exits, and the exit code is the verdict. That is the whole execution model, and for a cron agent it is exactly right. For an agent that holds something, it is exactly wrong. ```mermaid flowchart LR subgraph oneshot["mode = "oneshot" (default)"] m1([message]) --> s1[spawn] --> a1[agent] --> x1([exit]) m2([message]) --> s2[spawn] --> a2[agent] --> x2([exit]) end subgraph persistent["mode = "persistent""] m3([message]) --> p[agent, already running] m4([message]) --> p p --> r([answer]) end ``` ## When it is worth it Measured on a real Telegram bot, answering the same trivial message: | | 1st answer | 2nd answer | | ----------------------- | ---------- | ---------- | | one process per message | 3.91s | 4.94s | | one process kept alive | 3.46s | **1.90s** | Every message after the first is a second message. The gap is wider in practice, because a process that died also has to reload whatever it was holding before it can answer — for a chat assistant, that is the conversation. Three shapes benefit: * **A dispatcher with conversation history.** The expensive part is not the spawn, it is rebuilding the transcript. * **An agent with a warm cache.** A model loaded, an index built, a connection negotiated. * **A watcher with an open connection.** A socket that should not be renegotiated per event. Everything else should stay `oneshot`. A process that lives is a process that can leak, drift, and hold stale state — costs a cron agent has no reason to pay. ## Turning it on ```toml [agent] name = "telegram-assistant" # In persistent mode this is the deadline for ONE request, not the lifetime # of the process. timeout_seconds = 600 [run] command = "bash" args = ["./agent.sh"] [lifecycle] mode = "persistent" key = "chat_id" idle_timeout_seconds = 1800 max_invocations = 120 max_instances = 8 startup_timeout_seconds = 30 ``` `[lifecycle]` absent means `oneshot`, and `oneshot` is the default forever. No existing agent changes behavior. The agent itself changes: instead of running once and exiting, it reads requests and writes answers on a loop. That contract is [the persistent protocol](/reference/persistent-protocol), and [`examples/hello-persistent`](https://github.com/avelino/dotagent/tree/main/examples/hello-persistent/README.md) implements the whole thing in about twenty lines of bash. ## One process per what `key` names a field of the trigger payload. Each distinct value gets its own process. ```toml key = "chat_id" # one process per conversation ``` Without it there is one instance for the whole agent — right for a warm cache, wrong for anything holding per-sender context. A dispatcher with no key puts every conversation into one process, and whatever it remembers from one person is there for the next. `dotagent doctor` warns when the Telegram dispatcher is persistent with no key, because nothing about that setup looks broken until two people use the bot. The value is a **selector over the payload the daemon already attested**, never something a sender can point anywhere. Whatever is in the field is reduced to `[A-Za-z0-9_-]` before it reaches a process label; anything else becomes a stable digest of itself. Two different values never collapse into one process. `max_instances` caps how many live at once. Past the ceiling, the least recently used one is terminated to make room — its next message spawns a fresh process, which for a chat means it forgot the conversation. ## When an instance goes away | Reason | Trigger | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------- | | `idle` | Nothing asked it anything for `idle_timeout_seconds`. | | `max_invocations` | It answered its quota. A process holding a conversation degrades as that conversation grows; this is the ceiling that stops it. | | `timeout` | One request outlived `[agent] timeout_seconds`. | | `crashed` | It exited on its own. | | `evicted` | `max_instances` needed the room. | | `config_changed` | `dotagent reload` — a live process still carries the old manifest. | | `shutdown` | The daemon is stopping. | Each one writes `persistent_agent_recycled` to the audit log with the reason and how many requests it answered. `crashed` and `timeout` are `Critical`; the rest are routine. **A crash is not a lost message.** The next request finds the dead process, discards it, and spawns a replacement — and if the process dies in the narrow window between "we set the deadline" and "we wrote the request", the request is retried once against a fresh instance. Twice in a row is a real failure and is reported as one. ## What the daemon owes it Nothing here is new machinery. A persistent instance is spawned through the same supervisor as every other subprocess, which is the entire argument for this living in the orchestrator rather than beside it: * **Supervision.** Real process groups, real kill-tree. A deadline reaches grandchildren. * **Visibility.** It shows up in `dotagent status` under `persistent`, and its spawn and recycle land in the audit log. * **Reaping.** A graceful daemon shutdown retires it through the pool, then the supervisor kill-trees whatever is left. A daemon that dies *without* running that path — `SIGKILL`, a panic, `launchctl kickstart -k` — does orphan its instances: nobody is holding their deadline anymore. Those are collected on the next boot, before the daemon starts anything of its own. See [Boot orphan reap](/guides/daemon-lifecycle#boot-orphan-reap). * **One instance per key.** The pool lives inside the daemon, so there is no second one to race it. This replaces the `flock` an external pool would need. The idle timeout is not a timer of its own. The supervisor's reaper already kills anything past its deadline, so the pool simply re-points that clock: the request deadline while an answer is in flight, the idle window while it is not. One clock, one kill path, no second implementation to get wrong. ## Trigger gateway concurrency The trigger gateway is separate from the persistent-process pool. It owns one FIFO worker per `(source, session_id/reply_to/default)` conversation, so a second request for the same conversation queues rather than interleaving with the first. Different conversations can run concurrently up to four live gateway workers by default. The local Unix-socket API and Telegram ingress use this gateway. A local client also has a 64-job per-conversation queue and a 30-request-per-minute local admission limit. Telegram keeps its allowlist and its own configured rate limit before submission. New conversations over the gateway cap and full conversation queues are rejected; they do not block unrelated conversations. This is independent of `[lifecycle]`: * In `oneshot` mode, each admitted message still spawns one supervised agent process. * In `persistent` mode, the pool reuses a process, while the gateway still owns trigger ordering and admission. What persistence removes is the startup cost, not the gateway's ordering or capacity policy. See [Triggers](/concepts/triggers#serialization) for the shared state boundaries. `dotagent reload` retires persistent instances so the next request re-reads the manifest. Telegram ingress is restarted against the new configuration. The local API listener intentionally stays on the same socket and keeps one owner until daemon restart; a changed `dispatcher_agent` therefore does not replace the local handler during reload. ## Outside the daemon `dotagent run`, `dotagent run-now`, `dotagent tick` and MCP tool calls all live in short-lived processes. A persistent agent still speaks its protocol there — each command builds a pool that lasts exactly as long as the call and tears it down after. Running it one-shot instead would hand it a closed stdin, which every correct implementation reads as "shut down", so the agent would exit without answering. The startup cost is paid and thrown away, which is the point: a one-off invocation should behave like one, and `dotagent run` should exercise the real path rather than a fiction. ## See also * [Persistent protocol](/reference/persistent-protocol) — the wire format * [Agent spec](/reference/agent-spec#lifecycle--how-long-one-process-lives) — every field * [Triggers](/concepts/triggers) — what causes a run * [Local Client API](/reference/local-api) — Unix-socket streaming transport * [Architecture](/concepts/architecture) — where the pool sits * [Threat model](/security/threat-model) — state shared between senders # Telegram Telegram works in both directions. Outbound is the `telegram` notifier driver, which posts when a run fails — see [Notifications](/concepts/notifications). This page is about inbound: messages that cause agents to run. Off unless you configure it. dotagent never opens an inbound path you did not ask for. ## The shape ```mermaid sequenceDiagram actor me as me participant tg as Telegram participant p as poller participant d as dotagent daemon participant disp as dispatcher agent p->>tg: getUpdates (held open) me->>tg: "how's disk on the laptop?" tg-->>p: message Note over p: allowlist, then rate limit p->>d: TriggerRequest d->>disp: run, message in AGENT_TRIGGER_PAYLOAD disp-->>d: stdout d->>tg: sendMessage to the same chat tg-->>me: reply ``` The daemon holds a `getUpdates` connection open rather than polling on a timer, so replies are near-instant without a webhook, a public IP, TLS termination or a tunnel. dotagent runs on laptops behind NAT; a webhook would make the common case the hard one. ## Setup **1. Bot and user id.** Create a bot with [@BotFather](https://t.me/BotFather). Get your numeric id from [@userinfobot](https://t.me/userinfobot) — the number, not the `@username`. **2. Token in the secrets file**, `~/.config/dotagent/secrets.env`, mode `600`: ``` TELEGRAM_BOT_TOKEN=123456:AA... ``` **3. Config**, `~/.config/dotagent/config.toml`: ```toml [telegram] bot_token = "${TELEGRAM_BOT_TOKEN}" allowed_user_ids = [123456789] dispatcher_agent = "telegram-assistant" ``` **4. Reload.** `dotagent reload`. The log line `telegram ingress started` confirms it. Every `[telegram]` change takes effect on reload, including the allowlist — the daemon stops the running poller and starts a fresh one against the new config. Revoking a user id and reloading revokes it immediately, rather than leaving them able to trigger runs until the next restart. Full field list in the [config reference](/guides/config-reference#telegram). ## Why the config is daemon-level Telegram allows exactly one `getUpdates` consumer per bot token. If each manifest declared its own token, N pollers would compete for the same offset and silently drop each other's messages. One bot, one consumer, one place to configure it. ## The allowlist is the whole gate `allowed_user_ids` is what stands between a bot token and arbitrary local execution. Two consequences: **Numeric ids only.** A `@username` is changeable and therefore not an authorization input. **Empty means nobody.** A token with an empty allowlist leaves the ingress off, and the daemon logs why. The other reading — "empty means no restriction" — would turn one forgotten line into an open remote-execution endpoint. Messages from unlisted senders are refused before anything runs and recorded as `trigger_rejected` at `Critical` severity. That severity is deliberate: it means somebody found your bot. ### Open chats: sharing the bot with a group `open_chat_ids` lists group chats where **any member** may talk to the dispatcher — useful when the group itself is the team. The list is explicit chat ids on purpose: anyone can add a bot to a group they control, so the trust boundary is membership in a group the *operator* listed, not membership in any group. Two things an open chat deliberately does **not** grant: * **Direct messages** still require `allowed_user_ids`, always. * **`!` and `!!`** (typed commands that run binaries) stay owner-only even in an open chat — running a program is a different risk class than asking a question. The rate limit applies per sender regardless, and every accepted message still lands in the audit log with the sender's numeric id. ## Rate limiting `rate_limit_per_minute` (default 10) caps accepted messages per sender. Excess is dropped with an audit entry. The bot is reachable from anywhere on the internet, so this is the backstop that keeps one sender from occupying the daemon indefinitely. The window is in memory. A daemon restart clears it — acceptable, and cheaper than a disk write on every message. ## The dispatcher `dispatcher_agent` names the agent every accepted message goes to. It is an ordinary agent: env vars in, stdout out, exit code for success. It receives the message through `AGENT_TRIGGER_PAYLOAD` and whatever it prints becomes the reply. What it does with the message is entirely up to it. [`examples/telegram-assistant`](https://github.com/avelino/dotagent/tree/main/examples/telegram-assistant/README.md) hands it to `claude -p` with the [MCP server](/reference/mcp) attached, so a model picks the right agent from a closed catalog. A dispatcher that just matches `/disk` with `case` is equally valid and costs no tokens. dotagent itself interprets nothing. There is no model, no provider and no prompt in the daemon. A dispatcher that wants *continuity* declares `[assistant]` in its manifest: the daemon then keeps the conversation's pointers (model session id, toolkit hash, transcript size), reinjects them as `AGENT_ASSISTANT_*` on every trigger, retires sessions whose transcript outgrew the ceiling, and strips `MEMO:` capture lines from replies into the memory workspace. The one trigger after automatic retirement receives `AGENT_ASSISTANT_CONTEXT_RETIRED=true`, so the agent can distinguish a forced fresh context from an ordinary new thread. The chat transcript itself still never lives in the daemon. Schema in the [agent spec](/reference/agent-spec#assistant-conversational-harness-opt-in). ## Conversations and threads Which messages share a conversation depends on the kind of chat: * **Direct chat** — one chat, one conversation. Every message shares the session keyed by the chat id. This is the original keying and it does not change. * **Group** — every fresh mention roots a new conversation. Replying to any message of a thread — your own question or the bot's answer — continues that thread's conversation. Two subjects asked in parallel never see each other's context. * **Group with forum topics** — same as a group, and each topic keeps its own conversations. The bot answers inside the topic it was asked in. For a group, the opaque session key is `-r`; a forum topic adds `-t` before the root. For example, `trigger-telegram--1004457436194-r7` identifies Telegram chat `-1004457436194` and reply-chain root message `7`: `r7` is not retry number seven. ```mermaid flowchart TD M[group message arrives] --> R{is a reply?} R -->|no| N[new conversation
rooted at this message] R -->|yes| L[look up the replied-to message] L -->|known| H[continue that conversation] L -->|unknown| N ``` The binding lives in `state/notify/telegram/threads.json`, a bounded table of the last thousand messages per chat — the same philosophy as the notification correlation table, and it can be deleted at any time: an unresolvable reply simply starts a new conversation, which is what every message did before threads existed. `/novo` (or `/new`) resets the current thread's conversation — the next message there starts from zero, keeping memory facts but dropping the model session. It clears any pending automatic-retirement marker rather than emitting one. Like `/help`, it yields to a command file you install yourself. In a direct chat it resets the one conversation there is. `!!` and parked `!` confirmations are scoped the same way: a confirmation releases what *this thread* parked, so two threads cannot confirm each other's destructive commands. ## Commands [Commands](/concepts/commands) put a `/` menu in the chat. The daemon registers it with `setMyCommands` on start and every reload, scoped to each allowlisted chat rather than globally — the allowlist gates execution already, but a global menu would publish every command name to anyone who finds the bot. An invoked command arrives beside the text rather than instead of it: ```json { "text": "/standup disk-alert", "command": { "name": "standup", "args": "disk-alert" } } ``` `command` is `null` for ordinary prose. The daemon parses `/name args` — Telegram wire syntax, the same class of thing as reading `update_id` — and stops there. Resolving a name to a prompt is `command-get`, over MCP, and belongs to the dispatcher. Three answers do come straight from the daemon: `/help`, `/novo` and "no command named /typo". The first two are questions about *what exists* and *what to forget* — the catalog the daemon publishes and the registry the daemon owns. Letting `/typo` fall through would mean a model improvising an answer to something meant to be exact. ## Running a command yourself A message starting with `!` runs an installed binary directly, without the dispatcher seeing it: ``` !rg TODO src !kubectl get pods ``` No model call, no session, nothing stored. Output comes back as-is, and so do errors — the point of typing a command is that it is exact, and a paraphrased exit code is worse than the exit code. Destructive commands ask first. `!rm -r /tmp/x` quotes back what it will run and waits for `!!`; anything else in between cancels nothing but does not confirm either. The parked command is keyed by the conversation — reply in the same thread to release it, and a different thread's `!!` cannot. The list covers the usual suspects and every shell, because a guard on `rm` that lets `sh -c 'rm -rf /'` past guards nothing. It is off unless `[os]` is configured. The prefix obeys the same `allow` list as the assistant's own `os-run`, and it is read *after* the allowlist and rate limit, so `!` is never a way past either. Quotes group an argument (`!rg "hello world"`); nothing else from a shell applies, because no shell is involved. See [`../guides/config-reference.md`](/guides/config-reference#os) for the allowlist and [`../security/threat-model.md`](/security/threat-model) V17 for what it does and does not bound. ## Answering a notification The bot posts when a run fails. Replying to that message is the natural next move, and the reply carries which run it answers: ```jsonc "reply_to_run": { "agent": "disk-alert", "schedule": "every-15min", "event": "given_up" } ``` Resolved from the replied-to message id and the inbound chat id, through a chat-scoped table of the last few hundred notifications at `state/notify/telegram/sent.json`. Not from the text: one event reads `🚨 disk-alert/every-15min gave up after 3 attempts` and another reads only `preflight aborted by plugin preflight-warp`, so a dispatcher parsing the wording would work for one and be wrong on the other. Legacy records without a chat id fail closed rather than risking a match from another chat. With it, "por que falhou?" is answerable without guessing which agent — the dispatcher reads the field and calls `dotagent-logs` on the right one. If the manifest declared a [remediation](/reference/mcp#remediation-tools), the fix is a tool in the same catalog. ## Replies The dispatcher's stdout goes back to the chat that asked. Sent as plain text, not MarkdownV2 — the body is agent output, and an unescaped backtick or underscore in a log line would otherwise turn into a Bot API 400. Delivery beats formatting. Telegram caps a message at 4096 characters. Longer output is trimmed with a `[truncated]` marker; the full text stays in `dotagent logs`. ## Offset The last acknowledged `update_id` lives in `~/.config/dotagent/state/notify/telegram/offset.json`, written tmp-then-rename. Telegram redelivers every update until you acknowledge it. Without persistence, a daemon restart would replay the backlog — and for a bot that runs agents, replay means re-running whatever the last messages asked for. Delivery is at-most-once on purpose. ## What is never recorded Message bodies do not reach the audit log. `trigger_received` records the sender id and the chat id; the text is not attribution, it is content, and a chat can contain anything you pasted into it. ## Failure modes **Nothing happens.** Usually the allowlist. Check the audit log for `trigger_rejected`, and confirm the id is numeric and yours. **`telegram bot_token set but allowed_user_ids is empty`.** The ingress stayed off by design. Add your id. **`telegram poll failed`** repeating with a growing backoff. Transport problem — network, or a revoked token. The backoff doubles to a 60-second ceiling so a dropped connection does not become a tight retry loop. **The reply never comes but the log shows the run.** The dispatcher printed nothing on stdout, or printed to stderr. stdout is the reply. ## See also * [Triggers](/concepts/triggers) — the general concept * [MCP server](/reference/mcp) — how a model picks an agent * [Skills](/concepts/skills) — teaching the dispatcher a procedure without growing its prompt * [Threat model](/security/threat-model) — what changes when this is on * [Notifications](/concepts/notifications) — the outbound driver # Memory > Facts that outlive the conversation, in a workspace you can open and edit. An agent that talks to you needs two kinds of recall, and they are not the same thing. **Conversation history** is what was just said. It makes "yes" resolve to the thing that was proposed a minute ago. That lives in whatever the agent uses to hold a session, and it is the agent's business. **Memory** is what stays true next week: a preference, a decision, a correction. That is what this page is about. ```mermaid flowchart LR agent[agent] -->|remember / recall / supersede / forget| mcp[dotagent mcp] sched[scheduled agent] -->|MEMO: on stdout| daemon[daemon] daemon --> ws[(outl workspace)] mcp --> ws you([you]) -->|open, read, edit| ws you -->|dotagent memory| ws ``` ## It is a graph, not a list A fact is stored once, in the journal for the day it was learned, and tagged with topics. Each topic is a `[[link]]` to a page, and outl resolves the backlinks — so the topic page gathers every fact that mentioned it, from whichever day. ```mermaid flowchart TD j1["journal 2026-08-01
uses Rust [[dotagent]] [[rust]]"] --> d[dotagent] j1 --> r[rust] j2["journal 2026-08-04
runs under launchd [[dotagent]]"] --> d j3["journal 2026-08-04
CRDT + op log [[outl]]"] --> o[outl] ``` Ask for the day and you get what happened. Ask for `dotagent` and you get everything known about it, gathered across months. One copy, two ways in. Topics are normalized before they become slugs: `Roam Research`, `roam research` and `Roam Research` all land on `roam-research`. Without that, capitalization alone would split a subject into disconnected pages and each half would look incomplete. A `/` survives, because hierarchy is a real thing in outl — `ops/cost-report` is a page path. **Reuse topics.** The value is in the gathering, and gathering only works if the same subject keeps the same name. `memory-topics` exists so an agent can check before inventing one — and an agent running under the `[assistant]` harness does not have to ask: the topics already in use are appended to the memory block it receives, so the vocabulary is in front of it while it writes. ## Where it lives An [outl](https://github.com/avelino/outl) workspace at `~/.config/dotagent/outl`, scaffolded the first time the daemon starts. Nothing to configure. That choice is the point: memory is a normal outl workspace, not an opaque store. Open it in the desktop app, read what an agent decided to keep, fix a memory that is wrong, delete a page. One block per fact, in the journal for the day it was learned. ``` ~/.config/dotagent/outl/ ├── journals/2026-08-04.md ← the facts, one block each ├── journals/2026-08-04.outl ← block identity sidecar ├── pages/dotagent.md ← a topic; backlinks gather here └── ops/ ← the op log, source of truth ``` A journal reads as an ordinary outline — the fact on the bullet, its bookkeeping indented under it as properties: ```markdown - o dotagent roda no launchd [[dotagent]] agent:: telegram-assistant seen:: 1 source:: telegram ``` Point it somewhere else — including at the workspace holding your own notes — with `[memory] workspace` in `config.toml`. See the [config reference](/guides/config-reference#memory). ## The tools `dotagent mcp` exposes five, alongside the agent catalog. The same verbs are available from a shell as [`dotagent memory`](/reference/cli#memory), which is what lets a consolidation pass be an ordinary scheduled agent instead of logic buried in the daemon. | Tool | Arguments | Behavior | | ------------------ | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `memory-remember` | `text`, `topics[]` | Store one fact in today's journal, linked to each topic. Topic pages are created if absent. Restating a fact reinforces the one already there. | | `memory-recall` | `query` **or** `topic` | With `query`, ranks facts by relevance. With `topic`, returns every fact linked to it via backlinks. | | `memory-topics` | — | Lists the subjects that exist. | | `memory-supersede` | `id`, `text`, `topics[]` | Replace a fact that stopped being true. | | `memory-forget` | `id` | Delete a fact outright. | Recall results lead with the fact's id, and that id is what `supersede` and `forget` take — an agent that recalls something wrong can fix it in the next call instead of asking which one. ### Ranking is lexical, not semantic A fact competes only if it shares a real word with the question. Among those, recency and repetition break the tie: ``` score = 3.0 × (fraction of query terms matched, topics counting extra) + recency (decays to ~37% over two months) + 0.4 × ln(times the fact was stated) ``` Words split on punctuation, so `databricks-cost-daily` is findable as "databricks"; stopwords and two-letter tokens are dropped, because matching on "de" or "the" makes every fact a hit and flattens the ranking back into pure recency. Query and fact are normalized the same way before they meet: accents fold and a trailing plural `s` comes off. Portuguese gets typed both ways and a topic slug is whatever the agent coined, so a question about "pendência" has to find a fact tagged `pendencias`. Irregular plurals are left alone — a stemmer that guesses is how "reunião" starts matching "reunir". No vectors, on purpose. outl stores text, and an assistant states recall results as fact — a near-miss surfaced confidently is worse than a miss. So a question about kubernetes, in a store that knows nothing about kubernetes, returns nothing rather than the closest thing. Recall by topic is still the better question when it applies: it asks the graph rather than guessing which words the fact happened to use. ## What keeps it from rotting A store that only ever appends degrades into noise, and recall from noise is worse than no recall. **Restating a fact reinforces it.** "prefere reunião depois das 14h" said twice is one fact with `seen:: 2`, not two blocks. Repetition becomes a ranking signal instead of a duplicate that crowds out everything else. Any topic the restatement names is merged into the fact already stored. **A fact that changed is superseded, not contradicted.** When "prefere reunião de manhã" becomes "prefere depois das 14h", the old block is marked `superseded-by::` and stops coming back from recall. It stays readable in the journal for the day it was true — the history of a changed preference is worth keeping — but the assistant is never picking between two answers. **Every fact records where it came from.** A memory store you cannot audit is one you stop trusting: when an assistant states something wrong, the first question is which run put it there. ``` - prefere reunião depois das 14h [[agenda]] [[reuniao]] agent:: telegram-assistant last-seen:: 2026-08-21 seen:: 2 session:: 8f2a source:: telegram ``` Those are outl block properties, not text dotagent invented: the desktop app shows them as properties, they round-trip through the `.md` projection, and the fact itself stays one readable sentence. They are also never children of the block — a child block is another fact, and recall would hand back `agent:: telegram-assistant` as something the assistant knows. ## What belongs in it Whatever an agent writes here it will read back months later, with none of the surrounding context. That shapes what is worth keeping. Worth remembering: a stated preference, a decision and its reason, a correction of something the agent got wrong, a name it will need again. Always with topics — a fact stored without them is a fact that will only be found by accident. Not worth remembering: anything already in the current conversation, anything the agent can look up on demand (calendar, metrics, repository state), and small talk. **A memory store full of noise is worse than an empty one** — recall starts returning the wrong thing, and the agent starts answering from it. This is guidance for whoever writes the agent's prompt. dotagent stores what it is given; it does not judge. ## Who writes to it Three doors, all landing in the same workspace: | Writer | How | Provenance recorded | | ---------------------------------- | ------------------------------------------------------------------------------------------ | -------------------------- | | A conversational agent | `MEMO:` lines in its reply, captured by the `[assistant]` harness | agent, source, session | | The same agent, when it forgets | `[assistant.extractor]` — a command the daemon runs over the turn after the reply goes out | agent, source, session | | An ordinary agent, when it forgets | `[memory.extractor]` — a command the daemon runs after a successful scheduled run | agent, `source:: schedule` | | Any other agent | `MEMO:` lines on stdout, when its manifest declares `[memory]` | agent, `source:: schedule` | | Anything holding the tools | `memory-remember` over MCP, or `dotagent memory remember` | agent | Dates in a fact are linked on the way in. `TODO até 2026-08-24: abrir a issue` is stored as `TODO até [[2026-08-24]]: abrir a issue`, so the day's journal backlinks to the pendency and asking "what is due Monday" is a backlink walk rather than a text search. Only ISO dates, because that is the slug a daily page carries; a date already in brackets is left alone, so re-rendering a fact never nests them. The link is added by the store, not asked of a model — a rule that depends on whichever model wrote the fact remembering the syntax is not a rule. A linked day is a link, never a topic page: the journal for that date already exists, and creating a `pages/2026-08-24.md` beside it would leave two pages answering to one name with the backlink landing on the empty one. The second door is what keeps the first from being a single point of failure. `MEMO:` capture works only when the dispatcher's prompt asks for it *and* the model complies, and neither is something the daemon controls: a model that finishes a conversation without emitting a line leaves no error behind, just a journal that did not grow. Declaring an [`[assistant.extractor]`](/reference/agent-spec#assistantextractor--capturing-what-the-model-did-not-volunteer) moves the decision to a command the operator names, run by the daemon after every turn. The model's own lines remain a shortcut worth taking; they stop being the only way in. The third door is the one that makes the store grow on its own: a scheduled agent that learns something durable prints `MEMO: | topics: a, b` and the daemon files it. Opt-in per manifest, and only on a successful run — see [`[memory]` in the agent spec](/reference/agent-spec#memory--memory-capture-for-a-plain-agent-opt-in). An ordinary scheduled agent can configure `[memory.extractor]` when it does not control its output format. The daemon passes the successful run output as JSON to the configured command, parses returned `MEMO:` lines, merges them with explicit stdout memos, and adds the manifest's topics. Extractor errors are best-effort and do not turn a successful run into a failure. An agent with `[assistant] memory = true` is excluded from this path because the assistant harness owns its memory capture. Opt-in rather than automatic because most agents print status, not knowledge. A store that absorbs status is a store whose recall returns status, and then the assistant answers from it. ## Transcripts are somewhere else Memory is curated: an agent decides a fact is durable and stores it. It is not a log of what happened. For "what did the bot do today", read the agent's output: ```bash dotagent logs telegram-assistant -n 100 ``` Keeping the two apart is deliberate. A transcript in the same journal would make `memory-recall` compete with chatter — a search for "meeting" would return every time the word came up in conversation instead of the one preference that matters. ## The embedder contract `dotagent-memory` embeds `outl-ws` and `outl-actions` rather than shelling out. Three things follow from outl's [embedding contract](https://github.com/avelino/outl/blob/main/docs/embedding.md), and getting them wrong corrupts a workspace: **Mutate through actions, then project.** Every write goes through `outl-actions` and is projected back to `.md`. Skip the projection and the op log holds the fact while the file on disk does not. **Never edit the `.md` directly.** The op log is the source of truth; the markdown is a projection of it. **Each call opens and drops the workspace.** `dotagent mcp` is short-lived, and holding the lock across a session would block the desktop app. Getting an ephemeral actor because something else owns the config actor is the normal case, not an error. ## Turning it off ```toml # config.toml [memory] enabled = false ``` The tools disappear from `tools/list` and the workspace is left alone. `dotagent doctor` reports where memory lives, or that it is off. > **Two sections named `[memory]`.** This one, in `config.toml`, says where the workspace lives and whether memory exists at all. The one in an `agent.toml` says whether *that agent* writes to it. Different files, different questions. ## See also * [MCP server](/reference/mcp) — how the tools are exposed * [`dotagent memory`](/reference/cli#memory) — the same verbs from a shell * [Agent spec](/reference/agent-spec#memory--memory-capture-for-a-plain-agent-opt-in) — `[memory]` capture for a plain agent * [Config reference](/guides/config-reference#memory) * [Triggers](/concepts/triggers) — the other half of a conversational agent # Skills > Procedures an assistant loads when they apply, and ignores when they do not. An assistant wired to dotagent knows three things. It knows what can be **run** — one tool per installed agent. It knows what is **true** — [memory](/concepts/memory). It knows how things are **doing** — status, logs, inspect. A skill is the fourth: **how to do something**. ```mermaid flowchart LR a["agents
verbs"] --- m["memory
facts"] --- s["skills
procedures"] ``` Without them, a procedure has two bad homes. Put it in the system prompt and every message pays for something one message in fifty uses. Make it an agent and a procedure is forced to become a script — but "pull these three agents, compare against last week, answer in this shape" has nothing to execute. It has something to *follow*. ## The shape A skill is a directory with a `SKILL.md`: ``` ~/.config/dotagent/skills/weekly-numbers/ ├── SKILL.md ← frontmatter + the procedure ├── scripts/ │ └── compare.sh ← optional, runnable └── references/ └── glossary.md ← optional, readable ``` ```markdown --- name: weekly-numbers description: How to close out the week. Use when asked for "the numbers", the weekly summary, or how last week went. --- 1. Call `run-hn-digest` and `run-disk-alert`. 2. Compare against last week — `scripts/compare.sh` does the arithmetic. 3. Answer in one paragraph, leading with what changed. ``` `name` and `description` are the contract. `description` is the **trigger**: it is the only part a model sees before deciding, so a skill without one is a skill nothing will ever call. That is a validation error, not a warning. Everything else in the frontmatter is ignored, which is what lets a file written for another tool land here unchanged. ## How a model reaches it Each skill becomes a tool, `skill-`. The catalog costs one line per skill; the procedure costs nothing until it is asked for. ```mermaid sequenceDiagram actor me as me participant llm as the assistant participant d as dotagent mcp llm->>d: tools/list d-->>llm: run-* · skill-* · memory-* · dotagent-* me->>llm: "close out the week" Note over llm: description matched — body not loaded yet llm->>d: skill-weekly-numbers d-->>llm: the procedure + what files sit beside it llm->>d: skill-read glossary.md llm->>d: run-hn-digest llm-->>me: the answer, in the shape the skill asked for ``` Three tools, and the last two are why a multi-file skill works at all: | Tool | Arguments | Behavior | | -------------- | --------------------------- | ------------------------------------------------------------ | | `skill-` | — | Return the procedure, plus an index of the files next to it. | | `skill-read` | `skill`, `path` | Return one supporting file. | | `skill-run` | `skill`, `script`, `args[]` | Execute one `scripts/` entry, return its output. | Loading a skill **returns text**. It does not perform the procedure — the tool description says so out loud, because half of these are written in the imperative ("Cut a new release") and a model could reasonably read the call itself as doing the thing. ## Reusing what you already wrote The format is Anthropic's [Agent Skills](https://code.claude.com/docs/en/skills) layout, and `~/.claude/skills/` is searched by default. A skill you wrote for Claude Code is already installed here; nothing to copy, nothing to symlink. Search order, first match winning a name: 1. `[skills] paths` from `config.toml` 2. `$DOTAGENT_ROOT` and `$DOTAGENT_ROOT/skills` — for testing 3. `~/.config/dotagent/skills/` 4. `~/.claude/skills/` and `./.claude/skills/` 5. `./skills/` **Not everything ports, and the part that does not is worth naming.** A skill that is a procedure plus its own `scripts/` works anywhere. A skill whose steps are "use Read, then Grep, then invoke another skill" assumes the Claude Code harness — an assistant reached over MCP has no Bash, no filesystem tools and no nested skill invocation. dotagent serves the text faithfully; it cannot conjure the tools the text asks for. So the value is not in Claude Code, which already reads those files natively. It is in everywhere else: a Telegram assistant, Claude Desktop, an aggregating proxy — anywhere a model has no harness of its own. A bundle that keeps its skills one level down (`radar/skills/*/SKILL.md`) is not walked recursively. Point at it directly: ```toml [skills] paths = ["/Users/me/.claude/skills/radar/skills"] ``` ## Packaged scripts `scripts/` makes a skill a unit rather than a paragraph — the procedure and the mechanical part travel together. Two rules, both narrowing: **`scripts/` only.** A `references/` file is a document, and "run this document" is never the intent. **Executable bit required.** It is the author naming the entry points. Dropping a helper next to them does not silently make it callable. Execution goes through the supervisor, like every other orchestrated subprocess: a deadline that is enforced, and kill-tree so a script that spawns children cannot leave orphans. `timeout_seconds` in the frontmatter overrides the 300-second default. If the daemon itself dies before the deadline lands, the next boot collects what it left behind — see [boot orphan reap](/guides/daemon-lifecycle#boot-orphan-reap). The script gets `SKILL_NAME` and `SKILL_DIR` in its environment, runs with the skill directory as its working directory, and receives arguments through argv — never a shell. Every run is audited as `skill_invoked`. This is code executing outside any manifest, so without that entry "what ran on this machine" would have a hole in it. ## What a caller cannot reach `skill-read` and `skill-run` take a path from the model, which makes them a boundary rather than a convenience: * Absolute paths and `..` are refused before anything touches the disk. * The resolved path must still sit under the skill directory after canonicalization — which is what catches a symlink pointing outside. * A name that is not in the catalog is not a skill, and `../` in one is refused rather than followed. Installing a skill is the decision. Anyone who can write to a skill directory can put a script there and have it run — the same statement the [threat model](/security/threat-model) already makes about manifests. ## Configuration None required. Skills are on, the catalog starts empty, and an empty catalog lists no tools. ```toml [skills] enabled = false # drop the tools entirely claude_skills = false # stop searching ~/.claude/skills paths = ["/opt/team-skills"] # extra roots, searched first ``` `dotagent doctor` reports how many were found, which failed to parse, and any two names that collapse to the same tool name. ## Writing one worth calling The description is the whole interface. "Weekly report stuff" will never fire; "Use when asked for the weekly numbers, the summary, or how last week went" will. Keep the body short enough to act on. A procedure long enough to evict what the assistant needs in order to follow it has defeated itself — the body is capped at 32 KB and truncation is announced, but a skill that hits the cap should be split, with the detail moved into `references/` where it is fetched only when needed. Say what **not** to do. Half of a good procedure is ruling out the plausible wrong move, and that is exactly the part a model will not infer. ## See also * [Commands](/concepts/commands) — the same markdown, invoked by name instead of chosen * [MCP server](/reference/mcp) — how the tools are exposed * [Memory](/concepts/memory) — the facts half of the same problem * [Agents](/concepts/agents) — the verbs * [Threat model](/security/threat-model#v10--skill-script-execution) # Commands > A skill is a procedure the model picks. A command is a procedure **you** pick. Type `/` in Telegram and the client renders a menu of everything the assistant can do. Pick one, it runs. No phrasing to remember, no model deciding what you meant. ``` ~/.config/dotagent/commands/ standup.md commit-message.md git/status.md → /git_status ``` Each file is one command, in [Claude Code slash command](https://docs.claude.com/en/docs/claude-code/slash-commands) format: ```markdown --- description: What ran overnight, in three lines argument-hint: "[agent]" --- Report what the orchestrator did while I was away. Scope: $ARGUMENTS — if that is empty, every agent. ``` The name comes from the filename. `description` is required — it is the line the menu shows, and a command nobody can identify is one nobody picks. ## Why this exists next to skills [Skills](/concepts/skills) already load a procedure on demand. The difference is who chooses. | | Skill | Command | | ------------------ | --------------------------------- | --------------- | | Chosen by | the model, matching a description | you, by name | | Discovered through | `tools/list` | a Telegram menu | | Costs | a model deciding | nothing | | Wrong outcome | it loads the wrong procedure | there isn't one | When you already know what you want, dispatch is pure overhead: latency, tokens, and a chance of picking something else. Commands remove the decision because it was already made. Both read the same kind of markdown. A procedure worth having both ways can be written twice; that is deliberate for now — see [Open edges](#open-edges). ## The shape ```mermaid sequenceDiagram actor me as me participant tg as Telegram participant d as dotagent daemon participant disp as dispatcher agent participant mcp as dotagent mcp Note over d,tg: on start and every reload — setMyCommands me->>tg: "/" tg-->>me: standup · commit_message · help me->>tg: /standup disk-alert tg-->>d: message Note over d: lexical parse — name + args, nothing resolved d->>disp: AGENT_TRIGGER_PAYLOAD.command disp->>mcp: command-get {name, args} mcp-->>disp: the prompt, arguments substituted disp-->>d: the answer d-->>tg: reply ``` The daemon parses `/name args` and publishes the catalog. It never resolves a command to its body — that is `command-get`, and it belongs to whoever is dispatching. See [what the daemon does](#what-the-daemon-does-and-does-not). ## Arguments `$ARGUMENTS` takes everything after the name. `$1`, `$2`, … take whitespace-separated positions. ```markdown Review $1 against $2. ``` `/review src/ main` → "Review src/ against main." Substitution is textual and nothing reaches a shell. Two consequences worth knowing before they surprise you: **A body with no placeholder still gets the arguments.** They are appended under an `Arguments:` header rather than dropped. Typing `/simplify src/foo.rs` and having the path silently vanish would produce a confident answer to a question you did not ask. **`$5.00` loses its 5.** A `$` followed by digits is a position, always. The alternative — treating `$N` as literal when position N was not supplied — makes a body mean different things depending on what the caller typed, so `[$1][$2]` with one argument would emit a literal `$2` that a model reads as an instruction. Write `USD 5.00`. ## Two names, two collision rules One command carries two derived names, and they are lossy in different ways: | File | MCP | Telegram | | ------------------- | ------------------------ | ----------------- | | `commit-message.md` | `command-commit-message` | `/commit_message` | | `git/status.md` | `command-git-status` | `/git_status` | Telegram allows `[a-z0-9_]{1,32}` — **no hyphens**. So `weekly-numbers` and `weekly_numbers` are two distinct commands that both want `/weekly_numbers`. The catalog is deduped twice; first discovered wins, and `dotagent doctor` names the files so you know which to rename: ``` commands: 4 found ⚠ weekly-numbers (…/a/weekly-numbers.md), weekly_numbers (…/b/weekly_numbers.md) all want /weekly_numbers — only the first is in the menu ``` A name longer than 32 characters is **refused**, not truncated. A truncated name produces a menu entry that resolves to nothing. Typing either spelling works: `/commit-message` and `/commit_message` land on the same file, because those are two spellings of one thing. ## Discovery Searched in order, first match winning: 1. `[commands] paths` from `config.toml` 2. `$DOTAGENT_ROOT/commands` 3. `$DOTAGENT_HOME/commands/` ← typical 4. `~/.claude/commands/` and `$CWD/.claude/commands/` — **only** when `[commands] claude_commands = true` 5. `$CWD/commands/` Subdirectories nest with `:` up to three levels. A broken file is skipped and reported by `doctor` rather than emptying the catalog — a menu that silently loses entries reads as "that command was removed" rather than "that command is broken". ### Why `~/.claude/commands` is off by default This is the one place commands differ from skills, where the equivalent switch defaults to on. A skill costs a line in a list until a model judges it relevant. A command is *published as a menu*. And a Claude Code command catalog is typically full of things that assume a shell and a working directory — `/apply` switching a Nix profile, `/check` running a test suite, `/dx` opening a terminal layout. An assistant reached over chat has none of that, so importing them wholesale fills the menu with entries that cannot work. Turn it on when the catalog was written for an assistant: ```toml [commands] claude_commands = true ``` ## What the daemon does, and does not [Telegram](/concepts/telegram) says plainly: *dotagent itself interprets nothing.* Commands are the first feature that pushes on that, so the line is drawn explicitly. The daemon **does**: * **Parse lexically.** `/name@bot args` is Telegram wire syntax, the same class of thing as reading `update_id`. The result goes into the payload as-is. * **Publish the catalog** via `setMyCommands` on start and every reload. Reading a directory to list what is installed is what discovery already does for manifests. * **Answer about the catalog** — `/help`, and "no command named /typo". Both are questions about *what exists*, which is the daemon's own knowledge. Letting `/typo` fall through would mean a model improvising an answer to something meant to be exact. The daemon **never** resolves a command to its body, substitutes an argument, or decides what a command means. That is `command-get`. Writing your own `help.md` replaces the built-in — an explicit choice beats a default. ## The payload An invoked command arrives beside the text, never instead of it: ```json { "text": "/standup disk-alert", "chat_id": 123, "user_id": 456, "message_id": 7, "reply_to_text": null, "command": { "name": "standup", "args": "disk-alert" } } ``` `command` is `null` for ordinary prose. `name` is the **catalog** name, already resolved from whatever spelling was typed, so a dispatcher passes it straight to `command-get` without parsing anything. A dispatcher that is a `case` statement needs no model at all: ```bash case "$(jq -r '.command.name // empty' <<<"$AGENT_TRIGGER_PAYLOAD")" in standup) exec ./bin/standup ;; "") exec ./bin/ask-a-model ;; esac ``` ## MCP tools Two tools for the whole catalog — **not** one per command: | Tool | Arguments | Behavior | | -------------- | -------------- | ----------------------------------------------------- | | `command-get` | `name`, `args` | Resolve an invoked command into the prompt to follow. | | `command-list` | — | Every installed command, with what it does and takes. | This deliberately breaks the pattern skills follow, where each skill is its own tool. There, the catalog is a menu the model picks from. Here the choice was already made by a human before the model saw anything — publishing N command tools would re-open it, letting a model call `command-standup` when the sender typed `/simplify`. That is precisely what commands exist to make impossible. `command-get` returns the prompt with arguments substituted. It does not perform it. ## `allowed-tools` is a hint The frontmatter field is parsed and passed along, labeled: ``` The command suggests these tools: dotagent-status, dotagent-logs (A hint from its author, not a restriction dotagent enforces.) ``` dotagent does not own the dispatcher, so it cannot constrain what that dispatcher's model may call. A dispatcher that *does* control its own harness can act on the field. Presenting it as a guarantee would be worse than not having it. ## Configuration ```toml [commands] enabled = true # default claude_commands = false # default; see above paths = [] # extra roots, searched first ``` Nothing is required. An empty catalog costs nothing: no menu is registered and no tools are listed. ## Security Installing a command grants **no execution**. A command is a prompt; the only thing it can do is what the dispatcher's tools already allow. That is the whole difference from a skill's `scripts/`. What it does grant is influence over what a model is told to do, so the directory deserves the same care as any other input the assistant trusts. See the [threat model](/security/threat-model#v11--commands). The menu is registered per allowlisted chat rather than globally. The allowlist already gates execution, so a global menu would not be an authorization hole — but it would publish every command name and description to anyone who finds the bot. `dotagent audit` records `command_dispatched` with the **name only**. Arguments are content, the same reason a message body is never recorded. ## Open edges * **A command and a skill over one body** currently means two files. Whether they merge into one directory with two doors is worth deciding once both have been used enough for the overlap to be visible rather than hypothetical. * **`/cmd@otherbot` in a group chat** is treated as an invocation. dotagent's allowlist is per-user, so the 1:1 DM is the case that matters; a group with two bots would need the bot's own username to disambiguate. ## See also * [Skills](/concepts/skills) — the same markdown, chosen by the model * [Telegram](/concepts/telegram) — the ingress that publishes the menu * [MCP server](/reference/mcp) — `command-get` and `command-list` * [Config reference](/guides/config-reference#commands) # Running installed binaries An assistant that can read your calendar and your agents' logs, but cannot run `rg`, spends its day asking you to paste things. `[os]` is the section that lets it reach the binaries already on the machine — and the design work is almost entirely about what it must *not* be able to reach. It is **off by default**, and empty by default when on. Nothing in this page happens until you write the section. ## The shape ```toml [os] enabled = true allow = ["rg", "outl", "kubectl get"] ``` That is the whole minimum. `allow` is the policy; everything else refines it. Two things consult that list: ```mermaid flowchart LR M[assistant] -->|os-run| P{[os] policy} H[you, typing !rg foo] --> P P -->|deny| X[refused] P -->|confirm| C[parked, waits for !!] P -->|allow| R[argv → supervisor → output] ``` The model's door and yours land on the same policy. That is deliberate: two implementations of one rule become two rules the first time someone edits only one of them. ## Why an allowlist and not a sandbox dotagent already runs whatever your manifests say, as you. The question here is narrower: **what can a message cause to run?** A message is not a manifest. It arrives from a chat, it may be a forwarded article, and an assistant reading it composes tool calls from whatever is in its context. So the boundary cannot be "whatever the model decided" — it has to be something you wrote down before the conversation started. An entry is a binary name, optionally followed by the leading arguments that must match: | Entry | Admits | Refuses | | ------------- | ----------------------- | -------------------------------------- | | `rg` | `rg` with any arguments | — | | `kubectl get` | `kubectl get pods` | `kubectl delete`, `kubectl getsecrets` | Matching is on whole tokens, so a prefix never sneaks through. A path is refused where a name belongs — `/bin/sh` is not a way to spell a listed `sh`. Pick the granularity per binary. Bare for the ones that only read, pinned to a subcommand for the ones that can change something you care about. ## Opening everything ```toml allow = ["*"] ``` One entry, every binary on `PATH`, a shell included. It is supported, and it is the widest thing in this document. Prefer it over enumerating `PATH`. A list of a thousand names goes stale on the next install and reads as though a decision was made about each one; `*` says what is true. `doctor` prints a warning on every run while it is set, because this is exactly the kind of thing that gets configured once and forgotten. What it means concretely: whoever can send a message can run what you can run. Whatever guards the inbound channel — Telegram's `allowed_user_ids`, the local socket's uid check — is now guarding the machine rather than the agent catalog. ## Asking before it acts Two more lists say *how* something runs, not *whether*: ```toml deny = ["shutdown", "reboot"] # never, whoever asks confirm = ["rm", "dd", "sh", "bash"] # only after a person says yes ``` `deny` wins over `allow`, `*` included. A list that could be widened past its own refusals would not be a refusal. **`confirm` does not default to empty.** With `allow = ["*"]`, an empty one would mean a chat message can repartition a disk with nothing in between. The default covers the destructive classics — `rm`, `rmdir`, `dd`, `mkfs`, `shred`, `diskutil`, `fdisk`, `parted`, `shutdown`, `reboot`, `halt` — and every shell: `sh`, `bash`, `zsh`, `fish`, `dash`, `ksh`. The shells are the entry that makes the rest mean anything. A guard on `rm` that lets `sh -c 'rm -rf /'` through guards nothing. Matching per binary is also what makes it hold. `confirm = ["rm"]` catches all of these: ``` rm -rf / rm -r -f / rm -fr / rm --recursive --force / ``` A textual pattern like `"rm -rf"` catches the first and misses three. ## The `!` prefix A message beginning with `!` runs the command directly. The dispatcher never sees it: ``` !rg TODO src !outl page history buser-cto ``` No model call, no session, nothing stored. Errors come back raw, because the point of typing a command is that it is exact and a paraphrased exit code is worse than the exit code. For anything on the `confirm` list, it parks: ``` !rm -r /tmp/build This will run: rm -r /tmp/build Send `!!` to confirm. It expires in 120s. !! `rm` exited 0 and printed nothing. ``` One slot per conversation, so `!!` can only release what *that* chat parked. Pending confirmations live in memory: a restart forgets them, which fails in the safe direction. The prefix is read **after** the channel's allowlist and rate limit, never before. Reading it earlier would make `!` a way past both. Quotes group an argument (`!rg "hello world"`). Nothing else from a shell applies, because no shell is involved: `!ls; rm x` looks for a binary named `ls;` and does not find it. Works over Telegram and over the local socket (`dotagent api`). ## A model may not confirm itself This is the one asymmetry between the two doors, and it is the reason there are two. When the assistant asks for something on the `confirm` list, `os-run` refuses and tells it to have you type the line. A tool that could both request and grant a confirmation would be theatre. The reasoning is not that typed commands are trusted more. A typed line cannot be steered by content at all — there is nothing to inject. A model composes from whatever entered its context, so the prompt-injection surface exists on exactly one of the two paths, and that is the one where the brake has to be absolute. ## Naming a binary so the model knows it exists `os-run` makes every allowed binary reachable. Reachable is not discoverable: a model has to already know `outl` exists and guess what it is for. ```toml [[os.tool]] bin = "outl" description = "Personal outliner: search notes, read a page by slug, read a daily journal." [[os.tool]] bin = "kubectl" args = ["get"] description = "Read Kubernetes objects: pods, deployments, nodes. Read-only." ``` That publishes `os-outl` and `os-kubectl-get` in the MCP catalog, each carrying your sentence. The description is the entire point — a name without one is what `os-run` already offers. `args` fixes the leading arguments and the model can only append. That is what makes `kubectl get` a publishable read-only view of a binary that also deletes: asking that tool for `delete pods` runs `kubectl get delete pods`, which fails as it should. **Keep the list short.** A normal machine carries around a thousand executables on `PATH`. A tool each would bury the catalog and push the useful ones behind tool search. Name the ones that come up by name in conversation; let the rest fall through to `os-run`. `doctor` reports an entry whose binary `allow` does not admit, one with an empty description, and two entries that would resolve to the same name — a published tool that is then refused is worse than a missing one, because a model picks it from the description and the failure lands mid-conversation. ## What this is not It is not a sandbox, and the allowlist does not pretend to bound capability. An allowlisted binary is trusted with everything that binary can do. `git log` cannot write to a repository, but `git` with the right flags runs a pager, and any binary carrying an `--exec`-style flag will do what that flag says. What the list buys is that the set of programs is finite, declared and auditable — not that every invocation inside it is harmless. Listing `kubectl` bare on a machine holding production credentials means a chat message reaches production. That is a choice made per entry, which is why the granularity exists and why the default is an empty list. ## What always holds Whatever the lists say, these do not change: * **argv, never a shell.** Arguments reach the program as a token list. `|`, `&&`, `;` and `$(…)` in an argument are literal characters. * **A name, not a path.** Resolution goes through `PATH`; `/bin/sh`, `./sh` and anything carrying `..` are refused before any list is consulted. * **Supervised**, with the configured deadline and kill-tree on expiry. * **Audited** as `os_command_invoked` at `Critical`, recording the binary and the full argument list — the half that no config declared in advance. * **Output capped** at 8 KB, so one `rg` over a large tree cannot evict the conversation it was called from. ## See also * [Configuration reference](/guides/config-reference#os) — every key * [MCP reference](/reference/mcp#os-tools) — the tool surface * [Telegram](/concepts/telegram#running-a-command-yourself) — the `!` prefix in a chat * [Local client API](/reference/local-api) — the same prefix over the socket * [Threat model](/security/threat-model) — V17, and what it does not bound # Daemon lifecycle > Install, start, stop, reload, diagnose. macOS launchd + Linux systemd, side-by-side. The dotagent daemon is **one process per user**. The OS (launchd on macOS / systemd on Linux) keeps it alive. dotagent never installs itself — `dotagent install` only generates the unit file; you load it into the OS scheduler with one extra command. If anything below isn't behaving as expected, the [Diagnostics](#diagnostics) section at the bottom has the canonical "is this thing on?" check. ```mermaid flowchart LR A[dotagent install] -->|writes file| U["unit file
(.plist or .service)"] U -->|loaded into| OS["OS scheduler
(launchd / systemd)"] OS -->|spawn + supervise| D[dotagent daemon] D -->|writes| PID["state/daemon.pid"] PID -.->|read by| R[dotagent reload] R -->|SIGHUP| D ``` *** ## TL;DR ```bash # 1. Generate the unit file. dotagent install # 2. Load it into the OS scheduler (one of these two). launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist # macOS systemctl --user daemon-reload && systemctl --user enable --now run.avelino.dotagent # Linux # 3. Verify. dotagent status cat ~/.config/dotagent/state/daemon.pid ``` *** ## Generate the unit file ```bash dotagent install ``` Writes a single unit, regardless of how many agents you have: | Platform | File | | -------- | ----------------------------------------------------- | | macOS | `~/Library/LaunchAgents/run.avelino.dotagent.plist` | | Linux | `~/.config/systemd/user/run.avelino.dotagent.service` | The unit's `ExecStart` / `ProgramArguments` points at the **currently-running `dotagent` binary** (resolved via `std::env::current_exe`). If you move the binary after install, re-run `dotagent install` so the unit refreshes. Both unit files set: | Property | macOS (`launchd`) | Linux (`systemd`) | | -------------- | ------------------------------------------- | ------------------------- | | Auto-restart | `KeepAlive=true` | `Restart=always` | | Start at login | `RunAtLoad=true` | (enable with `--now`) | | Throttle | `ThrottleInterval=10` | `RestartSec=10` | | stdout capture | `StandardOutPath=…run.avelino.dotagent.log` | `StandardOutput=append:…` | | stderr capture | `StandardErrorPath=…-error.log` | `StandardError=append:…` | > The stderr file catches **crashes**, not activity. The daemon only mirrors its `tracing` stream to stderr when stderr is a terminal — otherwise it would grow forever in a file no rotation policy covers. Read `logs/daemon/dotagent.log` to see what the daemon is doing; read `…-error.log` to find out why it died. `DOTAGENT_LOG_STDERR=1`/`0` forces the mirror on or off. Templates live in [`crates/dotagent-unit-gen/templates/`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-unit-gen/templates/README.md). > **Note**: `dotagent install` accepts `--all` and a positional `NAME` for backwards compatibility with the legacy per-agent install flow. Both are **no-ops** now — the daemon manages every discovered manifest internally. The CLI prints a notice to remind you. *** ## Start ### macOS (launchd) ```bash launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` What this does: registers the plist with launchd's `gui/` domain (the user session domain) and immediately spawns the daemon since `RunAtLoad=true`. Verify: ```bash launchctl print "gui/$(id -u)/run.avelino.dotagent" 2>&1 | head -20 # Should print "state = running" plus the spawned PID. ``` > **`brew services start dotagent`** (coming soon, once the tap publishes) is a friendlier alternative — it runs `launchctl bootstrap` for you. ### Linux (systemd user units) ```bash systemctl --user daemon-reload systemctl --user enable --now run.avelino.dotagent ``` `daemon-reload` makes systemd re-scan `~/.config/systemd/user/`. `enable --now` does two things: marks the unit to auto-start at the next login AND starts it immediately. For the unit to **actually survive logout**, you need lingering enabled (otherwise systemd kills user sessions when you log out): ```bash loginctl enable-linger $USER ``` Verify: ```bash systemctl --user status run.avelino.dotagent # Should print "Active: active (running)". ``` *** ## Stop ### macOS ```bash launchctl bootout "gui/$(id -u)/run.avelino.dotagent" ``` `bootout` is the inverse of `bootstrap` — unloads the plist from launchd, sends SIGTERM, and removes the daemon from the user domain. The plist file stays put on disk; the daemon just isn't running. > Don't `kill -9` the daemon — `KeepAlive=true` will respawn it within 10s. Either bootout (above) or use `launchctl stop`: > > ```bash > launchctl stop run.avelino.dotagent # one-shot stop; bootstrap stays loaded → will restart > ``` ### Linux ```bash systemctl --user stop run.avelino.dotagent ``` Doesn't disable auto-start at next login. To prevent re-spawn at the next boot, also: ```bash systemctl --user disable run.avelino.dotagent ``` *** ## Restart After upgrading the `dotagent` binary itself, the running daemon is still pointing at the **old in-memory binary**. Replacing the file doesn't change the running process. Restart it explicitly: ### macOS ```bash launchctl kickstart -k "gui/$(id -u)/run.avelino.dotagent" ``` `kickstart -k` sends SIGTERM, waits for exit, then re-spawns. ### Linux ```bash systemctl --user restart run.avelino.dotagent ``` > If you only changed **manifests** (`agent.toml` files) or **`config.toml`**, use `dotagent reload` (SIGHUP) instead — it's cheaper. *** ## Boot orphan reap A daemon that exits through its shutdown path reaps its children and deletes `state/supervisor.json`. A daemon that is `SIGKILL`ed, panics, or is replaced by `launchctl kickstart -k` does neither. Its children survive with **nobody holding their deadline** — the supervisor that would have enforced it died with the daemon. Observed in production as an agent process alive for 26 minutes against a 600-second timeout. So a starting daemon opens its state store and audit log, and then — before it starts a supervisor of its own — looks at what the last one left behind: ```mermaid flowchart TD S([daemon starts]) --> E{"state/supervisor.json
exists?"} E -->|no| G["clean previous exit — nothing to do"] E -->|yes| P{"another daemon
alive (pidfile)?"} P -->|yes| A["abort the sweep
(two daemons is a misconfiguration,
not a licence to kill)"] P -->|no| R{"readable?"} R -->|no| V["signal nothing;
leave the file as evidence"] R -->|yes| C["classify every record"] C -->|identity confirmed| K["SIGTERM the process group,
grace, then SIGKILL"] C -->|any doubt| L["leave it alone"] K --> D[delete the snapshot] L --> D G --> N(["start supervisor, snapshot writer, tick loop"]) D --> N ``` The order matters: the reap runs **before** the snapshot writer starts, because the writer's first tick would overwrite the only record of those processes. ### It refuses to kill on doubt The OS recycles pids. A pid read off disk with no further proof eventually names somebody else's process, so every record is corroborated against what the OS reports for that pid *right now*. A record is reaped only when **all** of these hold: | Check | What it rules out | | ------------------------------------------------- | -------------------------------------------------------------------------------------- | | pid > 1, and not our own | signalling init or ourselves | | the record carries `pgid` and `command` | a snapshot from a build that predates identity checking | | recorded `pgid == pid` | a record this supervisor did not produce — every supervised spawn does `setpgid(0, 0)` | | the OS still reports the pid | a process that already exited (the common case) | | observed `pgid == pid` | a recycled pid that is not its own group leader | | observed start time within 5s of the recorded one | a process that inherited the number after the old daemon died | | observed command consistent with the recorded one | a recycled pid running something else entirely | Anything missing, ambiguous, or unparseable is a **skip**, never a kill. Leaving one orphan alive costs memory; killing the wrong process costs somebody's work. A snapshot that cannot be read or parsed signals nothing at all and is left on disk, since a corrupt registry is exactly when guessing is most expensive. Confirmed orphans get the same treatment a live deadline expiry gets: `SIGTERM` to the whole process group, one grace window, then `SIGKILL`. Each one is logged at `warn` with the agent, label, pid, pgid, and how far past its deadline it ran, and audited as [`orphan_reaped`](/reference/paths#auditlog) at `critical` — a process that outlived its deadline unsupervised is worth an out-of-band notification, not just a log line. Skips are logged at `debug` with the reason (`already_gone`, `start_time_mismatch`, `command_mismatch`, `not_group_leader`, `ambiguous_record`, `unusable_record`) and are never audited: refusing to signal is the expected outcome, not an event. ### Rollout `pgid` and `command` are new fields in the snapshot. A `state/supervisor.json` written by an older binary has neither, so every record in it classifies as `ambiguous_record` and nothing is killed. The reap becomes effective from the **second** restart after the upgrade — the first one is what writes a snapshot the next boot can actually check. *** ## Reload (SIGHUP) ```bash dotagent reload ``` Reads `~/.config/dotagent/state/daemon.pid`, sends `SIGHUP` to that process. The daemon picks up changes on its next tick: * New manifests in `~/.config/dotagent/agents/` * Modified manifests (drift is detected and audited) * Updated `config.toml` * Updated plugin binaries (resolution is re-done per tick) What `reload` does NOT do: * **Doesn't swap the binary.** A SIGHUP'd process keeps its in-memory code. For binary swaps, use `restart` (above). * **Doesn't drain in-flight runs.** Currently-spawned agents keep running until they exit. * **Doesn't cut short an in-flight answer.** Retiring [persistent](/concepts/lifecycle) instances waits for the slot each one holds, so an instance answering a trigger when the signal lands is retired after that answer is delivered. The wait is bounded by the agent's `timeout_seconds`. `dotagent reload` itself returns as soon as the signal is sent — it is the daemon-side effect that waits. * **Doesn't reset the audit chain.** The new tick continues from the current `prev_hash`. Errors: | Error | Meaning | | ----------------------------------- | -------------------------------------------------------------------------- | | `reading ... (daemon not running?)` | `state/daemon.pid` is missing. Daemon isn't running. | | `sending SIGHUP: No such process` | PID file is stale (daemon crashed without `Drop` cleanup). Start it again. | A reload replaces the **whole** in-memory config, not just the parts the ingress reads. Retention thresholds and the daily summary's time and destinations were pinned at boot until recently; editing `config.toml` and reloading now takes effect on the next tick, as the table above says it should. *** ## What wakes the daemon The daemon sleeps until the earliest thing it has to be awake for. Two independent reasons produce a wake-up, plus a safety cap: ```mermaid flowchart LR A["next agent window
(across every schedule)"] --> M{"earliest"} B["daily summary
at [daily_summary].time"] --> M C["safety cap
MAX_SLEEP = 30min"] --> M M --> S["sleep until it"] S -.->|SIGHUP / SIGTERM / SIGINT| S ``` The safety cap is what picks up a manifest dropped into `agents/` without a reload — within 30 minutes even if nothing is scheduled. An **inbound trigger is not on this list**, and that is the point. A chat message or an MCP tool call is drained by a worker task running beside the loop, so it is answered on its own clock instead of waiting for the loop to come back around: ```mermaid flowchart LR subgraph daemon L["tick loop
(awaits each scheduled run inline)"] W["trigger worker
(one task, FIFO, awaits each request)"] end ING(["telegram poller / MCP"]) -->|bounded channel, 64| W CLK([clock]) --> L ``` Triggers were an arm of the loop's `select!` until recently, which meant a scheduled run with a 20-minute deadline held every queued message for its whole duration. Now only triggers queue behind each other. See [Triggers → Serialization](/concepts/triggers#serialization). The daily summary is a wake-up reason **in its own right**: it is the one scheduled thing the daemon does that no agent schedule accounts for. Before that was made explicit it only landed inside its window by coincidence — the 30-minute sleep cap happened to match the 30-minute grace window, two unrelated constants with nothing connecting them. That coincidence broke the moment a tick overran its own sleep budget, and it never held at all for a `time` / `grace_minutes` someone picked themselves. `dotagent tick --dry-run` reports the **agent** next event only, so the real next wake-up can be earlier than what it prints when the summary is closer. See [`[daily_summary]`](/guides/config-reference#daily_summary). ### The background tasks do not add wake-ups Two supervisor tasks run beside the loop, and neither polls: | Task | While something is supervised | While nothing is | | --------------- | --------------------------------- | ---------------------- | | reaper | sleeps until the nearest deadline | parked | | snapshot writer | rewrites the file every 2s | parked after one write | Both park on the same signal, which a spawn — or a `retime` that pulls a deadline in — raises. A daemon supervising nothing therefore holds **no timers at all**: it is asleep until its next scheduled event, and nothing underneath it is ticking. This matters on a laptop, where periodic work is charged twice: once for the CPU, and again for keeping the machine out of deep idle. The reaper formerly swept on a fixed 5-second tick (\~17k wake-ups a day) and the snapshot writer rewrote a byte-identical `[]` every 2 seconds (\~43k writes a day) to record that nothing had changed. The reaper losing its tick also made it *more* precise, not less: it now fires on the deadline instead of up to one tick after it. ### Power Scheduled runs can be held back while the machine is on battery — see [`[power]`](/guides/config-reference#power). The check happens per dispatch, not per wake-up: the daemon still wakes on schedule, decides nothing is to be dispatched, and goes back to sleep. Deferred runs leave no trace, so plugging the charger back in dispatches the current window as if it had just come due. *** ## Uninstall Stop the daemon first, then remove the unit: ```bash # 1. Stop the running daemon (so the file we delete isn't actively in use). launchctl bootout "gui/$(id -u)/run.avelino.dotagent" # macOS systemctl --user disable --now run.avelino.dotagent # Linux # 2. Remove the unit file. dotagent uninstall # → removed ~/Library/LaunchAgents/run.avelino.dotagent.plist # (or systemd unit on Linux) ``` `dotagent uninstall` is idempotent — running it twice doesn't error, the second call just prints "nothing to remove". **`dotagent uninstall` does NOT delete your data.** Manifests, heartbeats, audit log, config — all stay in `~/.config/dotagent/`. See [`installation.md`](/getting-started/installation#uninstall) for a full wipe. *** ## Diagnostics ### "Is the daemon running?" ```bash # 1. PID file exists? ls -l ~/.config/dotagent/state/daemon.pid # → -rw-r--r-- ... daemon.pid # 2. PID is alive? ps -p $(cat ~/.config/dotagent/state/daemon.pid) -o command= 2>/dev/null # → dotagent daemon ``` If either step fails: * File missing → daemon never started (or was stopped cleanly). * PID exists but `ps` empty → stale pidfile (daemon crashed without the `Drop` guard firing). Just start the daemon again. ### Platform-native checks **macOS**: ```bash launchctl print "gui/$(id -u)/run.avelino.dotagent" 2>&1 | head -30 # state = running ← the line that matters # program = .../dotagent # arguments = .../dotagent → daemon ``` **Linux**: ```bash systemctl --user status run.avelino.dotagent # Active: active (running) ← the line that matters ``` ### "What's the daemon doing right now?" Tail the structured log: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq -c . # {"timestamp":"...","level":"INFO","fields":{"message":"daemon started"}} # {"timestamp":"...","level":"INFO","fields":{"message":"dispatching run","agent":"hello","schedule":"every-2min"}} ``` That is the file to watch. `dotagent.log` is where the daemon's whole `tracing` stream goes, and under launchd / systemd it is the **only** place it goes. The captured stdout/stderr files are a different thing: ```bash tail -F ~/.config/dotagent/logs/daemon/run.avelino.dotagent.log # stdout tail -50 ~/.config/dotagent/logs/daemon/run.avelino.dotagent-error.log # crashes only ``` `…-error.log` is **not** an activity log. The daemon installs its stderr `tracing` layer only when stderr is a terminal, because under a service manager stderr is a plain file that nothing rotates — an unbounded log growing next to the rotated one it duplicates. So the error file receives panics, and startup failures that happen before logging is up, and nothing else. A daemon that is running normally leaves it empty, which is the correct outcome and not a symptom. Override with `DOTAGENT_LOG_STDERR=1` (force the mirror on — useful when the unit is rewired to journald, which does rotate) or `DOTAGENT_LOG_STDERR=0` (force it off even in a terminal). Health dashboard: ```bash dotagent status ``` ### "What will the daemon do next?" ```bash dotagent tick --dry-run # (dry-run) scanned 4 agent(s); would dispatch 1; next event: 2026-05-19T08:30:00-0300 ``` That `next event` timestamp is the next **agent** window. The daemon wakes at whichever comes first between it, the daily summary, and the 30-minute safety cap — see [What wakes the daemon](#what-wakes-the-daemon). ### "Did the audit log break?" The daemon verifies the chain on startup and emits `AuditChainBroken` (with notify) if it fails. It checks the **live segment** — the only file that changes. Manual check: ```bash # Walk the file; each line's prev_hash must be sha256 of the previous line. # In practice, just look for the audit_chain_broken event: grep audit_chain_broken ~/.config/dotagent/audit.log # (no output = chain intact) ``` Past 32MB the log rotates: the live file becomes `audit.log.` and a fresh one opens with a **seam** (`audit_log_rotated`) whose `prev_hash` is the old file's tail hash, so the chain has no gap at the rename. Rotation is normal and verifies clean: ```bash ls ~/.config/dotagent/audit.log* # live file plus any segments grep audit_log_rotated ~/.config/dotagent/audit.log | jq . ``` Segments are **never deleted by dotagent** — the log sweeper in `[logs]` does not touch them. If you prune them yourself, verification still reads as intact and reports how far back it reaches. What it will *not* forgive is cutting lines off the head of the live file: that removes the seam too, and the orphaned hash left behind is what fires `audit_chain_broken`. Details in [`security/threat-model.md`](/security/threat-model#what-the-hash-chain-guarantees-and-what-it-does-not). *** ## Signal reference The daemon process responds to: | Signal | Effect | | --------- | ---------------------------------------------------------------------------------- | | `SIGHUP` | Wake immediately; re-read manifests + plugins on the next tick. `dotagent reload`. | | `SIGTERM` | Graceful shutdown. Drops `daemon.pid`. Audit gets `DaemonStopped`. | | `SIGINT` | Same as SIGTERM. | | `SIGKILL` | Immediate kill — no `Drop` runs → stale pidfile. Auto-restart via launchd/systemd. | Don't `kill -9` unless you have to — let `launchctl bootout` / `systemctl --user stop` do the work. *** ## Common patterns ### Run the daemon manually (development) For debugging the daemon itself, bypass launchd/systemd: ```bash RUST_LOG=debug dotagent daemon # Foreground; Ctrl+C to stop. Same code path as the supervised daemon. ``` Don't do this while the supervised daemon is also running — they'll both try to write `daemon.pid` and step on each other. ### Run on a non-standard root ```bash DOTAGENT_HOME=/tmp/sandbox dotagent install DOTAGENT_HOME=/tmp/sandbox launchctl bootstrap "gui/$(id -u)" \ ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` The unit file inherits the env var of the shell that ran `install` — **not great** for permanence. Better: set the env var inside the unit file directly (`EnvironmentVariables` for launchd / `Environment=` for systemd). ### Make config / env changes survive restart If you need persistent overrides (custom `DOTAGENT_HOME`, custom `OTEL_EXPORTER_OTLP_HEADERS`, etc.) edit the unit file: **macOS** (`~/Library/LaunchAgents/run.avelino.dotagent.plist`): ```xml EnvironmentVariables DOTAGENT_HOME /var/lib/dotagent OTEL_EXPORTER_OTLP_HEADERS x-honeycomb-team=YOUR_KEY ``` After editing, reload: ```bash launchctl bootout "gui/$(id -u)/run.avelino.dotagent" launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` **Linux** (`~/.config/systemd/user/run.avelino.dotagent.service`): ```ini [Service] Environment=DOTAGENT_HOME=/var/lib/dotagent Environment=OTEL_EXPORTER_OTLP_HEADERS=x-honeycomb-team=YOUR_KEY ``` After editing: ```bash systemctl --user daemon-reload systemctl --user restart run.avelino.dotagent ``` > The unit file is **regenerated** by `dotagent install` — your manual edits are lost if you re-run it. Long-term, treat the unit file as a generated artifact and keep custom envvars in `config.toml` instead where possible. *** ## Related * [`installation.md`](/getting-started/installation) — install paths (brew, release, cargo, source) * [`cli.md`](/reference/cli) — `install`, `uninstall`, `reload` * [`troubleshooting.md`](/guides/troubleshooting) — sintoma → diagnostic * [`observability.md`](/guides/observability) — log streams + OTel * [`paths.md`](/reference/paths) — every file the daemon touches # Troubleshooting > Sintoma → diagnostic → fix. Top-down by where the problem lives. The decision tree: ```mermaid flowchart TD A[Something's wrong] --> B{Daemon running?} B -->|no| C[Section: Daemon won't start] B -->|yes| D{Doctor passes?} D -->|no| E[Section: Doctor errors] D -->|yes| F{Agent firing?} F -->|no| G[Section: Agent never runs] F -->|yes| H{Run succeeds?} H -->|no| I[Section: Agent runs but fails] H -->|yes| J{Notify / sink working?} J -->|no| K[Section: Notifier / sink not working] ``` When in doubt, start with `dotagent doctor` + `dotagent status` — they're read-only and give the most signal per second. *** ## Daemon won't start ### Symptom: `dotagent reload` says "daemon not running" ```bash ls -l ~/.config/dotagent/state/daemon.pid ``` * **File missing** → daemon was never started (or was bootouted). Start it: * macOS: `launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist` * Linux: `systemctl --user enable --now run.avelino.dotagent` * **File present** but `ps -p $(cat …)` returns nothing → stale pidfile from a crashed daemon. Delete it and start again: ```bash rm ~/.config/dotagent/state/daemon.pid ``` ### Symptom: launchd / systemd loads the unit but the daemon dies immediately This is exactly what the captured stderr file is for — a daemon that dies before or during startup has nowhere else to say why: ```bash tail -50 ~/.config/dotagent/logs/daemon/run.avelino.dotagent-error.log ``` > Only crashes land there. Under launchd / systemd the daemon does not mirror its `tracing` stream to stderr (stderr is an unrotated file there, and the stream already goes to `dotagent.log`). So an **empty** `…-error.log` on a daemon that keeps dying means it is not crashing at all — check `logs/daemon/dotagent.log` and `dotagent status` instead. To force the mirror on anyway, set `DOTAGENT_LOG_STDERR=1` in the unit's environment. Common causes: | Error in stderr | Fix | | ------------------------------------- | ------------------------------------------------------------------------------------ | | `telemetry init failed: ...` | Bad `[telemetry]` config. Either fix `config.toml` or delete the section. | | `config.toml parse error: ...` | TOML syntax issue. Validate with `toml-cli get config.toml .` or remove the file. | | `No such file or directory: dotagent` | The unit's `ExecStart` points at a binary that was moved. Re-run `dotagent install`. | | `permission denied` | The dotagent binary lost the executable bit. `chmod +x` it. | ### Symptom: macOS — unit loads but `launchctl print` shows `state = exited` ThrottleInterval (10s) is in play — launchd waits 10s before respawning. After many crashes in quick succession, launchd will back off further. The real diagnosis is in the error log above. ### Symptom: Linux — `systemctl --user status` shows `Active: inactive (dead)` despite `enable --now` ```bash journalctl --user -u run.avelino.dotagent -n 50 ``` If you see `User lingering not enabled`, the user session is killed at logout. Enable lingering: ```bash loginctl enable-linger $USER ``` *** ## Doctor errors ### `agent.name is empty` The `[agent].name` field is missing or `""`. Open the manifest and add it. ### `run.command is empty` `[run].command` field missing. dotagent needs the command to invoke (e.g., `"fish"`, `"python3"`, `"./agent"`). ### `duplicate schedule id: ` Two `[[schedules]]` blocks have the same `id`. Schedule ids must be unique **within a single manifest** — across different manifests they can repeat. ### `✗ plugin not found` A `[[preflight]]`, `[[on_success]]`, `[[on_failure]]`, or `[[notifiers]] driver = "plugin"` references a plugin binary that isn't on `$PATH`. ```bash # Where would dotagent look? echo $DOTAGENT_PLUGIN_PATH # custom (if set) ls ~/.config/dotagent/plugins/ # user-local ls /usr/local/lib/dotagent/plugins/ # system-wide which dotagent-plugin-sink-roam # $PATH ``` If absent on all four: * Brew install: `brew install dotagent` (when tap publishes) ships every first-party plugin. * Cargo install: `cargo install --path plugins/sink-roam` (or whichever). * Release binaries: re-extract the tarball into a `$PATH` directory. ### `⚠ : no [security] section` Warning, not error. v0 is schema-only. Add a minimal block to silence it: ```toml [security] # Document intent — enforcement comes in a future release. network = "allow" ``` See [`security/threat-model.md`](/security/threat-model). ### `⚠ : manifest drift since last daemon run` The on-disk manifest's sha256 doesn't match what's cached in `state/known_manifests.json` from the last daemon load. Cause is one of: * **You edited it intentionally.** Run `dotagent reload`. The new hash gets cached. * **You DIDN'T edit it.** Drift detection is doing its job — investigate. Compare timestamps: ```bash stat ~/.config/dotagent/agents//agent.toml git log -p ~/dotfiles/agents//agent.toml # if version-controlled ``` See [V1 in the threat model](/security/threat-model#v1--manifest-hijack). *** ## Agent never runs ### Symptom: `dotagent doctor` says manifest is fine but the daemon never fires the agent Start with the dry-run: ```bash dotagent tick --dry-run # → (dry-run) scanned N agent(s); would dispatch M; next event: ``` If `dispatched = 0` and you expected dispatch: * The current window has already succeeded — heartbeat says `last_success_at >= expected_at`. Run `dotagent inspect ` to confirm. * The agent's `monitor = false` excludes it from the daemon. Set `monitor = true` (or remove the line — default is true). * The schedule's window is in the future. `next event` shows when. ### Symptom: `monitor = false` agents fire when I don't want them to `monitor = false` excludes the agent from `tick` / daemon dispatch, but NOT from `run` / `run-now`. The example `hello-*` agents use this so they only run when you explicitly invoke them. ### Symptom: agent fires but with the wrong `args` `AGENT_ARGV` is the schedule's `args`, NOT `[run].args`. The full argv the agent receives is `[run].command + [run].args + schedule.args`. ```toml [run] command = "python3" args = ["./agent.py", "--mode", "prod"] # always present [[schedules]] id = "weekly" args = ["--period", "last-week"] # appended at runtime ``` The Python script is invoked as `python3 ./agent.py --mode prod --period last-week`. Inside the script, `AGENT_ARGV = ["--period","last-week"]`. ### Symptom: schedule type "interval" doesn't fire on a fresh install Interval schedules with no previous run start "from now". A `interval_minutes = 60` agent installed at 14:32 next fires at 15:32. Set a smaller interval (or `dotagent run-now`) to verify. ### Symptom: cron-style schedule never matches ```toml [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] hours = [8] minute = 30 ``` `weekdays`: **0 = Sunday, 6 = Saturday** (matches launchd Weekday). Many folks expect 0 = Monday — that's cron(8), not us. dotagent matches launchd. Quick sanity: ```bash dotagent tick --dry-run # Compare "next event" with your expectation. ``` *** ## Agent runs but fails ### Symptom: `exit_code = 124` The agent was killed for exceeding `agent.timeout_seconds`. dotagent sends SIGTERM, waits 5 seconds, then SIGKILL. Either: * Bump `timeout_seconds` in the manifest. * Fix the agent (most timeouts are an external CLI hanging — wrap with `--timeout`). `timeout_seconds` is the **backstop**, not the first line of defense. It kills a wedged run; it does nothing to stop one slow network call from eating the whole budget and leaving the agent no time to finish the work it already did. Put a ceiling on the call itself — `curl --max-time 20`, an HTTP client timeout, `timeout 30 ` — so a single unresponsive endpoint degrades one step instead of the run. ### Symptom: agent succeeded once but the daemon keeps re-firing it The previous run failed and you fixed it without updating state. The window doesn't yet show `succeeded_at`. Run `dotagent run-now ` to force a fresh success, OR delete the window file: ```bash rm ~/.config/dotagent/state/windows/--.json ``` The daemon will recompute on the next tick. ### Symptom: `dotagent run` works but the agent fails when the daemon fires it Almost always an **environment difference**. The interactive shell has `$PATH` / `$HOME` / `$LANG` you take for granted; the daemon inherits launchd / systemd's much-poorer env. Check what the daemon sees: ```bash # macOS — what env launchd hands to the daemon launchctl print "gui/$(id -u)/run.avelino.dotagent" 2>&1 | grep -A 20 environment # Linux systemctl --user show-environment ``` Set explicit `$PATH` in the agent's manifest: ```toml [env.extra] PATH = "/usr/local/bin:/usr/bin:/bin:/opt/homebrew/bin" ``` Or `inherit = true` plus tighten the unit-level env vars in the plist. ### Symptom: preflight aborts the run but I want to see why ```bash tail ~/.config/dotagent/audit.log | jq 'select(.event.event_type == "preflight_failed")' ``` The `suggest` field is the message the plugin returned ("warp-cli connect", etc.). To run the preflight manually: ```bash echo '{ "kind": "preflight", "agent": "test", "schedule": "test", "event": "preflight", "config": { "connect_command": "warp-cli connect" } }' | dotagent-plugin-preflight-warp invoke ``` *** ## Notifier / sink not working ### Desktop notifier doesn't show banners (macOS) System Settings → Notifications → make sure the **terminal app running the daemon** (and possibly `dotagent` itself) is allowed. notify-rust uses `NSUserNotification`, which inherits the parent process's notification entitlements. For launchd-managed daemons, the entitlement lives with the daemon binary itself — TCC may show `dotagent` as the requestor first time. ### Desktop notifier doesn't show banners (Linux) `notify-rust` uses D-Bus. Verify the daemon process can talk to your notification daemon: ```bash notify-send "test" "hello" ``` If `notify-send` shows nothing, no D-Bus session is reachable. Common causes: * Headless / SSH session with no `$DBUS_SESSION_BUS_ADDRESS`. * systemd user unit started before the graphical target — set `Wants=graphical-session.target` in the unit. ### iMessage notifier doesn't send `imessage` runs `osascript` to talk to Messages.app. Verify manually: ```bash osascript -e 'tell application "Messages" to send "test" to buddy "+5511..."' ``` If that fails, automation isn't permitted: System Settings → Privacy & Security → Automation → Terminal (or `dotagent`) → toggle Messages on. The `imessage` driver also rate-limits via `$DOTAGENT_HOME/state/notify/imessage/.json`. If you're sending within `rate_limit_minutes` of a previous successful send, the call is **skipped** (not failed). Look for it in the audit log: ```bash tail ~/.config/dotagent/audit.log \ | jq 'select(.event.plugin == "notifier:imessage")' ``` ### Slack / ntfy / Pushover / Telegram notifier doesn't fire Native HTTPS — check connectivity from the daemon's environment: ```bash curl -v -X POST https://hooks.slack.com/services/T0000/B0000/SECRET \ -H 'Content-Type: application/json' \ -d '{"text":"test"}' ``` If that succeeds but dotagent's notifier doesn't, check the audit log for a `Backend(...)` error: ```bash tail ~/.config/dotagent/audit.log \ | jq 'select(.event.event_type == "plugin_invoked" and (.event.plugin | startswith("notifier:")))' ``` Three causes account for most of it: **`: env var ${X} is unset`.** A `${VAR}` in a credential field did not resolve. dotagent fails the send rather than posting the literal placeholder, so nothing arrives. Check that the key is in `~/.config/dotagent/secrets.env`, that the file is mode `0600` (otherwise the daemon refuses the whole file), and that you sent `SIGHUP` after editing it — the store is read at startup and on reload, not per send. ```bash dotagent doctor # reports the secrets file's state dotagent reload # SIGHUP: re-read secrets.env + config.toml ``` **` transport error (...)`.** Network or TLS. The message carries a kind and, when there was one, an HTTP status — deliberately **not** the URL, because for Slack and Telegram the URL is the credential. Reproduce with the `curl` above from the daemon's own environment (a proxy set in your shell but not in the plist is a classic). **Nothing arrived, no error.** Check the event filter first — `events = ["given_up"]` on an agent that keeps recovering never fires. Then check size: an over-limit body is now trimmed with a `[truncated]` marker rather than rejected, so if you see a truncated message that is the cap doing its job (ntfy counts **bytes**, not characters — see [notifications](/concepts/notifications#message-size-limits)). For Telegram specifically, an API refusal now carries the Bot API's own `description` (`"Bad Request: message is too long"`, `"chat not found"`), which is usually the whole answer. ### Sink plugin appears not to run Sinks fire **only on agent exit 0**. If the agent failed, only the notifier path runs. ```bash # Did the agent succeed? cat ~/.config/dotagent/state/agents//.heartbeat.json | jq .exit_code # Did the sink get invoked? tail ~/.config/dotagent/audit.log \ | jq 'select(.event.event_type == "plugin_invoked" and .event.plugin == "sink-roam")' ``` ### `sink-roam` writes but the previous block doesn't get replaced Your `marker_regex` doesn't match the old root block. See [`plugins/sink-roam.md`](/built-in-plugins/sink/sink-roam#block-keeps-duplicating-after-re-runs). *** ## A manifest is broken ### One agent stopped running and nothing said why Check for a parse failure: ```bash dotagent doctor 2>&1 | grep '✗' grep manifest_invalid ~/.config/dotagent/state/audit.log | tail -5 ``` A manifest that fails to parse or validate is skipped, audited as `manifest_invalid` (Critical, so it fires out-of-band notification), and the other agents keep running. > Before this behavior existed, one broken file aborted the entire scan: the daemon saw **zero** agents, dispatched nothing, and the only trace was a single `warn!` line. If you are on an older build and everything stopped at once, look for a manifest you edited recently. *** ## Inbound Telegram ### The bot never answers Check whether the ingress even started: ```bash dotagent doctor | grep -i telegram ``` `telegram ingress: OFF — bot_token set but allowed_user_ids is empty` is the most common cause. An empty allowlist means **nobody**, never everybody. Add your numeric user id (from [@userinfobot](https://t.me/userinfobot)) — a `@username` will not work, it is not an authorization input. If the ingress is on, your message may be getting refused: ```bash grep trigger_rejected ~/.config/dotagent/state/audit.log | tail -5 ``` `reason: "user id not in allowed_user_ids"` means the id in `config.toml` is not the one that sent the message. `reason: "rate limit exceeded"` means you passed `rate_limit_per_minute` (default 10). ### `telegram poll failed` repeating in the log Transport problem — network down, or a revoked/wrong token. The backoff doubles to a 60-second ceiling, so this will not spin. Verify the token resolves: ```bash dotagent doctor | grep -A3 "secrets file" ``` An unresolved `${TELEGRAM_BOT_TOKEN}` shows up there as an unresolved reference. ### The run happens but no reply arrives The dispatcher printed nothing on stdout, or logged to stdout instead of stderr and the agent then failed. stdout **is** the reply: ```bash dotagent logs telegram-assistant -n 50 ``` ### The reply is cut off mid-sentence Telegram caps a message at 4096 characters. Longer output is trimmed with a `[truncated]` marker; the full text is in `dotagent logs`. ### `dispatcher agent '' not found` `doctor` says this when `dispatcher_agent` in `config.toml` does not match any installed agent. Every accepted message would fail after passing the allowlist. *** ## MCP server ### `tools/list` returns an error instead of tools One or more `agent.toml` files failed to load. The healthy agents are fine — the daemon still runs them — but the MCP catalog refuses to be served incomplete, because a model handed a list that quietly lost entries would answer confidently and wrongly about what it can do. The error names each failing path. `dotagent doctor` shows the same list with the parse messages. ### An agent is missing from `tools/list` Two agent names can sanitize to the same tool name (`a.b` and `a/b` both become `run-a-b`). The first one discovered wins and the log says: ``` tool name collides with an earlier agent — skipping ``` Rename one of them. ### Runs started via MCP don't show in `dotagent status` Expected. `dotagent mcp` runs agents in its own process, like `run-now`; `status` reflects only the daemon's supervisor. Their heartbeats live under the `trigger-mcp` slug, visible via `dotagent inspect`. *** ## Logs ### `dotagent logs ` says "no logs found" The agent has never run, or the log directory was deleted. Run: ```bash dotagent run-now ls ~/.config/dotagent/logs/agents// ``` ### Logs are noisy / full of `tracing` debug spam ```toml # config.toml [logging] level = "warn" ``` Or transient: ```bash RUST_LOG=warn dotagent daemon ``` ### Disk filling despite retention being set * `retention_days` in `config.toml` only kicks in during the 03:00 sweep. If the daemon hasn't been alive at 03:00 since the last rollover, nothing got swept yet. * The sweep needs write permission on `logs/agents//`. If your agent script chowns its log directory (don't), the sweep fails silently. * The audit log is **never** swept. It rotates at 32MB into `audit.log.` segments, and those segments stay forever — it is the only forensic artifact dotagent keeps, so pruning it is your call, not a sweeper's. Plan for \~1KB per event, roughly a hundred events/day. * `state/windows/` is swept by the same 03:00 pass, on `[state] window_retention_days` (default 30). A 15-minute agent writes \~96 files a day there, so a daemon that never sees 03:00 accumulates thousands. A window whose `.lock` is currently held is skipped — it gets collected on the next pass. *** ## Audit log ### Ask the log what it can still prove ```bash dotagent audit verify --full ``` Start here for anything in this section. It prints one of four verdicts — intact from `GENESIS`, intact since a named rotation, unexplained truncation, or broken at a position — and exits non-zero for the last two. `--json` for scripts. Without `--full` it checks only the live `audit.log`, which is what the daemon does at boot. Full output reference: [`reference/cli.md`](/reference/cli#audit-verify). ### `audit_chain_broken` event in the log The daemon detected tampering (or a partial write) of `audit.log`. The event is itself a chained entry — investigation: ```bash grep audit_chain_broken ~/.config/dotagent/audit.log | jq . # {"event":{"event_type":"audit_chain_broken","position":42,...}} ``` `position` is the line number where the chain broke. Read context: ```bash sed -n '40,45p' ~/.config/dotagent/audit.log | jq . ``` Cause is almost always: * Someone (or you) edited the file by hand. * A crash mid-write left a half-line, or bytes cut inside a multibyte character. dotagent steps over that garbage on the next append rather than refusing to write — the line stays on disk and verification keeps reporting it, which is the loud half of the trade. * Disk corruption. * The head of the file was removed — see the next section, which is the one case that is easy to cause by accident. dotagent continues operating — the new chain is anchored to the broken position. Forensics is on you; `dotagent audit verify --full` names the position and the segment. ### `position: 0` with `expected_prev_hash: "GENESIS"` The first line of `audit.log` does not chain to `GENESIS`, and no **seam** explains why. Verification calls this *unexplained truncation*: somebody removed the head of the log. The legitimate way for the file to start somewhere other than `GENESIS` is rotation, which leaves a seam as line 1: ```bash head -1 ~/.config/dotagent/audit.log | jq . # {"event":{"event_type":"audit_log_rotated","rotated_to":"audit.log.20260806T101500", # "entries":38513,"tail_hash":"c8d2..."},"prev_hash":"c8d2..."} ``` If that line is missing, someone `sed`'d or `tail`'d the file. Deleting a whole **segment** is fine and stays quiet — the seam lives in the current file and keeps explaining the gap. Deleting lines from the current file takes the seam with them, and that is what fires here. ### Rotated segments ```bash ls -la ~/.config/dotagent/audit.log* ``` `audit.log` is live; `audit.log.` are sealed segments, oldest first by name. To read the whole history in order: ```bash cat $(ls ~/.config/dotagent/audit.log.* 2>/dev/null) ~/.config/dotagent/audit.log | jq . ``` Removing old segments is supported and does not break verification — it reports "intact since ``" instead of "intact from GENESIS". The reasoning is in [`security/threat-model.md`](/security/threat-model#what-the-hash-chain-guarantees-and-what-it-does-not). *** ## Plugin protocol ### Plugin always returns `ok=false` ```bash # Run the exact payload dotagent would send. echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "smoke", "config": }' | dotagent-plugin- invoke ``` Stderr has the human-readable error. Stdout has the JSON response. ### Plugin works manually but not from the daemon Cause is almost always **environment**: * `$PATH` differences (covered in [Agent runs but fails](#symptom-dotagent-run-works-but-the-agent-fails-when-the-daemon-fires-it)). * `$DOTAGENT_PLUGIN_PATH` not inherited by launchd. Set it in the plist's `EnvironmentVariables`. * HOME differences (the plugin reads `~/.config/` but the daemon's HOME is somewhere unexpected). ### Plugin info JSON is malformed — `dotagent doctor` won't parse it Run the plugin directly: ```bash dotagent-plugin- info | jq . ``` If `jq` errors out, the plugin is printing log lines to stdout (forbidden — stdout is reserved for JSON). Patch the plugin to use stderr for logs. *** ## Performance ### Daemon CPU usage is high The daemon **should** be near-zero CPU outside of a tick. If `top`/`htop` shows persistent CPU: * A plugin is spinning. `dotagent plugin list` then check each. * A schedule with `interval_minutes = 0` or similar — fix it (manifest validation should catch this; file an issue if not). * Verbose tracing flooding the file. `RUST_LOG=info` and retry. ### Agent timeout fires every run Profile the agent outside dotagent (`time fish ./agent.fish`). If it genuinely takes longer than `timeout_seconds`, bump the manifest. If it's fast in your shell but slow under the daemon — that's an environment difference (see env section above). *** ## When all else fails ```bash # 1. State of the daemon dotagent status dotagent tick --dry-run ps -p $(cat ~/.config/dotagent/state/daemon.pid 2>/dev/null) -o command= 2>/dev/null launchctl print "gui/$(id -u)/run.avelino.dotagent" 2>&1 | head -30 # macOS systemctl --user status run.avelino.dotagent # Linux # 2. State of the configured agents dotagent doctor dotagent plugin list for a in ~/.config/dotagent/agents/*/; do dotagent inspect "$(basename $a)" done # 3. Recent events tail -100 ~/.config/dotagent/audit.log | jq -c tail -100 ~/.config/dotagent/logs/daemon/dotagent.log | jq -c # 4. Resource sanity df -h ~/.config/dotagent # disk full? ls -la ~/.config/dotagent/logs/agents/ # rogue agent filling logs? ``` If you still can't pin it down, open an issue with the output of (1) through (4) and a redacted copy of the affected `agent.toml`. *** ## Related * [`cli.md`](/reference/cli) — every subcommand at a glance * [`daemon-lifecycle.md`](/guides/daemon-lifecycle) — start / stop / reload * [`observability.md`](/guides/observability) — log streams + jq recipes * [`paths.md`](/reference/paths) — where every file lives * Plugin-specific troubleshooting under [`docs/plugins/`](/built-in-plugins/plugins) # Config reference (config.toml) > Schema for `~/.config/dotagent/config.toml`. Every field is optional — dotagent ships with sensible defaults out of the box. ``` $DOTAGENT_HOME/config.toml # default: ~/.config/dotagent/config.toml ``` If this file is **missing**, dotagent uses the baked-in defaults (spelled out below). You only need a `config.toml` to: | Want to | Section | | ---------------------------------------------------------------------------- | ----------------------------------- | | Bump verbosity, change log retention | [`[logging]`](#logging) | | Export traces to Honeycomb / Tempo / Datadog | [`[telemetry]`](#telemetry) | | Keep secrets somewhere other than the default path | [`[secrets]`](#secrets) | | **Talk to your agents from Telegram** | [`[telegram]`](#telegram) | | **Move or disable long-term memory** | [`[memory]`](#memory) | | Retime the daily health summary, or send it somewhere other than the desktop | [`[daily_summary]`](#daily_summary) | | **Stop agents from running on battery** | [`[power]`](#power) | Two of those differ in kind. `[logging]`, `[telemetry]` and `[memory]` tune something that already works; `[telegram]` **turns on** a path that does not exist otherwise, and it is the one section that changes the threat model — inbound messages mean untrusted input from the internet can cause a local process to run. There is no required field. Anything you don't write falls back to the default. After editing, run `dotagent reload` — the daemon picks up changes on the next tick. *** ## Full example (every field) ```toml # ~/.config/dotagent/config.toml [logging] level = "info" # off | error | warn | info | debug | trace format = "json" # json | pretty | compact retention_days = 30 # daemon logs older than this are deleted per_agent_retention_days = 14 # agent logs (noisier; shorter horizon) compress_after_days = 1 # rotated files older than N days → gzip [state] window_retention_days = 30 # state/windows/ files older than this are deleted [telemetry] otlp_endpoint = "" # empty = OTel disabled (default) protocol = "grpc" # grpc | http service_name = "dotagent" [telemetry.headers] # Vendor-specific auth headers. Sent on every OTLP request. # OTEL_EXPORTER_OTLP_HEADERS env var wins over this table. "x-honeycomb-team" = "your-api-key" [telemetry.resource] # Resource attributes attached to every span/log. "deployment.environment" = "production" "host.name" = "workstation-01" [secrets] file = "" # empty = default path or DOTAGENT_SECRETS_FILE [telegram] bot_token = "${TELEGRAM_BOT_TOKEN}" # empty = inbound Telegram off (default) allowed_user_ids = [123456789] # numeric ids; empty = nobody dispatcher_agent = "telegram-assistant" poll_timeout_seconds = 30 # long-poll hold, capped at 50 by Telegram rate_limit_per_minute = 10 # accepted messages per sender [memory] enabled = true # default; false removes the memory tools workspace = "" # empty = $DOTAGENT_HOME/outl [skills] enabled = true # default; false removes the skill tools claude_skills = true # default; also read ~/.claude/skills paths = [] # extra roots, searched before the defaults [commands] enabled = true # default; false removes the menu and tools claude_commands = false # default; opt in to ~/.claude/commands paths = [] # extra roots, searched before the defaults [power] on_battery = "run" # default; run | defer min_battery_percent = 0 # default; 0 = charge is never consulted [daily_summary] enabled = true # default; governs the daemon's fire only time = "22:45" # default; local HH:MM or HH:MM:SS grace_minutes = 30 # default; how late a wake-up still delivers [[daily_summary.notifiers]] # Same shape as a manifest's [[notifiers]]. None declared = desktop. driver = "telegram" bot_token = "${TELEGRAM_BOT_TOKEN}" chat_id = "123456789" ``` *** ## `[telegram]` Inbound Telegram. **Off unless configured** — dotagent opens no inbound path you did not ask for. Outbound notifications are unrelated and live in the manifest's `[[notifiers]]`; see [Notifications](/concepts/notifications). | Field | Default | Meaning | | ----------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | `bot_token` | `""` | Bot API token. Accepts `${VAR}`, resolved against the secrets store at poll time. | | `allowed_user_ids` | `[]` | Numeric Telegram user ids allowed to trigger runs. | | `open_chat_ids` | `[]` | Group chats where **any member** may talk to the dispatcher. Direct messages and `!`/`!!` typed commands stay restricted to `allowed_user_ids`. | | `dispatcher_agent` | `"telegram-assistant"` | Agent every accepted message is handed to. | | `poll_timeout_seconds` | `30` | Seconds to hold `getUpdates` open. Telegram caps this at 50. | | `rate_limit_per_minute` | `10` | Accepted messages per sender per minute. | The ingress starts only when `bot_token` **and** at least one entry in `allowed_user_ids` are present. A token with an empty allowlist stays off and says so in the log: reading empty as "no restriction" would turn one forgotten line into an open remote-execution endpoint. This section is daemon-level rather than per-manifest because Telegram allows exactly one `getUpdates` consumer per bot token. N manifests each polling would compete for the same offset and silently drop each other's messages. Enabling this changes the threat model — a message from the public internet can cause a local process to run. Read [V8 in the threat model](/security/threat-model) before turning it on, and [Telegram](/concepts/telegram) for the full setup. *** ## `[memory]` Long-term memory for agents, stored in an embedded [outl](https://github.com/avelino/outl) workspace. On by default. | Field | Default | Meaning | | ----------- | ------- | -------------------------------------------------------- | | `enabled` | `true` | Expose the `memory-*` tools from `dotagent mcp`. | | `workspace` | `""` | Workspace path. Empty resolves to `$DOTAGENT_HOME/outl`. | The default workspace is scaffolded when the daemon starts, so memory works without writing any config. A **configured** path is never scaffolded: a typo there must fail loudly rather than create an empty workspace nobody will look at. `dotagent doctor` reports which case you are in. Pointing this at a workspace you already use puts what an agent remembers next to your own notes, synced to your peers. Convenient, and also means an agent writes where you write. > **Not the same `[memory]` as `agent.toml`.** This section says *where* the workspace lives and whether memory exists at all. The per-agent section of that name says whether *that agent* writes to it — see [`[memory]` in the agent spec](/reference/agent-spec#memory--memory-capture-for-a-plain-agent-opt-in). Full behavior in [Memory](/concepts/memory). *** ## `[skills]` Procedures exposed to MCP clients as `skill-*` tools. On by default, with an empty catalog that costs nothing. | Field | Default | Meaning | | --------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | `enabled` | `true` | Expose `skill-*`, `skill-read` and `skill-run` from `dotagent mcp`. | | `claude_skills` | `true` | Also search `~/.claude/skills/` and `$CWD/.claude/skills/`. | | `paths` | `[]` | Extra roots, each holding one subdirectory per skill. Searched **first**, so a name declared here overrides one found later. | `claude_skills` defaults to on because the skills worth exposing are usually already written for Claude Code, and requiring a copy would mean two versions drifting apart. Turn it off when that catalog is large and mostly irrelevant to an assistant that has no shell. *** ## `[commands]` Procedures **you** invoke by name, published as a Telegram menu and resolved through `command-get`. On by default, with an empty catalog that costs nothing. See [Commands](/concepts/commands). | Field | Default | Meaning | | ----------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | `enabled` | `true` | Register the `/` menu and expose `command-get` / `command-list`. | | `claude_commands` | `false` | Also search `~/.claude/commands/` and `$CWD/.claude/commands/`. | | `paths` | `[]` | Extra roots, each holding one `.md` file per command. Searched **first**, so a name declared here overrides one found later. | `claude_commands` defaults to **off**, unlike `claude_skills`. A skill costs a line in a list until a model judges it relevant; a command is published as a menu, and a Claude Code catalog is typically full of things that assume a shell and a working directory. Menu entries that cannot work are worse than absent ones. Turn it on when the catalog was written for an assistant rather than a terminal. Use `paths` for a bundle that keeps its skills one level down — discovery does not walk recursively: ```toml [skills] paths = ["/Users/me/.claude/skills/radar/skills"] ``` `dotagent doctor` reports how many skills were found, which failed to parse, and any two names that collapse to the same tool name. Full behavior in [Skills](/concepts/skills). *** ## `[daily_summary]` The end-of-day health roll-up: one line per unhealthy `(agent, schedule)`, healthy ones collapsed into a count. **On by default** — nothing here is required to receive it. | Field | Default | Meaning | | --------------- | --------- | ---------------------------------------------------------------------------------------------- | | `enabled` | `true` | Deliver on the daemon's schedule. `false` stops the nightly fire and the wake-up it schedules. | | `time` | `"22:45"` | Local time of day, `HH:MM` or `HH:MM:SS`. | | `grace_minutes` | `30` | How long after `time` a delivery still counts. Clamped to `[1, 1440]`. | | `notifiers` | `[]` | Array of tables, same shape as a manifest's `[[notifiers]]`. Empty = the `desktop` driver. | ```toml [daily_summary] time = "07:30" # a morning report reads better than a bedtime one grace_minutes = 60 # laptop opens late; still deliver [[daily_summary.notifiers]] driver = "telegram" bot_token = "${TELEGRAM_BOT_TOKEN}" chat_id = "123456789" ``` Declaring more than one entry delivers to all of them, `driver = "plugin"` included. See [Notifications](/concepts/notifications) for every driver and its fields. ### Why the defaults are what they are **No notifier means `desktop`, not silence.** Every other driver needs a chat id, a phone number or a webhook, and there is no universal default for those. `desktop` is the only one with nothing to fill in and nothing to leak — no credential, no network, nothing leaving the machine. Delivering only when configured is how this feature spent its early life: it fired nightly at a constant nobody owned and left no trace when the message went nowhere. **`[telegram]` is not used as a destination.** That section is *ingress* — the bot that accepts your messages. Wiring it to *egress* would send a nightly report to anyone who set up a bot to talk to their agents and never asked for one. **A `time` that does not parse falls back to `22:45`** rather than disabling delivery. A typo should cost you the wrong hour, not a silent month. `grace_minutes = 0` becomes `1` for the same reason: a zero-width window is an empty one. **An `events` filter inside `[[daily_summary.notifiers]]` is ignored.** The list is already scoped to a single event, so a filter there could only subtract. Entries get copied out of manifests, and `events = ["given_up"]` riding along would match nothing and drop the summary without a word. ### When it actually fires The daemon schedules a **wake-up for `time`**, so delivery does not depend on some other schedule happening to be due nearby. `grace_minutes` covers the case where the wake-up could not happen at all — machine asleep, machine off, a tick that overran its own sleep budget. It fires once per window; re-entering it (a `dotagent reload`, say) does not double-send. `enabled = false` does **not** block `dotagent daily-summary` typed by hand — that flag governs the daemon, and someone who ran the command asked for that one. Each delivery is audited as `plugin_invoked` with `plugin: "notifier:"`, failures included. Full command reference in [`cli.md`](/reference/cli#daily-summary). *** ## `[secrets]` Override the path to the daemon-loaded secrets file. The default (empty `file`) resolves to `$DOTAGENT_HOME/secrets.env`, with the `DOTAGENT_SECRETS_FILE` env var as second-tier override. | Field | Type | Default | Notes | | ------ | ------ | ------- | ------------------------------------------------------------------- | | `file` | string | `""` | Absolute path to the `KEY=VALUE` secrets file. Must be mode `0600`. | See [`concepts/secrets.md`](/concepts/secrets) for the file format, posture, and which notifier configs honor `${VAR}` today. ```toml [secrets] file = "/run/secrets/dotagent.env" # populated by a secret manager ``` *** ## `[logging]` Controls dotagent's own operational logs — the daemon's tracing output under `logs/daemon/dotagent.log` and the per-agent rotated files under `logs/agents//.log`. | Field | Type | Default | Valid values | | -------------------------- | ------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `level` | string | `"info"` | `off`, `error`, `warn`, `info`, `debug`, `trace` | | `format` | string | `"json"` | `json`, `pretty`, `compact`. **File output is always JSON regardless** — this controls the stderr stream the daemon writes for launchd/systemd to capture. | | `retention_days` | uint | `30` | Days to keep daemon logs (`logs/daemon/`). | | `per_agent_retention_days` | uint | `14` | Days to keep per-agent logs (`logs/agents//`). | | `compress_after_days` | uint | `1` | Rotated files older than N days are gzipped in-place. | ### `level` semantics Same as the `RUST_LOG` env-var grammar — but here you set a **single filter** that applies to all targets. Per-target tuning is only available via env var. ```toml [logging] level = "debug" # → everything at debug level ``` Override transiently: ```bash RUST_LOG=info,dotagent_runner=trace,dotagent_state=debug dotagent daemon ``` `RUST_LOG` wins when both are set. ### Retention behavior A daily sweep at **03:00 local time** (single-shot per day): 1. Walks `logs/daemon/` and every `logs/agents//`. 2. Rotated files older than `compress_after_days` → gzipped in-place. 3. Rotated files older than `retention_days` (daemon) or `per_agent_retention_days` (agents) → deleted. Both horizons apply to **rotated** files only. The active log is never compressed and never deleted, whatever its age: launchd and systemd hold an open fd on it, so unlinking it would strand every subsequent write. The same 03:00 pass also sweeps `state/windows/` — see [`[state]`](#state). The audit log (`audit.log`) is **never** swept regardless of these settings — by design. See [`observability.md`](/guides/observability#audit-log-vs-operational-log). *** ## `[state]` Retention for what dotagent writes under `state/`. **Nothing to configure** — the default already bounds the only directory that grows without limit. | Field | Type | Default | Valid values | | ----------------------- | ---- | ------- | --------------------------------------------------------------- | | `window_retention_days` | uint | `30` | Days to keep `state/windows/`. `0` disables the sweep entirely. | A schedule writes one window file per fired window and never revisits it, so an agent on a 15-minute interval leaves \~96 files a day behind — a `.json` plus the `.lock` next to it. Left alone, that directory grows forever. The nightly sweep deletes each aged-out window together with its `.lock`, and skips any window a writer currently holds the lock on. Windows are deleted, never gzipped: the daemon reads them as JSON. ### Why 30 days The horizon has to clear the oldest window the daemon might still consult. A window stops being actionable once it is older than the schedule's [`stale_after_minutes`](/reference/agent-spec) (default 120), which 30 days exceeds by \~360×. Even a wildly permissive `stale_after_minutes` of a full week still leaves four days of headroom. Erring high costs a few MB. Erring low deletes retry state under a running daemon, which resets `attempts` and re-fires an alert someone already gave up on — so widen it freely, narrow it carefully: ```toml [state] window_retention_days = 90 # keep a quarter of retry history ``` Heartbeats are deliberately not covered: there is exactly one per `(agent, schedule)` and it is rewritten in place, so `state/agents/` is bounded by how many schedules exist. *** ## `[power]` Whether a due run happens while the machine is on battery. **Off by default** — dotagent dispatches exactly as it always did until you opt in, and never probes the power source when the settings can't defer anything. | Field | Type | Default | Valid values | | --------------------- | ------ | ------- | ------------------------------------------------------------------ | | `on_battery` | string | `run` | `run` dispatches regardless. `defer` holds runs until mains power. | | `min_battery_percent` | uint | `0` | Defer below this charge whatever `on_battery` says. `0` disables. | The problem it solves is specific to laptops. A daemon that sleeps between events costs nothing, but the *agents* it wakes up to run are not free: a 15-minute interval agent fires 96 times a day whether the machine is plugged in at a desk or in a bag at 12%. ```toml [power] on_battery = "defer" # nothing scheduled runs on battery min_battery_percent = 20 # ...and never below 20%, even if you set "run" ``` The two rules are independent. `min_battery_percent` is the common case on its own: agents are welcome to run on battery in general, just not when there is nearly none left. ```toml [power] min_battery_percent = 20 # on_battery stays "run" ``` ### Deferring does not queue A deferred run is not stored and replayed. Both schedule kinds resolve to the *current* window rather than a backlog, so an agent deferred across four hours of battery runs **once** when the charger goes in — not sixteen times. This is the behavior that makes `defer` safe to set on an aggressive interval. The check sits after the staleness check, so a window that ages past [`stale_after_minutes`](/reference/agent-spec) while on battery is dropped rather than run hours late. That is the same call staleness always makes; if you want a deferred agent to survive a long unplugged stretch, widen its `stale_after_minutes`. ### Per-schedule override `[power]` is the default. Any schedule can override it, because the cost is per-schedule — an agent can keep a cheap hourly check running on battery while its expensive every-15-minutes sync waits for a charger: ```toml [[schedules]] id = "every-15min" type = "interval" interval_minutes = 15 on_battery = "defer" # overrides [power] for this schedule only ``` See [`agent-spec.md`](/reference/agent-spec) for the field in context. `min_battery_percent` is deliberately **not** overridable: "the battery is nearly empty" is a fact about the machine, not about one schedule's appetite. ### Detection | Platform | Probe | | -------- | ---------------------------------------------------------- | | macOS | `pmset -g batt` | | Linux | `/sys/class/power_supply/*` (`type`, `online`, `capacity`) | | Other | undetectable | An undetectable power source is treated as mains power: a machine whose battery cannot be read must keep running its agents. Failing to run is the worse failure. `dotagent tick` honors these settings too — a tick that dispatched what the daemon would have held back would misreport the thing it exists to reproduce. *** ## `[telemetry]` Opt-in OpenTelemetry OTLP export. **Disabled by default** — nothing leaves your machine until you set `otlp_endpoint`. | Field | Type | Default | Notes | | --------------- | ------ | ------------ | --------------------------------------------------------- | | `otlp_endpoint` | string | `""` | Empty = disabled. e.g., `"https://api.honeycomb.io:443"`. | | `protocol` | string | `"grpc"` | `grpc` or `http` (HTTP/protobuf). | | `service_name` | string | `"dotagent"` | `service.name` resource attribute on every span/log. | ### `[telemetry.headers]` Inline TOML table. Keys/values sent verbatim as HTTP/gRPC headers on every OTLP request. ```toml [telemetry.headers] "x-honeycomb-team" = "your-api-key" "x-custom-tenant" = "acme-tech" ``` The `OTEL_EXPORTER_OTLP_HEADERS` env var (comma-separated `k=v`) wins when both are set — useful for keeping secrets out of the config file. ### `[telemetry.resource]` Inline TOML table of OpenTelemetry resource attributes attached to every span and log record. Vendor-agnostic. ```toml [telemetry.resource] "deployment.environment" = "production" "host.name" = "workstation-01" "service.version" = "0.0.1" ``` Standard OTel semantic conventions apply — `deployment.environment`, `service.namespace`, `host.name`, `service.version`, etc. ### What gets exported Today the OTel pipeline exports **spans**: * `daemon` — root span for the daemon process lifetime * `tick` — one per scheduler tick * `agent_run` — one per agent invocation * `plugin_invoke` — one per plugin call (preflight / sink / notify-via-plugin) Logs are NOT yet exported via OTLP — that bridge is on the roadmap. For now, ship logs via a sidecar (`fluent-bit`, `vector`, `promtail`) reading the JSON file directly. See [`observability.md`](/guides/observability#opentelemetry-export) for per-vendor recipes (Honeycomb, Tempo, Jaeger, Datadog). *** ## `[os]` Installed binaries an assistant may run. **Off by default, and empty by default even when on** — the only section here that starts closed. ```toml [os] enabled = true allow = ["rg", "outl", "kubectl get", "gh pr list"] timeout_seconds = 60 ``` | Key | Default | Meaning | | ----------------- | ------- | --------------------------------------------------- | | `enabled` | `false` | Expose the `os-run` / `os-list` tools | | `allow` | `[]` | What may run. Empty runs nothing, even when enabled | | `timeout_seconds` | `60` | Wall-clock ceiling for one invocation | ### Naming a binary so the model knows it exists `os-run` makes every allowed binary reachable. Reachable is not discoverable: a model has to already know `outl` exists and guess what it is for. An `[[os.tool]]` entry publishes one under its own name, with a description: ```toml [[os.tool]] bin = "outl" description = "Personal outliner: search notes, read a page by slug, read a daily journal." [[os.tool]] bin = "kubectl" args = ["get"] description = "Read Kubernetes objects: pods, deployments, nodes. Read-only." ``` That publishes `os-outl` and `os-kubectl-get`. The description is the whole point — a name without one is what `os-run` already offers. `args` fixes the leading arguments. The model appends to them and cannot replace them, so `kubectl get` is a read-only view of a binary that can also delete: asking that tool for `delete pods` runs `kubectl get delete pods`, which fails as it should. Two entries for one binary with different fixed arguments get distinct names and never collide. **Keep the list short.** A normal machine has around a thousand executables on `PATH`; a tool each would bury the catalog and push the useful ones behind tool search. Name the ones that come up by name in conversation and let the rest fall through to `os-run`. Named tools obey the same policy as everything else: `deny` refuses them, and a `confirm`-class binary is refused with a note to have the person type it. `doctor` reports an entry whose binary `allow` does not admit, one with an empty description, and two entries that would resolve to the same name. ### Asking before it acts `allow` says what may run. Two more lists say how: ```toml [os] allow = ["*"] deny = ["shutdown", "reboot"] # never, whoever asks confirm = ["rm", "dd", "sh", "bash"] # only after a person says yes confirm_ttl_seconds = 120 ``` | Key | Default | Meaning | | --------------------- | ------------- | -------------------------------------------- | | `deny` | `[]` | Refused always. Beats `allow`, including `*` | | `confirm` | *(see below)* | Runs only after a `!!` reply | | `confirm_ttl_seconds` | `120` | How long a parked command stays answerable | **`confirm` does not default to empty.** With `allow = ["*"]` an empty one would mean a chat message can repartition a disk with nothing in between, so the default covers the destructive classics — `rm`, `rmdir`, `dd`, `mkfs`, `shred`, `diskutil`, `fdisk`, `parted`, `shutdown`, `reboot`, `halt` — and the shells: `sh`, `bash`, `zsh`, `fish`, `dash`, `ksh`. The shells are the entry that makes the rest mean anything. A guard on `rm` that lets `sh -c 'rm -rf /'` through guards nothing. Set `confirm = []` to opt out deliberately. The flow: ``` !rm -r /tmp/x This will run: rm -r /tmp/x Send `!!` to confirm. It expires in 120s. !! `rm` exited 0 and printed nothing. ``` One slot per conversation, so `!!` can only ever release the last thing *that* chat parked. Pending confirmations live in memory: a daemon restart forgets them, which fails in the safe direction. **A model cannot confirm.** When `os-run` asks for something on the `confirm` list, it is refused and told to have the person type the line. A tool that could both ask and agree would be a confirmation in name only. Matching is per binary, not per pattern, which is what makes it hold: `confirm = ["rm"]` catches `rm -rf /`, `rm -r -f /`, `rm -fr /` and `rm --recursive --force /` alike, where a textual `"rm -rf"` would catch one of the four. ### Opening the whole machine A single entry `*` allows every binary on `PATH`, a shell included: ```toml [os] enabled = true allow = ["*"] ``` Prefer this over enumerating `PATH`. An enumerated list goes stale the next time something is installed, and reading four hundred names suggests a decision was made about each one — `*` says what is true. What it means concretely: anyone who can send a message can run what the daemon's user can run. Whatever guards the inbound channel (the Telegram `allowed_user_ids`, the socket's uid check) is now guarding the machine rather than the agent catalog. `doctor` prints a warning on every run while this is set, and each invocation is still audited with its full argument list. A path is refused even here: `/bin/sh` is not a name, and `sh` is what runs. ### Granularity is per entry An entry is a binary name, optionally followed by the leading arguments that must match: * `rg` allows the binary and every argument it takes. * `kubectl get` allows that subcommand only. `kubectl delete` is refused by the catalog, not by the cluster. Matching is on whole tokens, so `kubectl get` never admits `kubectl getsecrets`. Choose bare for binaries that only read, and pin the subcommand for anything that can change something you care about. ### Two ways to reach it The allowlist is one policy with two doors: * **`os-run`**, where an assistant decided a binary would help. * **`!` in a message**, where you typed the command yourself. `!rg foo src` runs it directly: no model, no session, nothing stored, and errors come back raw. Works over Telegram and over the local socket (`dotagent api`). The prefix is read after the Telegram allowlist and rate limit, and obeys the same `allow` list — a stolen session types `!` as easily as you do. Quotes group an argument (`!rg "hello world"`). Nothing else from a shell applies: `!ls; rm x` looks for a binary named `ls;`. ### What it does not do The allowlist bounds *which programs* run, not what each one is capable of. A binary listed bare is trusted with everything it can do, and some read-ish commands can still execute things through their own flags. Listing `kubectl` bare on a machine holding production credentials means a chat message reaches production. Arguments never touch a shell. `os-run` spawns the program with an argument list, so `|`, `&&` and `$(…)` are literal characters. A path is refused where a name is expected, so `/bin/sh` cannot stand in for a listed `sh`. Every invocation is audited as `os_command_invoked` at `Critical`, recording the binary and the full argument list. See [`security/threat-model.md`](/security/threat-model) V17. ## What's NOT in `config.toml` | Concern | Where instead | | -------------------------------------- | -------------------------------------------------------------------------- | | Per-agent retry policy | `[defaults]` in the agent's own `agent.toml` | | Per-agent notifications | `[[notifiers]]` in the agent's own `agent.toml` | | Per-agent security policy | `[security]` in the agent's own `agent.toml` | | Notifier defaults across agents | (Not yet supported — declare per-agent for now.) | | Daemon binary path / unit file content | Generated by `dotagent install` from the running binary. No override knob. | *** ## Migrating partial configs `config.toml` is **partial-overlay**: missing fields keep their defaults. The minimal "I want debug logs" config: ```toml [logging] level = "debug" ``` Everything else (`format`, `retention_days`, `[telemetry]`, …) stays default. You don't need to write empty tables for sections you don't customize. *** ## Reloading `config.toml` is re-read on: * Daemon startup * The next tick after a SIGHUP (`dotagent reload`) Changes that need a full **restart** (not just reload): * Switching `[logging].format` between `json` / `pretty` / `compact` for the stderr stream — the subscriber is initialized once at boot. * Changing OTel `protocol` (gRPC ↔ HTTP) — the exporter is built once. For those, use: ```bash launchctl kickstart -k "gui/$(id -u)/run.avelino.dotagent" # macOS systemctl --user restart run.avelino.dotagent # Linux ``` *** ## Verifying your config ```bash # Parse-check the file (any syntax error fails here). toml-cli get config.toml . # Make sure the daemon actually loaded it. tail -F ~/.config/dotagent/logs/daemon/dotagent.log \ | jq -c 'select(.fields.message | contains("config"))' ``` To confirm OTel went live: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log \ | jq -c 'select(.fields.message | contains("otel") or contains("OTLP"))' ``` You should see a "telemetry initialized" or similar message after the next reload/restart. *** ## Related * [`observability.md`](/guides/observability) — logging architecture + OTel vendor recipes * [`env-vars.md`](/reference/env-vars) — `RUST_LOG` and `OTEL_EXPORTER_OTLP_HEADERS` overrides * [`paths.md`](/reference/paths) — where logs land on disk * [`agent-spec.md`](/reference/agent-spec) — per-agent config (manifest) # Observability > dotagent ships with structured JSON logs, daily rotation, and opt-in OpenTelemetry export from day zero. No need to bolt on a logging stack later — pick what you want to plug into and go. ## TL;DR **There's nothing to configure.** Install dotagent, start the daemon, and you immediately get: * Daemon stderr → captured by launchd / systemd (compact human-friendly text) * Daemon structured logs → `~/.config/dotagent/logs/daemon/dotagent.log` (JSON, rotated daily) * Each agent's stdout+stderr → `~/.config/dotagent/logs/agents//.log` (raw, rotated daily) * Rotated files older than 1 day → gzipped automatically * Files older than 30 days (daemon) / 14 days (agents) → deleted automatically * Audit log → `~/.config/dotagent/audit.log` (append-only, hash-chained; rotates at 32MB into segments that are never deleted) The `config.toml` file is **entirely optional**. Create one only if you want to: * Change verbosity / retention horizons * Enable OpenTelemetry export to a remote backend * Override defaults for your environment This guide covers: 1. [What gets logged](#what-gets-logged) — daemon, agents, plugins, audit 2. [Where things live](#filesystem-layout) — log directory layout 3. [Log format](#log-format) — JSON schema with examples 4. [Rotation and retention](#rotation-and-retention) — defaults + customizing 5. [Inspecting locally](#inspecting-locally) — `dotagent logs`, `jq` queries 6. [OpenTelemetry export](#opentelemetry-export) — opt-in, two-line setup 7. [Audit log vs operational log](#audit-log-vs-operational-log) — different purpose, different rules 8. [Troubleshooting](#troubleshooting) ## What gets logged dotagent has **four** distinct log streams: ```mermaid flowchart LR daemon[daemon process] --> daemon_log["logs/daemon/dotagent.log
(JSON, rotated daily)"] agent[agent subprocess] --> agent_log["logs/agents/<name>/<name>.log
(raw stdout/stderr, rotated daily)"] daemon -.only when stderr is a TTY.-> stderr["stderr
(compact, human-readable)"] daemon --> audit["audit.log
(append-only, hash-chained)"] daemon -.OTLP optional.-> otel[OpenTelemetry collector] ``` | Stream | Producer | Format | Rotation | Purpose | | ------------------------------- | --------------------------- | ------------------------------------ | ---------------------------------------- | -------------------------------------------- | | `logs/daemon/dotagent.log` | daemon's `tracing` | JSON (one line per record) | daily, gzip → delete | Structured operational logs for debugging | | `logs/agents//.log` | agent process stdout+stderr | Raw text | daily, gzip → delete | What the agent script actually said | | stderr | daemon's `tracing` | Compact text | **only installed when stderr is a TTY** | Human-friendly stream for an interactive run | | `audit.log` | daemon | Newline-delimited JSON, hash-chained | 32MB → sealed segment, **never deleted** | Tamper-evident security event ledger | OpenTelemetry (when enabled) re-emits the **same** structured records as OTLP spans, so any backend gets the full picture. ### stderr is conditional, on purpose Run `dotagent daemon` in a terminal and you get the compact stream. Run it under launchd or systemd and you do not: there stderr is an appended plain file (`logs/daemon/run.avelino.dotagent-error.log`) that no rotation policy covers, so mirroring into it would duplicate an already-rotated log into one that grows forever. That leaves the stderr file as a **crash channel** — panics, and startup failures that happen before logging is up. An empty `…-error.log` under a service manager is the healthy state. To follow what the daemon is doing, read `logs/daemon/dotagent.log`. `DOTAGENT_LOG_STDERR=1` forces the mirror on (units rewired to journald, which does rotate); `DOTAGENT_LOG_STDERR=0` forces it off. ANSI colour follows the same TTY rule and additionally honours `NO_COLOR`, in the daemon and in every subcommand. ## Filesystem layout Everything under `$DOTAGENT_HOME` (default `~/.config/dotagent`): ``` ~/.config/dotagent/ ├── config.toml # global config (logging + telemetry) ├── agents/ # manifests ├── plugins/ # custom plugin binaries ├── state/ │ ├── agents//.heartbeat.json │ ├── windows/--.json │ ├── plugins//.json │ ├── known_manifests.json │ └── daemon.pid ├── logs/ │ ├── daemon/ │ │ ├── dotagent.log # current (today) │ │ ├── dotagent.log.2026-05-19 # yesterday │ │ └── dotagent.log.2026-05-18.gz # older, compressed │ └── agents/ │ ├── finops-weekly/ │ │ ├── finops-weekly.log │ │ └── finops-weekly.log.2026-05-19 │ └── inbox-triage/ │ └── inbox-triage.log ├── audit.log # append-only, hash-chained (live) └── audit.log.20260806T101500 # sealed segment, never deleted ``` ## Log format Daemon records are JSON with `tracing`'s standard envelope: ```json { "timestamp": "2026-05-19T10:34:12.518472-03:00", "level": "INFO", "fields": { "message": "dispatching run", "agent": "team-standup", "schedule": "daily", "attempt": 1, "max_retries": 20, "expected": "2026-05-19T08:30:00-0300" }, "target": "dotagent::commands::daemon", "span": { "name": "tick", "agents_scanned": 4 }, "spans": [ { "name": "daemon" } ] } ``` Key field conventions used across the codebase: | Field | Where | Notes | | ------------------ | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | `agent` | run/dispatch/plugin records | agent name (matches `agent.toml` `agent.name`) | | `schedule` | run/dispatch records | schedule id (matches `[[schedules]].id`) | | `attempt` | retry records | 1-based attempt number | | `exit_code` | post-run records | agent's exit code (`124` = timeout) | | `duration_seconds` | post-run records | wall-clock duration | | `plugin` | plugin / notifier invocations | plugin short name (`sink-roam`) or `notifier:` for built-in notifiers (`notifier:desktop`, `notifier:imessage`, ...) | | `event` | plugin / notifier invocations | `attempt_failed`/`given_up`/`recovered`/... | Agent log files (`logs/agents//.log`) are NOT JSON — they're the raw stdout+stderr the agent script produced, prefixed with a header per run: ``` === 2026-05-19T08:30:01-0300 run started · schedule=daily · slug=period_dia-anterior === mount: / free: 87.3 GB / 460.4 GB (18%) threshold: 20% --- stderr --- 🚨 disk-alert: 18% free on / avail: 87.3 GB / 460.4 GB threshold: 20% host: workstation-01 ``` Mixing structured (daemon) + raw (agent) is deliberate: the daemon's metadata is uniform across all runs, the agent's output is whatever shape its author chose. ## Rotation and retention **You don't need to configure anything.** Out of the box, dotagent ships these defaults: | Setting | Default | What it means | | -------------------------- | --------- | ------------------------------------------------------------------------------- | | `level` | `info` | tracing filter (`off`, `error`, `warn`, `info`, `debug`, `trace`) | | `format` | `compact` | stderr format. Files are always JSON regardless. | | `retention_days` | `30` | daemon logs older than this are deleted | | `per_agent_retention_days` | `14` | agent logs (typically noisier; shorter horizon) | | `compress_after_days` | `1` | rotated files older than this are gzipped | | `window_retention_days` | `30` | `state/windows/` files older than this are deleted (`[state]`, not `[logging]`) | If those match what you want, **don't create a `config.toml`** — the defaults kick in automatically. ### Customizing Override any subset by writing `~/.config/dotagent/config.toml`. Missing fields keep their defaults — the file is purely partial overrides. ```toml # ~/.config/dotagent/config.toml [logging] level = "debug" # turn up verbosity retention_days = 90 # keep three months ``` Apply with `dotagent reload` (sends SIGHUP; daemon re-reads the file on the next tick). You can also override the level transiently via env var (overrides the config file): ```bash RUST_LOG="dotagent=debug,dotagent_runner=trace" dotagent daemon ``` ### How rotation works `tracing-appender` rolls files daily — at midnight local time, the current file gets renamed to `.log.YYYY-MM-DD` and a fresh `.log` is opened. ### How retention sweeps work The daemon runs an internal cleanup pass once per day at **03:00 local time**. It: 1. Walks `logs/daemon/` and every `logs/agents//` directory. 2. Rotated files older than `compress_after_days` get gzipped in-place. 3. Files older than `retention_days` (or `per_agent_retention_days` inside agent dirs) get deleted — **rotated files only**. The active (non-rotated) log is skipped no matter how old it is: launchd and systemd hold an open fd on it, and unlinking a file someone is still writing to strands every subsequent line in an unnamed inode. A long-lived daemon with a quiet log would otherwise have its live file deleted out from under it at day 31. 4. Walks `state/windows/` and deletes windows older than `window_retention_days`, each together with its `.lock`. Windows are never gzipped — the daemon reads them as JSON, so a compressed window is a corrupted one. A window whose lock is currently held is skipped. The audit log is **never** swept. It rotates by size, but its segments are not subject to `[logs]` retention and nothing deletes them — see [Audit log vs operational log](#audit-log-vs-operational-log). To trigger a sweep manually: ```bash # Restart the daemon at 03:00, OR — simpler — wait. The sweeper is # idempotent; nothing happens if there's nothing aged out. dotagent reload # SIGHUP doesn't re-sweep, but the next 03:00 will ``` ## Inspecting locally ### Tail the daemon log ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq . ``` Or via the launchd-captured stderr (compact, human format): ```bash tail -F ~/.config/dotagent/logs/daemon/run.avelino.dotagent.log ``` ### Tail an agent's output ```bash dotagent logs team-standup dotagent logs team-standup --follow dotagent logs team-standup -n 200 ``` ### Query the daemon log with `jq` ```bash # All runs of a specific agent today jq -c 'select(.fields.agent == "team-standup")' \ ~/.config/dotagent/logs/daemon/dotagent.log # Failures only jq -c 'select(.level == "WARN" or .level == "ERROR")' \ ~/.config/dotagent/logs/daemon/dotagent.log # Runs that timed out jq -c 'select(.fields.exit_code == 124)' \ ~/.config/dotagent/logs/daemon/dotagent.log # Plugin invocations and their outcomes jq -c 'select(.fields.plugin) | {ts: .timestamp, plugin: .fields.plugin, ok: .fields.ok}' \ ~/.config/dotagent/logs/daemon/dotagent.log ``` ### Health snapshot ```bash dotagent status # textual dashboard dotagent inspect team-standup # heartbeat + schedule + windows ``` ## OpenTelemetry export **Disabled by default.** Out of the box, dotagent writes everything to local files (stdout/stderr captured by launchd/systemd + the JSON file described above). Nothing leaves your machine. That's intentional — zero-config, zero network egress. Turn it on by writing **just two lines** of `config.toml`: ```toml # ~/.config/dotagent/config.toml [telemetry] otlp_endpoint = "https://api.honeycomb.io:443" ``` Authentication uses the standard OTel env var so dotagent doesn't need vendor-specific config: ```bash export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_API_KEY" dotagent reload # daemon picks up the change on SIGHUP ``` That's it — every span the daemon emits (`tick`, `agent_run`, `plugin_invoke`) shows up in your backend. ### Fuller config (all fields optional) ```toml [telemetry] otlp_endpoint = "https://api.honeycomb.io:443" protocol = "grpc" # "grpc" (default) or "http" service_name = "dotagent" # default [telemetry.resource] # extra attributes attached to every span "deployment.environment" = "production" "host.name" = "workstation-01" ``` ### Vendor recipes #### Honeycomb ```toml [telemetry] otlp_endpoint = "https://api.honeycomb.io:443" service_name = "dotagent" ``` ```bash export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_KEY" ``` EU region: use `api.eu1.honeycomb.io:443`. #### Grafana Tempo (self-hosted or Grafana Cloud) ```toml [telemetry] otlp_endpoint = "https://tempo-prod-04-prod-eu-west-0.grafana.net:443" service_name = "dotagent" ``` Grafana Cloud auth: ```bash export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(printf %s "$STACK_ID:$API_TOKEN" | base64)" ``` Self-hosted Tempo without auth: ```toml [telemetry] otlp_endpoint = "http://tempo.lan:4317" service_name = "dotagent" ``` #### Jaeger (local) ```bash docker run -d --name jaeger \ -p 4317:4317 -p 16686:16686 \ jaegertracing/all-in-one:latest ``` ```toml [telemetry] otlp_endpoint = "http://localhost:4317" service_name = "dotagent" ``` Open `http://localhost:16686`, pick `dotagent` from the service list. #### Datadog Datadog wants the OTLP receiver enabled on its agent: ```toml [telemetry] otlp_endpoint = "http://localhost:4317" # local Datadog agent service_name = "dotagent" ``` ```bash # In your datadog-agent.yaml: # otlp_config.receiver.protocols.grpc.endpoint: 0.0.0.0:4317 ``` ### What gets exported Today the OTel pipeline exports **spans**: * `daemon` — root span for the daemon process lifetime * `tick` — one per scheduler tick * `agent_run` — one per agent invocation * `plugin_invoke` — one per plugin call (preflight / on\_success / on\_failure) Logs are NOT yet OTLP-exported (the `tracing → OTLP logs` bridge is on the roadmap). For now, ship logs via your filesystem agent (`fluent-bit`, `vector`, `promtail`) reading the JSON file. ## Audit log vs operational log These look similar but serve different purposes — **do not conflate them**. | | Audit log | Operational logs | | ---------- | ------------------------------------------------ | --------------------------------- | | Path | `~/.config/dotagent/audit.log` | `~/.config/dotagent/logs/` | | Format | NDJSON, hash-chained (`prev_hash`) | NDJSON (`tracing` style) | | Producer | daemon, on consequential events | daemon's `tracing` macros | | Frequency | sparse (5–50 entries/day) | dense (thousands/day) | | Rotation | by size (32MB), across a hash seam | daily, gzip, then delete | | Retention | indefinite — segments are **never** auto-deleted | configurable (default 30/14 days) | | Mutability | append-only; tamper detectable | overwritten on rotation | | Schema | strict (typed `AuditEvent` variants) | unstructured (`tracing` Fields) | | Use case | forensics, security audit | debugging, tail-the-app | Rotation is the one thing the two now share, and they share only the word. An operational log rolls and is eventually deleted; the audit log rolls into a **sealed segment** stitched to the new file by a hash seam, and dotagent never removes it. Pruning old segments is supported and stays legible to verification — the reasoning is in [`security/threat-model.md`](/security/threat-model#what-the-hash-chain-guarantees-and-what-it-does-not). You should `grep` the audit log when answering "was this run authorized? who changed manifest X? when did the daemon last say a plugin was phantom?". You should `jq` the operational log when answering "why did the last 5 ticks take so long?". ## Troubleshooting ### Log directory missing after install dotagent creates `logs/daemon/` on first daemon start. Run: ```bash dotagent run hello-fish --schedule manual # smoke ls -la ~/.config/dotagent/logs/ ``` ### Log file not rotating `tracing-appender` rotates at **first write past midnight**. If the daemon is idle (no event after midnight), the rename is deferred until the next log line. Force one: ```bash dotagent reload ``` ### Sweeper not running The sweeper fires at 03:00 ± 30min and is single-shot per calendar day. Check the daemon log for the line `log retention sweep completed`. If absent for >24h, file an issue with the daemon log timestamps. ### Disk filling despite retention * Check `retention_days` isn't unreasonable: 90+ days × verbose level = real disk. * Check that the sweeper has write permission. `logs/agents//` inherits the daemon's umask; if your agent script chowns its log dir, the sweep fails silently. ### OTel pipeline not exporting Confirm the daemon picked up the config: ```bash grep otlp_endpoint ~/.config/dotagent/config.toml dotagent reload tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq 'select(.fields.message | contains("otel"))' ``` Common gotchas: * **TLS error**: the OTLP endpoint requires `https://` (gRPC over TLS). Local Jaeger uses `http://localhost:4317`. * **Headers**: vendor-specific auth goes in `OTEL_EXPORTER_OTLP_HEADERS`, not in `config.toml`. Comma-separated `k1=v1,k2=v2`. * **Reload required**: changing config without restarting the daemon is a no-op for OTel. Use `dotagent reload`. ### Logs full of noise Lower the verbosity: ```toml [logging] level = "warn" ``` Or per-target via env: ```bash RUST_LOG="dotagent=info,dotagent_runner=debug" dotagent daemon ``` ## Related * [`docs/concepts/agents.md`](/concepts/agents) — what an agent is * [`docs/concepts/plugins.md`](/concepts/plugins) — how plugins emit events * [`docs/reference/agent-spec.md`](/reference/agent-spec) — manifest fields * [`docs/security/threat-model.md`](/security/threat-model) — audit log's role in detection # LLM agents (claude -p) > How to call a model from an agent without the model becoming the agent. The headless gotchas, the failure modes, and the two-layer timeout that keeps a hung run from becoming a zombie. dotagent has **zero LLM dependencies**. There's no SDK to import, no API key it reads, no provider it knows about. An agent that wants a model spawns a CLI, exactly like it spawns `jq`: ```bash claude --model sonnet --system-prompt "$system_prompt" -p - out.txt ``` That's the whole integration. Everything below is about doing it without getting burned. Working example to read alongside this page: [`examples/hn-digest`](https://github.com/avelino/dotagent/tree/main/examples/hn-digest/README.md). ## The inverted loop The default assumption in 2025-2026 is that an "AI agent" means a model in charge: it decides what to run, when, and in what order, and calls tools to get there. dotagent inverts it. A scheduler that cannot think orchestrates, and the model is a subprocess that does one bounded thing: ```mermaid flowchart LR subgraph det["deterministic (dotagent + your script)"] sched["schedule"] --> collect["collect
curl, gh, jq"] collect --> prompt["build prompt"] end prompt --> llm["claude -p
the one judgment call"] llm --> render subgraph det2["deterministic"] render["render + validate"] --> sink["sink / notify"] end ``` This sounds backwards until you run it daily for a month. The reasoning is in [`concepts/agents.md`](/concepts/agents#the-dotagent-definition) and the story of arriving at it is in the [introduction](/#how-this-got-here). The short version: an LLM regenerating your orchestration on every run pays tokens to rewrite code that already worked yesterday, and it's slower and less predictable while doing it. So the rule of thumb: | Task | Who does it | | ----------------------------------------- | ----------- | | Fetch the top 30 stories from an API | Code | | Decide which 3 are worth reading | Model | | Sort by score, filter nulls, dedupe | Code | | Write one sentence on why a story matters | Model | | Build the output the sink parses | Code | | Classify an ambiguous email | Model | If a task has one correct answer, code it. If it needs judgment, prompt it. Most agents need the model for one step, sometimes zero. ## The invocation Two separate channels, and keeping them separate is the point: ```bash # The contract: static, lives in git, reviewed like code. system_prompt=$(cat "$AGENT_HOME/prompt.md") # The data: this run only, built into a temp file. { echo "Today's stories:" jq . "$AGENT_TMPDIR/stories.json" echo "Pick the 3 most interesting." } >"$AGENT_TMPDIR/prompt.txt" claude --model sonnet \ --system-prompt "$system_prompt" \ -p - <"$AGENT_TMPDIR/prompt.txt" >"$AGENT_TMPDIR/out.txt" ``` **`-p -` reads the user prompt from stdin.** Use it. Passing a large prompt as an argv string runs into length limits and drags every shell quoting problem along with it. Stdin has neither. **The system prompt is a file in the agent directory.** It diffs in review, and when output quality drifts you can `git log prompt.md` and see what changed. A prompt built inline by string concatenation can't be audited after the fact. > `--system-prompt` **replaces** the CLI's default system prompt. `--append-system-prompt` adds to it. For a scheduled job that must produce one exact output shape, replacing is usually what you want. ## Headless is not your terminal `claude -p` in a daemon-spawned process is a different environment from `claude` in your shell, in ways that fail silently. ### There is nobody to approve a tool call Interactively, a tool call prompts you. Headless, there's no TTY and no human. Grant tools explicitly: ```bash claude --allowedTools "WebFetch,WebSearch" --system-prompt "$sp" -p - ``` Without the flag the run can't use tools at all, which is a fine default when the model only has to write prose about data you already collected. List exactly what the task needs and nothing else — this is the one place where you decide what a model running unattended on your machine is able to touch. ### MCP servers are not inherited An interactive session reads `mcpServers` from your user config. **A headless `-p` run does not.** Your MCP tools are simply absent, and the model reports it couldn't find the tool rather than failing loudly. Pass the config explicitly: ```bash # Extract the mcpServers block from the user config into a headless config. python3 - <<'PY' >"$AGENT_TMPDIR/mcp.json" import json, os, sys with open(os.path.expanduser("~/.claude.json")) as f: d = json.load(f) json.dump({"mcpServers": d.get("mcpServers", {})}, sys.stdout) PY claude --mcp-config "$AGENT_TMPDIR/mcp.json" \ --allowedTools "mcp__myserver__search" \ -p - <"$prompt_file" ``` ### MCP tool names must match exactly Headless resolves tool names by exact string, with no fuzzy matching. The name is `mcp____`, where `` is the key under `mcpServers` in your config, not the product name. If your config nests servers behind a proxy whose key is itself `mcp`, the real name has both prefixes: ``` mcp__roam__get_page # wrong, if "roam" is reached through a proxy mcp__mcp__roam__get_page # right, when the server key is "mcp" ``` Get it wrong and the tool is silently unavailable. Verify against your config rather than guessing. ### Skills don't fire reliably A skill that triggers dependably in an interactive session may not fire under `-p`. If a skill's behavior is load-bearing, invoke it by name in the prompt **and** inline the parts you can't lose. Defense in depth, not a single hook. ## Two layers of timeout A model call can hang. Guard it in the script *and* in the manifest, with the script's limit lower: ```mermaid flowchart TD A["HN_CLAUDE_TIMEOUT = 300s
(script, wraps claude)"] -->|fires first| B["clean error:
'claude exited 142'"] C["timeout_seconds = 420
(manifest, supervisor)"] -->|only if the script itself wedges| D["kill-tree of the
whole process group"] ``` The script's timeout tells you **which step** hung. The supervisor's timeout only tells you the agent didn't finish. You want the first one to win. macOS ships no `timeout` (that's GNU coreutils). Perl's `alarm` is built in everywhere, and `exec` preserves the PID so the supervisor's kill-tree still reaches the real process: ```bash run_with_timeout() { local seconds="$1"; shift perl -e 'alarm shift; exec @ARGV' "$seconds" "$@" } run_with_timeout 300 claude --model sonnet -p - <"$prompt_file" >"$out" ``` ## How LLM calls actually fail Not the way scripts fail. Handle these explicitly: ### Empty output with a zero exit The most common failure, and it does **not** set a non-zero exit code. Without a guard you publish a blank block over yesterday's good one: ```bash if [[ ! -s "$out" ]]; then echo "[$AGENT_NAME] error: claude returned empty output" >&2 exit 1 fi ``` Exiting non-zero here is the right move: dotagent retries per your `[defaults]`, and pages you only when retries are exhausted. ### Output that ignores the format The model wraps the answer in a fence, adds "Here's your digest:", or appends a closing remark. When you need structured output, parse tolerantly instead of trusting the first attempt: ````bash # Try raw, then a ```json fence, then the first {...} block. if jq -e . "$out" >/dev/null 2>&1; then cp "$out" "$parsed" elif sed -n '/```json/,/```/p' "$out" | sed '1d;$d' | jq -e . >/dev/null 2>&1; then sed -n '/```json/,/```/p' "$out" | sed '1d;$d' >"$parsed" else echo "[$AGENT_NAME] error: no JSON in claude output" >&2 cat "$out" >&2 exit 1 fi ```` ### Formatting that breaks the downstream parser Sink plugins parse stdout by indentation ([the contract](/concepts/plugins)). A stray blank line flattens the hierarchy. Tell the model in `prompt.md`, then enforce it in the script anyway: ```bash sed '/^[[:space:]]*$/d' "$out" ``` **Never let a contract live only in the prompt.** State it there so the model has the best chance, and re-assert it in code so a bad run degrades instead of corrupting the destination. Related: depend on ranges rather than exact values. The sink treats `indent ≤ 3` as L1, so a model emitting 1 space on one line and 2 on the next produces the same tree. A prompt that requires exact space counting is a prompt that breaks weekly. ### Anything the output's correctness depends on Generate it in the script, not the model. The marker a sink matches on, the date, the page name, the root line: ```bash echo "#hn-digest $(date +%Y-%m-%d)" # script owns this sed '/^[[:space:]]*$/d' "$out" # model owns only the body ``` If the model hallucinates a marker, an idempotent sink stops recognizing its own previous block and starts appending duplicates every run. ### Fall back to something deterministic when you can If a degraded output beats no output, don't fail — drop to code: ```bash body=$(claude --model sonnet -p - <"$prompt_file" 2>/dev/null) if [[ -z "$body" ]]; then echo "[$AGENT_NAME] warn: claude failed, falling back to flat list" >&2 body=$(jq -r '.[] | " " + .title' "$AGENT_TMPDIR/stories.json") fi ``` ## Where the latency actually is For a scheduled agent, none of this matters — nobody is waiting. For anything conversational, it is the whole experience, and the intuition about where the time goes is usually wrong. Numbers below are measured on a real Telegram assistant, answering a trivial prompt with no tool call, so they isolate fixed cost from the model's thinking. ### The MCP proxy you spawn per run | toolkit | time | | -------------------------------------------------- | ----- | | no MCP server at all | 4.0s | | `dotagent mcp` (28 tools) | 3.8s | | plus an aggregating proxy over **stdio** | 12.0s | | plus the same proxy over **HTTP, already running** | 3.4s | `dotagent mcp` is free — it reads the catalog off disk and answers in \~20ms. An aggregating proxy is not: spawned per run it reopens its database, rediscovers every backend and reclassifies every tool before the first request. If a proxy is already listening, connect to it instead of spawning a private copy: ```json { "mcpServers": { "proxy": { "type": "http", "url": "http://127.0.0.1:7332/mcp" } } } ``` ### The transcript you replay `--resume` sends the whole conversation as input, and nothing trims it: | transcript | answer | | ------------------------------------ | ------- | | \~90 KB | 8-10s | | 977 KB (228 messages, \~128k tokens) | 26-141s | A chat gets monotonically slower the more you use it, which reads as "the bot got worse" rather than as a size problem. Put a ceiling on it and start a fresh session past that, and keep what must survive in [memory](/concepts/memory) instead of in the transcript. ### The process you fork per message `claude -p` pays startup every invocation. `--input-format stream-json` keeps one process reading turns from stdin, so the session never leaves memory: | | 1st answer | 2nd answer | | ---------------------- | ---------- | ---------- | | fork per message | 3.91s | 4.94s | | one persistent process | 3.46s | **1.90s** | Every message after the first is a second message. Holding a process alive used to mean running something the supervisor did not manage — deadlines, reaping and crash recovery became yours. It no longer does: declare `[lifecycle] mode = "persistent"` and dotagent keeps the process alive, delivers requests over [JSON lines](/reference/persistent-protocol), and supervises it like every other subprocess. Set `key = "chat_id"` so each conversation gets its own. Still worth it for a chat and not for a cron job. A process that lives is a process that can leak and drift, and a scheduled agent has nothing to gain. ### Tools you publish but never call Clients stop putting tool schemas in the prompt past a few hundred and defer them behind a lookup step, which turns one answer into several round trips — 16 lookups in a single reply, in one measured case. Publishing 204 tools when the agent uses a dozen is not free, so narrow the catalog to what the agent actually needs. ### One that is easy to miss Anything feeding the invocation must be **deterministic**, or the caching underneath it silently never hits. A proxy listing its servers from a hash map returned them in a different order every call; that reordered the allowlist, which changed the session fingerprint, which threw away the warm session on every single message. Sorting one list fixed it. If a cache "does not seem to work", check that its key is stable before assuming the cache is broken. ## Cost control **Dry run must not spend tokens.** `AGENT_DRY_RUN=true` should collect, dump what it gathered, and exit before the model call. That's what makes `dotagent run --dry-run` safe to hammer while you're debugging collection: ```bash if [[ "$AGENT_DRY_RUN" == "true" ]]; then jq . "$AGENT_TMPDIR/stories.json" exit 0 fi ``` **Retry policy costs money.** Every retry is another full call. Two is usually right for a model: it absorbs a rate limit or a network blip, and a third failure means something real is broken. ```toml [defaults] max_retries = 2 retry_backoff_minutes = [5, 15] ``` **Match the model to the task.** Picking 3 items out of 20 and writing a sentence each is not frontier work. Make it a tunable so you can change your mind without touching the script: ```toml [env.extra] HN_CLAUDE_MODEL = "haiku" ``` **Shrink the prompt, not just the model.** Filter and compact in `jq` before the call. Sending 20 pre-trimmed records beats sending 200 raw ones and asking the model to ignore most of them. ## Secrets The `claude` CLI uses the credentials it already has (keychain, `~/.claude.json`). Agents in this shape export **no** API key. If you call a provider that does need one, it goes in `~/.config/dotagent/secrets.env` and never in `agent.toml` — see [`concepts/secrets.md`](/concepts/secrets). Manifests are committed; `[env.extra]` is for tunables, not credentials. ## Security A model choosing what to do with granted tools is a wider blast radius than a shell script following a fixed path. Declare it: ```toml [security] allowed_commands = ["bash", "curl", "jq", "perl", "claude"] allowed_plugins = ["sink-file"] network = ["hacker-news.firebaseio.com", "api.anthropic.com"] filesystem_writable = ["/tmp"] ``` Enforcement is still landing (see [`security/threat-model.md`](/security/threat-model)), so today this documents intent and gives `dotagent doctor` something to audit. Writing it is still worth the two minutes: it forces you to answer "what can this thing reach?" while you still remember. Two habits that matter more than the manifest: * **Keep `--allowedTools` minimal.** It's the actual boundary right now. * **Treat collected data as untrusted.** You're pasting HN titles, email bodies and Slack messages into a prompt. Anything that reaches the model can try to instruct it. This is why the script, not the model, owns the marker line and the destination. ## When not to reach for a model * **The output is structured data.** If you know the fields, `jq` is faster, free, and can't hallucinate. * **The rule is expressible.** "Sender matches this list → label it" is an `if`. Save the model for the residue. * **You need the same answer every time.** Models are not idempotent. * **It runs every 5 minutes.** Multiply tokens by 288 runs a day before committing. The strongest agents in practice are mostly deterministic with one model call in the middle. Some have none at all — see [`examples/disk-alert`](https://github.com/avelino/dotagent/tree/main/examples/disk-alert/README.md). ## Related * [`examples/hn-digest`](https://github.com/avelino/dotagent/tree/main/examples/hn-digest/README.md) — everything on this page, working, in bash * [`concepts/agents.md`](/concepts/agents) — the agent patterns, including "Pattern 2 — Brief / digest" * [`reference/env-vars.md`](/reference/env-vars) — the `AGENT_*` vars your script reads * [`concepts/secrets.md`](/concepts/secrets) — where API keys go * [`examples/telegram-assistant`](https://github.com/avelino/dotagent/tree/main/examples/telegram-assistant/README.md) — the conversational shape, with per-chat sessions and a ceiling on the transcript * [`concepts/memory.md`](/concepts/memory) — what should outlive a conversation, and why a transcript is the wrong place for it * [`guides/troubleshooting.md`](/guides/troubleshooting) — when a scheduled run misbehaves * [`faq.md`](/misc/faq#will-dotagent-run-my-llm-agents) — the short version of this page # Migrating from the Fish framework If you came from the Fish framework in [avelino/dotfiles](https://github.com/avelino/dotfiles) (`lib/agent.fish`, `agents/agent-orchestrator/`), this guide maps every concept you already use to its dotagent equivalent. The plan is **no big bang** — old and new run side by side until you migrate the last agent. ## Mapping at a glance | Fish framework concept | dotagent equivalent | | ------------------------------------------- | ------------------------------------------------------------------------------------------ | | `meta.json` | `agent.toml` (richer, with comments and TOML types) | | `lib/agent.fish` — `agent_init` | env vars dotagent injects + tmpdir auto-cleanup | | `lib/agent.fish` — heartbeat | dotagent writes the same shape, before+after the run | | `lib/run-with-timeout.fish` | built into `dotagent run` (`agent.timeout_seconds`) | | `agents/agent-orchestrator` (tick) | `dotagent daemon` (adaptive, single plist) + `dotagent tick` (one-shot) | | `agents/agent-orchestrator --status` | `dotagent status` (read-only dashboard) | | `agents/agent-orchestrator --daily-summary` | `dotagent daily-summary` + the daemon's own fire at `[daily_summary].time` (default 22:45) | | `cron.nix` launchd config | `dotagent install` (single `run.avelino.dotagent` plist) | | `agent_output_imessage` | built-in notifier `[[notifiers]] driver = "imessage"` (with `rate_limit_minutes`) | | `lib/roam.fish` (`roam_publish`) | plugin `sink-roam` (full port: sanitize + parse + idempotent replace) | | inline WARP check | plugin `preflight-warp` | | `IMESSAGE_TO="+55..."` hardcoded | per-plugin `config.to`, or profile defaults | | retry loop in `agent-orchestrator` | `WindowState`-backed retry with `max_retries` + `retry_backoff_minutes` | | (no equivalent) | `dotagent logs`, `dotagent inspect`, `dotagent reload`, `dotagent run-now` | ## Step-by-step ### 1. Install dotagent ```bash git clone https://github.com/avelino/dotagent cd dotagent cargo install --path crates/dotagent cargo install --path plugins/preflight-warp cargo install --path plugins/sink-roam # (whichever plugins your agents use — notifiers are built in) ``` > Notifiers (`desktop`, `imessage`, `slack`, `ntfy`, `pushover`, `telegram`) ship inside the `dotagent` binary. No extra `cargo install` needed. ### 2. Convert `meta.json` → `agent.toml` > If you're managing your dotfiles with **Nix / home-manager** (as avelino/dotfiles does), the conversion is mechanical and there's a reference Nix module that does the work for every agent in one place — see [`modules/home-manager/dotagent/`](https://github.com/avelino/dotfiles/tree/main/modules/home-manager/dotagent) in avelino/dotfiles. Copy `default.nix` from there, adjust the `builtinAgents` list, set `dotagent.enable = true`, and rebuild. Skip the manual conversion below if so. For the `team-standup` agent (example), the Fish manifest… ```json { "name": "team-standup", "monitor": true, "timeout_seconds": 1200, "schedules": [ { "id": "daily", "weekdays": [1, 2, 3, 4, 5], "hours": [8], "minute": 30, "args": ["--period", "dia-anterior"] } ], "max_retries": 20, "retry_backoff_minutes": [30] } ``` …becomes: ```toml [agent] name = "team-standup" description = "DORA metrics daily standup" monitor = true timeout_seconds = 1200 [run] command = "fish" args = ["./agent.fish"] # the original entrypoint, unchanged [defaults] max_retries = 20 retry_backoff_minutes = [30] [[schedules]] id = "daily" type = "cron" weekdays = [1, 2, 3, 4, 5] hours = [8] minute = 30 args = ["--period", "dia-anterior"] [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } [[notifiers]] driver = "imessage" to = "+5511999999999" rate_limit_minutes = 60 events = ["given_up"] ``` ### 3. Trim the Fish entrypoint You can drop everything dotagent now owns. The reduced agent.fish: * Keeps your data collection + Claude invocation. * **Removes** `agent_init` (env vars are already injected). * **Removes** `agent_heartbeat_start` / `_agent_heartbeat_finish` (dotagent writes the same heartbeat file shape). * **Removes** any inline WARP check (now done by `preflight-warp`). * **Removes** `agent_output_imessage` calls (now done by the built-in `imessage` notifier — declared under `[[notifiers]]`). * Continues to use `roam_publish` / `lib/roam.fish` while `sink-roam` is in progress. Once `sink-roam` is feature-complete you can drop the Roam helpers too. A thin shim (`lib/agent-shim.fish`) that reads the env vars and re-exports the legacy names (`$AGENT_TMPDIR`, `$AGENT_HEARTBEAT_FILE`, ...) lets you migrate without touching the agent body. Source it instead of the original `lib/agent.fish`. ### 4. Install the dotagent daemon dotagent uses **one** plist for the entire system (the daemon manages every agent internally — no per-agent plist): ```bash dotagent install # wrote ~/Library/LaunchAgents/run.avelino.dotagent.plist launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` Now remove each migrated agent's entry from `modules/home-manager/cron.nix` and run `darwin-rebuild switch`. As soon as the entry is gone from `cron.nix`, the dotagent daemon picks up scheduling for that agent. ### 5. Verify the monitor ```bash dotagent status # health dashboard for every discovered agent dotagent tick --dry-run # show what the daemon would dispatch right now tail -F ~/.config/dotagent/logs/run.avelino.dotagent.log ``` The Fish `agent-orchestrator` can be left running during the migration — both read/write the same heartbeat shape, so they don't fight (note the new path: `~/.config/dotagent/state/agents/...` for dotagent vs. `~/.local/state/agents/...` for the Fish framework). When you're confident, remove its launchd entry and delete `agents/agent-orchestrator/`. ### 6. (Optional) move the agent out of dotfiles Since each agent is now self-contained (`agent.toml` + script + assets), you can lift it into its own repo or share it without dragging the rest of your dotfiles along. ## Things that intentionally differ * **No iMessage hardcoding.** The phone number lives in the manifest's `[[notifiers]] driver = "imessage"` block (or in a profile default). dotagent has no opinion about your notification target — pick the driver you want. * **No `agent_mcp` wrappers.** dotagent isn't an MCP proxy; the `mcp` CLI you already use stays where it is. Agents continue to call `mcp` directly. * **`tick` is read-mostly.** dotagent's `tick` doesn't depend on `claude`, `osascript`, or `jq` — it only does scheduling math. Notifications go through the daemon's built-in drivers (`[[notifiers]]`) or, for third-party services, through plugins. # CLI > Every `dotagent` subcommand, what it does, and a minimal example. ``` dotagent [ARGS] [FLAGS] ``` Run `dotagent --help` for the cheat-sheet, `dotagent --help` for per-command flags. Every subcommand documented below mirrors the clap-generated help. | Command | Purpose | | --------------------------------- | ----------------------------------------------------------------------------------- | | [`run`](#run) | Run a single schedule of an agent in the foreground. | | [`tick`](#tick) | One-shot dispatch pass (what the daemon does on each cycle). | | [`daemon`](#daemon) | Long-lived adaptive scheduler. Invoked by launchd / systemd. | | [`api`](#api) | Raw JSONL bridge to the daemon's local Unix socket. | | [`status`](#status) | Textual health dashboard. | | [`daily-summary`](#daily-summary) | Send the end-of-day health summary. | | [`bootstrap`](#bootstrap) | Mark every schedule's window as ok (one-shot, post-install). *Not implemented yet.* | | [`install`](#install) | Generate + write the daemon unit file (launchd / systemd). | | [`uninstall`](#uninstall) | Remove the daemon unit file. | | [`doctor`](#doctor) | Validate manifests, resolve plugin references, warn on drift. | | [`plugin list`](#plugin-list) | List discovered plugins. | | [`plugin invoke`](#plugin-invoke) | Invoke a plugin manually for debugging. *Not implemented yet.* | | [`audit verify`](#audit-verify) | Verify the audit log's hash chain and report how far back the guarantee reaches. | | [`logs`](#logs) | Tail the daemon-captured stdout/stderr for one or all agents. | | [`inspect`](#inspect) | Dump heartbeat + manifest hash + schedule state for one agent. | | [`reload`](#reload) | Send SIGHUP to the running daemon. | | [`run-now`](#run-now) | Force-run an agent immediately, ignoring schedule windows. | | [`mcp`](#mcp) | Serve every discovered agent as an MCP tool over stdio. | | [`memory`](#memory) | Read and curate the long-term memory workspace. | | [`completions`](#completions) | Print a shell completion script (bash / zsh / fish / elvish / powershell). | *** ## `run` Run a single schedule of an agent in the foreground. Useful for development — no daemon involvement, no `on_*` plugin hooks, no notifiers fire. ```bash dotagent run --schedule [--dry-run] ``` | Arg/flag | Meaning | | --------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `NAME` | The agent's `agent.name` (not the directory name). | | `--schedule ID` | Which schedule to use — picks the `args` and the slug from this schedule. | | `--dry-run` | Inject `AGENT_DRY_RUN=true`. The script can decide what to do (typically: skip side effects). dotagent skips writing the heartbeat. | **What it does**: discovers the manifest, injects the [env vars](/reference/env-vars), spawns the command, captures stdout/stderr, prints stdout to your terminal, and exits with the agent's exit code. **Notifiers and sinks do NOT fire** — use `run-now` for that. **Example**: ```bash dotagent run hello --schedule every-2min dotagent run finops-weekly --schedule weekly --dry-run ``` *** ## `tick` One-shot dispatch pass. Same logic the daemon loop runs once, but without sleeping or installing signal handlers. Used by the daemon internally; exposed so you can debug "what would dotagent do RIGHT NOW". ```bash dotagent tick [--dry-run] [--verbose] ``` | Flag | Meaning | | ----------- | ------------------------------------------------------------------------------------ | | `--dry-run` | Don't actually spawn anything. Prints `scanned N; would dispatch M; next event: ts`. | | `--verbose` | (Reserved — currently a no-op.) | **Example**: ```bash dotagent tick --dry-run # → (dry-run) scanned 4 agent(s); would dispatch 1; next event: 2026-05-19T08:30:00-0300 ``` When debugging "is my agent gonna fire?", run this. It tells you which schedules dotagent considers due. *** ## `daemon` The long-lived adaptive scheduler. **You don't invoke this directly** — launchd / systemd does, via the unit file `dotagent install` writes. ```bash dotagent daemon ``` If you run it manually (e.g., for development), it stays in the foreground until you `Ctrl+C` (SIGINT) or kill it (SIGTERM). It: * Discovers manifests + plugins * Writes `state/daemon.pid` * Initializes structured JSON logging into `logs/daemon/dotagent.log` * Initializes OTel export if `[telemetry]` is configured * Loops forever: `tick → sleep until next event → wake on SIGHUP` * Cleans up the PID file on exit (Drop guard) The daemon process responds to: | Signal | Effect | | --------- | --------------------------------------------------------------- | | `SIGHUP` | Wake immediately, re-read manifests + plugins on the next tick. | | `SIGTERM` | Graceful shutdown. Audit log gets a `DaemonStopped` entry. | | `SIGINT` | Same as SIGTERM. | See [`guides/daemon-lifecycle.md`](/guides/daemon-lifecycle) for how to install / start / stop / reload the daemon end-to-end. *** ## `api` Bridge raw JSON Lines between stdin/stdout and the daemon's local Unix socket. The command does not parse, render, or persist messages, so it can be used as a backend for scripts or a TUI. ```bash dotagent api [--socket ] ``` By default it connects to `$DOTAGENT_HOME/api.sock` (normally `~/.config/dotagent/api.sock`). `--socket` overrides the path for tests or an explicit endpoint. Every input frame is forwarded with its original bytes and line ending; every socket frame is forwarded to stdout the same way. When stdin reaches EOF, the client half-closes its request side and continues reading responses/events until the daemon closes the socket. Diagnostics go to stderr. There is no prompt or TUI rendering. **Example**: ```bash printf '%s\n' '{"id":"status","method":"status.get"}' | dotagent api ``` See [`Local Client API`](/reference/local-api#raw-cli-bridge) for the wire contract. *** ## `status` Textual health dashboard. Read-only — never writes to audit, never dispatches. Takes no flags. ```bash dotagent status ``` **Output**: ``` ═══ Agent Health · 2026-08-06 17:51 ═══ ✅ ok 2/5 ⚠️ degraded 1 ❌ failing 1 🕑 stale 1 AGENT/SCHEDULE STATE LAST RUN REASON ──────────────────────────────────────────────────────────────────────────────────────────────────── disk-alert/every-15min ❌ failing 2026-08-06T16:00:04-0300 2 attempts, will retry hn-digest/weekday-morning ⚠️ degraded 2026-08-06T08:12:41-0300 recovered after 1 attempt backup-offsite/nightly 🕑 stale 2026-08-01T11:40:23-0300 window missed 7330min ago (stale) inbox-triage/every-90min ✅ ok 2026-08-06T17:42:58-0300 ok inbox-triage/weekly-cleanup ✅ ok 2026-08-03T10:44:53-0300 no window today · last success ok Logs: ~/.config/dotagent/logs/ State: ~/.config/dotagent/state/agents/ Audit: ~/.config/dotagent/audit.log ``` One row per `(agent, schedule)` pair — `inbox-triage` above declares two schedules and gets two rows, each with its own health. The counter at the top counts rows, not agents: `2/5` there is five schedules across four agents. Rows are ordered **most-urgent-first**: `failing` → `degraded` → `stale` → `ok`. Within a bucket, discovery order. | Column | Meaning | | ---------------- | -------------------------------------------------------------------------------------------------------------------- | | `AGENT/SCHEDULE` | `agent.name` + the schedule's `id`. | | `STATE` | One of the four [health states](/reference/agent-spec#health-states). | | `LAST RUN` | `finished_at_iso` from the heartbeat — when the run **ended**, success or not. `never` if there is no heartbeat yet. | | `REASON` | Why that state, see below. | Agents with `monitor = false` in their manifest are excluded (those are typically one-shot/manual examples). ### Reading the `REASON` column The state comes from the heartbeat *and* from the window file for the window currently due — the same file the daemon writes retries against. The reason tells you which of the two decided: | State | Reason | What produced it | | ---------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `ok` | `ok` | The due window succeeded on the first try. | | `ok` | `no window today · last success ok` | Nothing is due — a weekday-only schedule on a Sunday, an hour outside `hours` — and the last run succeeded. | | `degraded` | `recovered after N attempts` | The window did succeed, but burned `N` **failed** attempts first. | | `failing` | `N attempts, will retry` | The window passed without success; the retry budget is not spent yet. | | `failing` | `gave up after N attempts` | Retry budget exhausted — the window is marked `given_up`. | | `failing` | `window due Nmin ago, no attempt` | The window is overdue and **no window file exists** — the daemon never dispatched. Usually means the daemon is not running. | | `stale` | `window missed Nmin ago (stale)` | Older than `stale_after_minutes`; retrying is no longer useful. | | `stale` | `never ran` | No heartbeat and no window due — the schedule has never fired. | The count is singular-aware: `1 attempt`, `2 attempts`. `N` in `recovered after N attempts` counts **failed** attempts, not dispatches. The window file's `attempts` counter is bumped after every dispatch including the one that succeeded, so a first-try success lands on disk as `attempts: 1` and still reports `ok`. See [Health states](/reference/agent-spec#health-states). For `stale`, an `interval` schedule is judged against the **first** window it missed after its last success, not the rolling window dispatch uses — so a schedule broken for weeks reports the full age. ### Live subprocesses When the daemon is running and has children alive, `status` appends a second table read from the supervisor snapshot (`state/supervisor.json`). It is omitted entirely when the snapshot is missing or empty. ``` ─── Live subprocesses (2 running) ─── KIND PID OWNER LABEL AGE / DEADLINE ──────────────────────────────────────────────────────────────────────────────────────────────────── ⚠️ agent 41207 hn-digest hn-digest.weekday-morning 246s / 300s (82%) sink 41219 hn-digest sink-file.invoke 3s / 30s (10%) ``` `KIND` is one of `agent`, `persistent`, `preflight`, `sink`, `notify`, `skill`, `plugin_info`, `plugin_validate`. Rows sort by deadline pressure, highest first, and are flagged `⚠️` at ≥80% of the deadline and `🔴` at ≥100%. A `persistent` row never gets a warning icon: its clock is the idle window before recycling, not a run running late, so reaching it is the pool working as designed. Those rows mark the clock `idle`, and their `AGE` is time since the last answer. See [Lifecycle](/concepts/lifecycle). Agents launched by `dotagent mcp` or `run-now` run in **that** process, not under the daemon, so they never appear here. The snapshot carries two fields this table deliberately does not show: `pgid` and `command`. They are identity proof for the next daemon's [boot orphan reap](/guides/daemon-lifecycle#boot-orphan-reap), not information an operator reads at a glance. Full schema in [`paths.md`](/reference/paths#statesupervisorjson). *** ## `daily-summary` Send the end-of-day health summary. The daemon delivers it once a day at `[daily_summary].time` (default `22:45` local) and schedules its own wake-up for that moment. `grace_minutes` (default `30`) is the tail that still counts as on-time when the wake-up could not happen at all — laptop closed, machine off. Both are configurable; see [`[daily_summary]`](/guides/config-reference#daily_summary). ```bash dotagent daily-summary [--dry-run] ``` | Flag | Meaning | | ----------- | -------------------------------------------------------------------------- | | `--dry-run` | Print the message and its destinations to stdout instead of delivering it. | Typing the command delivers even when `enabled = false`. That flag governs the daemon's scheduled delivery; someone who ran this asked for this one. **Destinations** come from `[[daily_summary.notifiers]]`, which takes the same entries a manifest's `[[notifiers]]` takes. With none configured the summary goes to the `desktop` driver — the only one that needs no address and sends nothing off the machine. **Output** — the same classification `status` renders, folded into a message short enough for a phone. Healthy schedules collapse into the count; only the unhealthy ones are named, and empty buckets are dropped: ``` 📊 Agents · 2026-08-06 2/5 ok ❌ Failing: · disk-alert/every-15min — 2 attempts, will retry ⚠️ Degraded: · hn-digest/weekday-morning — recovered after 1 attempt 🕑 Stale: · backup-offsite/nightly — window missed 7330min ago (stale) ``` An all-green day is three lines: the header, `N/N ok`, and nothing else. `--dry-run` appends where it would have gone: ``` → would deliver to: telegram, desktop ``` Each delivery is audited as `plugin_invoked` with `plugin: "notifier:"`, on failure as well as on success — a summary that reached nobody is answerable from `dotagent audit` instead of from silence. *** ## `bootstrap` > **Not yet implemented.** Calling it returns `bootstrap — not yet implemented`. Intent: mark every schedule's "current window" as ok in one shot, so a fresh install doesn't trigger a flood of `failing` notifications for schedules that haven't had a chance to run yet. Workaround for now: run the daemon for one full cycle of each schedule and it'll fill in. *** ## `install` Generate and write the daemon unit file. **One unit per system** — not one per agent. ```bash dotagent install [--all] [NAME] ``` | Arg/flag | Meaning | | -------- | ------------------------------------------------------------- | | `--all` | (Accepted for backwards compat — **no-op**, prints a notice.) | | `NAME` | (Accepted for backwards compat — **no-op**, prints a notice.) | `--all` and `NAME` are accepted but ignored — dotagent now uses one daemon unit (`run.avelino.dotagent`) that manages every discovered manifest internally. **What gets written**: * **macOS**: `~/Library/LaunchAgents/run.avelino.dotagent.plist` * `RunAtLoad=true`, `KeepAlive=true`, `ThrottleInterval=10` * **Linux**: `~/.config/systemd/user/run.avelino.dotagent.service` * `Restart=always`, `RestartSec=10` The unit points at the currently-running `dotagent` binary (resolved via `std::env::current_exe`). If you move the binary, re-run `install`. After writing the file, the command prints the platform-specific "Next step" line to actually load the unit: ```bash # macOS launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/run.avelino.dotagent.plist # Linux systemctl --user daemon-reload && systemctl --user enable --now run.avelino.dotagent ``` See [`guides/daemon-lifecycle.md`](/guides/daemon-lifecycle). *** ## `uninstall` Remove the daemon unit file. Idempotent (no error if nothing to remove). ```bash dotagent uninstall [--all] [NAME] ``` Same flag caveats as `install` — both are no-ops. **Does NOT stop the daemon** if it's running — `launchctl bootout` / `systemctl --user disable --now` first if you want a clean stop. *** ## `doctor` Validate every discovered manifest, resolve plugin references, and warn on inconsistencies. **Always safe to run** — read-only. ```bash dotagent doctor ``` **What it checks**: * Manifest TOML is parseable. * `agent.name` and `[run].command` are non-empty. * Schedule ids are unique within each manifest. * Every plugin referenced by `[[preflight]]` / `[[on_success]]` / `[[on_failure]]` / `[[notifiers]] driver = "plugin"` resolves to a binary on `$PATH` / the discovery dirs. * Each manifest has a `[security]` section. (If absent, emits a warning — schema-only in v0; see [`security/threat-model.md`](/security/threat-model).) * Compares each manifest's sha256 against `state/known_manifests.json`. Mismatch → "manifest drift since last daemon run" warning. **Exit code**: 0 if 0 errors, non-zero otherwise. Warnings do not trigger a non-zero exit. **Example**: ```bash dotagent doctor # ✓ hello: manifest ok # notifier driver=desktop (built-in) # ⚠ hello: no [security] section — blast radius is unbounded. # ✓ finops-weekly: manifest ok # plugin sink-roam → /opt/homebrew/bin/dotagent-plugin-sink-roam # plugin preflight-warp → /opt/homebrew/bin/dotagent-plugin-preflight-warp # notifier driver=imessage (built-in) # # summary: 2 agent(s), 0 error(s), 1 warning(s) ``` *** `doctor` also reports inbound Telegram when `[telegram]` is present in `config.toml` — whether the ingress is on, and the two ways it can be half-configured: ``` telegram ingress: on — 1 allowed user(s), dispatcher 'telegram-assistant' telegram ingress: OFF — bot_token set but allowed_user_ids is empty ⚠ an empty allowlist means nobody, never everybody. Add your numeric user id. ``` It stays silent when the section is absent, since the ingress is off by default. A `dispatcher_agent` that does not resolve is a warning: every accepted message would fail after passing the allowlist. It also prints where long-term memory lives: ``` memory: /Users/me/.config/dotagent/outl (default) ``` A path set in `[memory] workspace` that holds no outl workspace is a warning — the default path is scaffolded automatically, a configured one is not. And how many skills it found, across every search root: ``` skills: 8 found, including ~/.claude/skills ``` A `SKILL.md` that fails to parse is a warning, not an error — nothing stops running, an assistant just answers without a procedure it should have had. Two skill names that sanitize to the same tool name are also warned about: only the first is callable, and a skill you wrote and cannot call is otherwise a mystery. Commands get the same treatment, with one extra check. They carry two derived names — the MCP tool and the Telegram menu entry — and Telegram's `[a-z0-9_]{1,32}` has no hyphens, so two commands can be distinct in the catalog and collide in the menu: ``` commands: 4 found ⚠ weekly-numbers (…/a/weekly-numbers.md), weekly_numbers (…/b/weekly_numbers.md) all want /weekly_numbers — only the first is in the menu ``` The files are named, not just the commands: two names differing only by `-` versus `_` are near-identical on screen, and the useful question is which one to rename. See [Commands](/concepts/commands#two-names-two-collision-rules). Manifests that fail to parse are listed with `✗` and counted as errors. They no longer abort the scan — the healthy agents are still reported below them. *** ## `plugin list` List every plugin referenced by any discovered manifest, with its resolved path + advertised version + kinds. ```bash dotagent plugin list ``` **Output** (tab-separated): ``` preflight-warp 0.0.1 "preflight" /opt/homebrew/bin/dotagent-plugin-preflight-warp sink-roam 0.0.1 "sink" /opt/homebrew/bin/dotagent-plugin-sink-roam ``` If a plugin is referenced but not on `$PATH`, you'll see `(not found: ...)`. Run `dotagent doctor` for a friendlier report. *** ## `plugin invoke` > **Not yet implemented.** Currently returns `plugin invoke — not yet implemented`. Intent: run a plugin manually with a JSON payload. Until this lands, invoke directly: ```bash echo '{ "kind": "preflight", "agent": "test", "schedule": "test", "event": "preflight", "config": {} }' | dotagent-plugin-preflight-warp invoke ``` See [`reference/plugin-protocol.md`](/reference/plugin-protocol) for the payload shape. *** ## `audit verify` Verify the hash chain in `$DOTAGENT_HOME/audit.log` and print what verification could actually establish. ```bash dotagent audit verify # the live audit.log only dotagent audit verify --full # follow the seams back through every segment dotagent audit verify --json # one JSON line, for scripts ``` The daemon runs the same check at boot, but it only asks yes/no. This asks *how far back*, which is the question a rotated log makes real: a history the operator pruned must not read the same as one somebody beheaded. **Verdicts:** | Output | Meaning | Exit | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `chain intact from GENESIS` | Every entry still on disk was checked, back to the first line ever written | `0` | | `chain intact since ` | Every entry checked links, and the oldest is a **seam** naming where the rest went. The text says whether that segment is still on disk (re-run with `--full`) or gone (retention — or evidence removed; the chain cannot tell those apart) | `0` | | `unexplained truncation` | The oldest entry links to a hash nothing accounts for: no `GENESIS`, no seam. The head of the log was removed | `1` | | `chain broken at position N` | A link mismatch, or a line nobody can parse, inside the data that is present. Names the segment and both hashes | `1` | ``` ✗ chain broken at position 42 in: audit.log.20260806T101500 expected: c8d2f1… actual: 9a3b07… file: ~/.config/dotagent/audit.log scope: full — following seams back through every segment on disk segments: 2 on disk (audit.log.20260806T101500, audit.log.20261104T093000) ``` Without `--full` the walk stops at the first seam, which is what the daemon does at boot — the live file is the only one that changes, so it is the cheap check worth running every time. `--full` is the one to run when you actually want the guarantee to reach `GENESIS`. What the chain does and does not prove (short version: it catches partial edits, never a total rewrite) is in [`security/threat-model.md`](/security/threat-model). *** ## `logs` Tail the daemon-captured stdout/stderr. ```bash dotagent logs [NAME] [-n LINES] [--follow] [--schedule ID] ``` | Arg/flag | Meaning | | ----------------- | ------------------------------------------------------------------------------------------------------------------- | | `NAME` (optional) | Tail one agent's logs. **Omit** to tail every agent at once (each chunk is prefixed by `tail` with `==> path <==`). | | `-n LINES` | Print the last `N` lines (default 50). | | `--follow` / `-f` | `tail -F`-style follow. Survives rotation. | | `--schedule ID` | (Reserved — currently unused.) | Reads from `~/.config/dotagent/logs/agents//.log` plus any rolled `.log.YYYY-MM-DD` files (skips `.gz` — `tail` can't follow compressed files). **Examples**: ```bash # One agent, follow dotagent logs hello --follow # Last 200 lines of every agent at once dotagent logs -n 200 # Pipe to jq is doable but the file is raw text, not JSON — see the # daemon log if you want structured data. ``` For **structured** logs (the daemon's own tracing output) read `~/.config/dotagent/logs/daemon/dotagent.log` directly: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq . ``` See [`guides/observability.md`](/guides/observability) for the log schema. *** ## `inspect` Dump heartbeat + manifest hash + schedule state for one agent. ```bash dotagent inspect ``` **Output**: ``` agent: hello manifest_dir: /Users/me/.config/dotagent/agents/hello manifest_sha: a3f9... (first seen 2026-05-19T14:00:00-0300) monitor: true timeout: 30s ─── schedule 'every-2min' (slug=default) ─── { "name": "hello", "slug": "default", "args": [], "started_at": 1747680001, "started_at_iso": "2026-05-19T14:30:01-0300", "finished_at": 1747680002, "exit_code": 0, ... } ``` Use this when "is the heartbeat fresh?" / "did the last run succeed?" is your question. *** ## `reload` Send SIGHUP to the running daemon. The daemon picks up new manifests and plugin changes on its next tick. ```bash dotagent reload ``` Reads `~/.config/dotagent/state/daemon.pid` and sends `SIGHUP`. Fails if: * The PID file is missing (daemon not running). * The PID exists but the process is gone (stale pidfile). If you swapped the **`dotagent` binary itself**, SIGHUP isn't enough — restart the daemon via launchctl/systemctl. See [`guides/daemon-lifecycle.md`](/guides/daemon-lifecycle). *** ## `run-now` Force-run an agent immediately, ignoring schedule windows. Unlike `run`, this DOES fire preflight, sinks, and notifiers — it's a single-shot version of what the daemon would do. ```bash dotagent run-now [--schedule ID] [--json] ``` | Arg/flag | Meaning | | --------------- | ----------------------------------------------------------------------------- | | `NAME` | The agent's `agent.name`. | | `--schedule ID` | Which schedule's `args` to use. If omitted, uses the first schedule declared. | | `--json` | Emit one machine-parseable JSON line instead of the human report. | **Example**: ```bash dotagent run-now finops-weekly --schedule weekly ``` ``` ✓ finops-weekly/weekly ok 47s stdout: wrote 12 rows to /Users/me/reports/weekly.md ``` A preflight abort names the plugin that stopped it instead of a run result: ``` ⊘ finops-weekly/weekly aborted by preflight 1s plugin: preflight-warp suggest: warp-cli connect ``` Colour follows the TTY and honours `NO_COLOR`. Truncated output says so explicitly rather than trailing off. Use this to: * Trigger an agent after fixing a problem (don't wait for the next window). * Manually exercise the full plugin chain (preflight → spawn → sink → notify). *** ## `mcp` Serve every discovered agent as an MCP tool over stdio. JSON-RPC 2.0, one object per line. ```bash dotagent mcp ``` Takes no flags — the catalog comes from the manifests already on disk. Point any MCP client at it: ```json { "mcpServers": { "dotagent": { "command": "dotagent", "args": ["mcp"] } } } ``` **Example** — list the catalog by hand: ```bash printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | dotagent mcp # → {"jsonrpc":"2.0","id":1,"result":{"tools":[{"name":"run-disk-alert",...}]}} ``` The catalog is not only agents: `skill-*` tools carry the procedures found under `~/.config/dotagent/skills/` and `~/.claude/skills/`, `command-get` and `command-list` resolve the commands under `~/.config/dotagent/commands/`, and the five `memory-*` tools appear when `[memory]` is on. See [Skills](/concepts/skills) and [Commands](/concepts/commands). Agents run in **this** process, like `run-now` — not through the daemon — so the subprocess tree does not appear in `dotagent status`. State is keyed off the `trigger-mcp` slug so an on-demand run never overwrites the scheduled history of the same agent. stdout carries protocol only; logging goes to stderr, so `RUST_LOG=debug dotagent mcp` stays safe to run under a client. Full protocol reference: [MCP server](/reference/mcp). *** ## `memory` Read and curate the long-term memory workspace — the same store `dotagent mcp` exposes to agents, reachable from a shell and **without a running daemon**. That last part is the point: consolidating a day's raw facts into durable ones is a job an agent should be able to do on a schedule, and it can only do that if the verbs exist outside the daemon process. ```bash dotagent memory recall [QUERY] [--topic ] [--limit ] [--json] dotagent memory remember [--topic ]... dotagent memory supersede [--topic ]... dotagent memory forget dotagent memory topics dotagent memory stats ``` `recall` ranks by shared words first and recency second; an empty query lists the most recent facts. `--topic` asks the graph instead — every fact linked to that subject, which is the better question when you know the subject. **Example** — find a fact, then correct it: ```bash dotagent memory recall "reunião" # → [01M0JSC92RBJFXKAX25JX34M2Q] 2026-08-21: prefere reunião depois das 14h (agenda, reuniao) ×2 dotagent memory supersede 01M0JSC92RBJFXKAX25JX34M2Q "prefere reunião só de manhã" --topic agenda # → Replaced [01M0JSCG3ETZ65MX9Q05DYNP0D]: prefere reunião só de manhã ``` Every line leads with the fact's id, because the follow-up move takes it. The `×2` suffix means the fact was stated more than once — restating a fact reinforces the one already stored rather than filing a copy. `supersede` keeps the old fact readable in its journal and stops recalling it; `forget` deletes outright. Correcting something that changed wants the first; removing something that should never have been stored wants the second. `stats` reports what the store holds: ```bash dotagent memory stats # → facts: 2 # reinforced: 1 # superseded: 0 # topics: 4 ``` `--json` emits one JSON object per line, provenance included, for scripting. Full reference: [Memory](/concepts/memory). *** ## `completions` Print a shell completion script. The script wires **dynamic** completion of agent names: pressing `` after `dotagent run`, `inspect`, `run-now`, `logs`, `install`, or `uninstall` runs `dotagent _list-agents` (a hidden helper) and lists every manifest currently on disk — no stale baked-in list to refresh. ```bash dotagent completions ``` | Arg | Values | | ------- | ------------------------------------------------- | | `SHELL` | `bash` · `zsh` · `fish` · `elvish` · `powershell` | > Dynamic agent-name completion is wired for `bash`, `zsh`, and `fish`. `elvish` / `powershell` get subcommand + flag completion only. **Install**: ```bash # fish — eval in current shell, or save to ~/.config/fish/completions/dotagent.fish dotagent completions fish | source # zsh — drop into a fpath dir dotagent completions zsh > ~/.zfunc/_dotagent # bash — drop into the bash-completion dir dotagent completions bash > ~/.local/share/bash-completion/completions/dotagent ``` Example: ```bash $ dotagent run hel hello-fish hello-go hello-python hello-rust ``` *** ## Exit codes | Code | Meaning | | ------------- | -------------------------------------------------------------- | | `0` | Success. | | `1` | Generic failure (manifest invalid, plugin not found, etc.). | | `124` | `dotagent run` only — the agent timed out (SIGTERM + SIGKILL). | | Anything else | `dotagent run` only — the agent's exit code is passed through. | *** ## Environment variables dotagent reads a small set of env vars for configuration overrides (`DOTAGENT_HOME`, `DOTAGENT_ROOT`, `DOTAGENT_PLUGIN_PATH`, `RUST_LOG`, `OTEL_EXPORTER_OTLP_HEADERS`). See [`env-vars.md`](/reference/env-vars) for the complete list. dotagent INJECTS env vars into the agent subprocess (`AGENT_NAME`, `AGENT_TMPDIR`, etc.). Same doc. *** ## Related * [Daemon lifecycle](/guides/daemon-lifecycle) — install / start / stop / reload * [Troubleshooting](/guides/troubleshooting) — sintoma → diagnostic * [Agent spec](/reference/agent-spec) — `agent.toml` schema * [Plugin protocol](/reference/plugin-protocol) — for `plugin list` / `plugin invoke` * [Observability](/guides/observability) — log format, OTel # Agent spec (agent.toml) This document defines the contract between an agent and `dotagent`. If your binary or script honors this contract, dotagent can schedule, run, monitor, retry, and notify on it — regardless of the language it's written in. ## Layout Every agent lives in a single directory. The minimum is: ``` my-agent/ agent.toml # manifest (this spec) agent.fish # entry point (or `agent.py`, `agent.go`, a binary, ...) ``` The manifest tells dotagent **what** the agent is, **how** to run it, **when** to run it, **what** to do on success/failure, and **which preflight checks** to run before invoking it. ## Manifest — `agent.toml` ```toml # Required: identity [agent] name = "my-agent" # unique within the dotagent root description = "What this does." # optional monitor = true # default: true. false = excluded from `tick` timeout_seconds = 1800 # hard kill SIGTERM→SIGKILL; default 1800 # persistent mode: deadline for ONE request # Required: how to run it [run] command = "fish" # the executable args = ["./agent.fish"] # static args (the schedule's args are appended) working_dir = "." # optional, relative to the manifest dir protocol = "assistant-v1" # optional stdout protocol for triggered delivery # Optional: environment variable injection [env] inherit = true # default true — inherit parent env extra = { LOG_LEVEL = "info" } # added on top # Optional: how long one process lives. Absent = "oneshot", the shape every # agent had before this section existed. See docs/concepts/lifecycle.md. [lifecycle] mode = "persistent" # "oneshot" (default) | "persistent" key = "chat_id" # payload field deciding which instance # answers; absent = one instance total idle_timeout_seconds = 1800 # recycle after this long with no request max_invocations = 120 # recycle after this many; 0 = unlimited max_instances = 8 # ceiling; past it, LRU is evicted startup_timeout_seconds = 30 # window to answer the `hello` handshake # Optional: agent-wide defaults for retry/backoff/stale [defaults] max_retries = 3 retry_backoff_minutes = [5, 15, 30] stale_after_minutes = 120 # Optional: schedules. Trigger-only agents may declare none. [[schedules]] id = "daily" # unique within this manifest type = "cron" # "cron" | "interval" | "expression" weekdays = [1, 2, 3, 4, 5] # 0=Sun..6=Sat (matches launchd Weekday) hours = [8] minute = 30 args = ["--period", "dia-anterior"] # appended to [run].args # Per-schedule overrides (optional): max_retries = 20 retry_backoff_minutes = [30] stale_after_minutes = 240 [[schedules]] id = "every-90" type = "interval" interval_minutes = 90 args = [] on_battery = "defer" # "run" (default) | "defer" # overrides [power] in config.toml for # this schedule only # Optional: preflight checks (run BEFORE the agent; abort if any fail) [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } # Optional: what clears this check. Declaring it publishes one MCP tool, # `remediate--`, so an assistant can offer to fix it instead # of only reporting it. Split on whitespace into argv — there is no shell. # # The plugin's own `suggest` string is never executable: a command a plugin # wrote, run from a chat message, is arbitrary execution. This is the # operator saying it, in a file under review. See threat model V12. # remediation = "warp-cli connect" # Optional per-hook deadline (seconds). Default: 30s for preflight, # 300s for on_success/on_failure invocations. The supervisor kills the # whole process group (TERM → 5s grace → KILL) when exceeded. # timeout_seconds = 60 # Optional: notifications (built into the daemon, no plugin subprocess) [[notifiers]] driver = "imessage" to = "+5511999999999" rate_limit_minutes = 60 events = ["attempt_failed", "given_up", "recovered"] # empty = all events [[notifiers]] driver = "desktop" title = "finops-weekly" sound = true events = ["given_up"] # Optional: post-success sinks (still plugin-protocol) [[on_success]] plugin = "sink-file" config = { path = "/tmp/last-success.txt", mode = "overwrite" } # timeout_seconds = 600 # override default 300s for slow sinks # Optional: security intent (schema-only in v0 — see threat-model) [security] allowed_commands = ["fish", "/usr/local/bin/fish"] allowed_plugins = ["preflight-warp", "sink-roam"] network = "allow" # or "deny" or ["api.github.com", "..."] filesystem_writable = ["/Users/me/reports"] # $AGENT_TMPDIR / heartbeat always writable env_passthrough = ["PATH", "HOME", "LANG"] ``` ### `[lifecycle]` — how long one process lives Absent means `mode = "oneshot"`: spawn, run, exit, one process per event. That is the default and it is not going to change. `mode = "persistent"` keeps the process alive between runs and delivers requests over [JSON lines](/reference/persistent-protocol). The agent's own shape changes with it — it reads requests and writes answers on a loop instead of running once — so this is opt-in per manifest and never inferred. | Field | Default | Meaning | | ------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `mode` | `"oneshot"` | `"oneshot"` or `"persistent"`. Anything else is a parse error, not a silent fallback. | | `key` | *(none)* | Trigger-payload field that decides which instance answers. Absent = one instance for the whole agent. | | `idle_timeout_seconds` | `1800` | Recycle after this long with no request. Must be > 0. | | `max_invocations` | `120` | Recycle after this many requests. `0` = unlimited. | | `max_instances` | `8` | Ceiling on live instances. Past it, the least recently used is terminated. Must be ≥ 1. | | `startup_timeout_seconds` | `30` | Window to answer the `hello` handshake. Must be below `[agent] timeout_seconds` — a handshake that outlasts the request deadline means the first message can never land, and `doctor` refuses the manifest. | Two things change meaning in this mode: * **`[agent] timeout_seconds` is the deadline for one request**, not the lifetime of the process. Exceeding it recycles the instance. * **`AGENT_TRIGGER_*` is not injected.** Those variables describe a single message and the process outlives many; the context rides in each request frame instead. `AGENT_LIFECYCLE` and `AGENT_PERSIST_KEY` are added — see [env-vars](/reference/env-vars). Full reasoning in [Lifecycle](/concepts/lifecycle). ### `[assistant]` — conversational harness, opt-in Absent means none of this happens — a plain triggered run, byte for byte what every agent did before the section existed. Present, the daemon becomes the conversation's bookkeeper **for pointers, never transcripts**: it records which model session served the chat, which toolkit it ran under and how much transcript it produced, reinjects the pointer on the next trigger, retires it when the transcript outgrows the ceiling, marks that next trigger as context-retired, and captures `MEMO:` lines from replies into the memory workspace. | Field | Default | Meaning | | ------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `enabled` | `true` | Master switch — disable without deleting the block. | | `memory` | `true` | Recall stored facts before the run (`AGENT_ASSISTANT_MEMORY`) and persist `MEMO:` capture lines after it. Restating a fact already stored reinforces it rather than duplicating it. | | `transcript_bytes_max` | `409600` | Retirement ceiling. A session frame reporting more bumps the generation and clears the pointer. Must be > 0. | | `[[assistant.toolkit.servers]]` | *(empty)* | MCP servers the conversation runs with. Empty = the agent provisions its own. | Toolkit servers are `kind = "dotagent"` (the daemon's own MCP server), `kind = "http"` + `url`, or `kind = "stdio"` + `command`/`args`. Names must be unique — two servers mapping to the same `mcp.json` key are a load error, not a silent last-write-wins. The assembled config lands content-addressed at `state/assistant/toolkit-.json` (0600); the hash doubles as the session-invalidation key, so adding a server starts a fresh model session on the next message without any agent-side state. What the agent gets out of it: the five `AGENT_ASSISTANT_*` variables (see [env-vars](#environment-variables-dotagent-injects)), and the guarantee that trailing `MEMO: | topics: a, b` lines never reach the chat — they are stripped after the run and flushed to memory off the reply path. Full reasoning in [Telegram](/concepts/telegram) and [Local Client API](/reference/local-api). ### `[memory]` — memory capture for a plain agent, opt-in The `[assistant]` harness above reads `MEMO:` lines out of a conversation reply. This section is the same capture for every other agent: a scheduled run that learns something durable prints it, and the fact outlives the run instead of scrolling past in a log. ```toml [memory] capture = true topics = ["ops"] ``` | Field | Default | Meaning | | --------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `capture` | `true` | Scan this agent's stdout for `MEMO:` lines. Writing the section at all is the opt-in; the field exists to switch it off without deleting the topics next to it. | | `topics` | *(empty)* | Topics added to every fact this agent files, on top of whatever the `MEMO:` line named. | #### `[memory.extractor]` - extracting facts from ordinary agent output An ordinary agent can delegate memory extraction to a command when its output does not contain explicit `MEMO:` lines. The command receives the successful run output as JSON on stdin and must print zero or more `MEMO:` lines on stdout. ```toml [memory.extractor] command = "bash" args = ["./extract-memory.sh"] timeout_seconds = 45 ``` | Field | Default | Meaning | | ----------------- | ------------ | ----------------------------------------------------------------- | | `command` | *(required)* | Program to run. Arguments are passed directly; no shell is added. | | `args` | `[]` | Arguments passed to `command`. | | `timeout_seconds` | `45` | Maximum time allowed for the extractor. | The extractor runs only after a successful scheduled run and is best-effort: an extractor failure does not change the run result. Manifest `topics` are added to extracted facts, and duplicate topics are removed. Agents using the `[assistant]` harness with `memory = true` skip this extractor, so the same turn cannot be captured twice. Absent means the agent's output is never read for facts. That default is deliberate: most agents print status, not knowledge, and a store that absorbs status is a store whose recall returns status. Two more rules apply, and both exist to keep the store trustworthy: * **Successful runs only.** A failing run's output is a symptom. Filing "could not reach the API" as durable means recalling it as true next month, long after the outage ended. * **The assistant harness wins.** An agent that also declares `[assistant]` with `memory = true` is skipped here, so the same fact is never filed through two paths. Facts captured this way record the agent's name as provenance, so `dotagent memory recall --json` can tell you which run put a fact there. Unlike the assistant path, the `MEMO:` line is **not** stripped from the output. There is no chat to keep it out of — the line is part of the agent's log, and seeing what it filed is how you notice it filing the wrong things. > **Not the same `[memory]` as `config.toml`.** The daemon-level section of that name says *where* the workspace lives and whether memory exists at all ([config reference](/guides/config-reference#memory)). This one says whether *this agent* writes to it. #### `[assistant.extractor]` — capturing what the model did not volunteer Without it, memory exists only when the dispatcher's own prompt tells the model to end replies with `MEMO:` lines and the model complies. Both are outside the daemon. Swap the dispatcher for one whose prompt never mentions `MEMO`, or let a model finish a long conversation without one, and the store stops growing — with no error anywhere, because nothing failed. Silence is the failure mode. ```toml [assistant.extractor] command = "bash" args = ["/path/to/extract-memory.sh"] timeout_seconds = 45 ``` After a reply is delivered, the daemon hands the turn to this command on stdin as one JSON object and files whatever `MEMO:` lines come back: ```json {"message": "…", "reply": "…", "source": "telegram", "session": "chat-9"} ``` | Field | Default | Meaning | | ----------------- | ------------ | ----------------------------------- | | `command` | *(required)* | Program to run. argv, never a shell | | `args` | `[]` | Arguments | | `timeout_seconds` | `45` | Wall-clock ceiling | Stdout is read with the same parser as the reply path — one format, two producers — so an extractor is written against syntax already documented. **It runs off the reply path.** The answer has already reached the chat by the time this starts, so a slow extractor costs nobody a wait, and a broken one costs a turn's memory rather than the turn. **The model's own `MEMO:` lines stay a shortcut, not a requirement.** When it volunteers one, the daemon keeps it: the model has the turn in context and it costs nothing. When it does not, the extractor is the net. Both sources merge, and restating a fact reinforces it rather than duplicating it. **Why a declared command rather than inference in the daemon.** dotagent is not an AI runtime ([architecture](/concepts/architecture)). The daemon decides *when* to extract and what to do with the result; the judgment runs in a process the operator named, supervised and deadlined like everything else. #### What a dispatcher can run on the machine Nothing, unless `[os]` is configured in `config.toml`. That section is daemon-level rather than per-agent for the same reason `[telegram]` is: one machine has one answer about which binaries are reachable, and a manifest that could widen it would make every manifest a security boundary. When it is on, a dispatcher gains two tools — `os-list` and `os-run` — and the conversation gains the `!` prefix. They are not the same door: | | who chooses | what guards it | | --------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------- | | `os-run` | the model | `[os] allow`, and `confirm`-class commands are **refused outright**: a model may not answer its own confirmation | | `!rm -r /tmp/x` | the sender, typed | `[os] allow`, then a `!!` reply for `confirm`-class commands | A model composes from whatever entered its context, so a forwarded message can steer it. A typed line cannot be steered by content at all — which is why the confirmation belongs to the person and `os-run` cannot supply it. When the model wants something on the confirm list, the tool answers by telling it to ask the person to type the line. Beyond those two, `[[os.tool]]` entries publish individual binaries as named tools with a description, so the dispatcher can tell that `outl` exists and what it is for without being told in a prompt. They run under the same policy; a fixed `args` prefix cannot be replaced by the model. The dispatcher never sees a `!` message. It is screened out before the trigger is built, so there is no turn, no session and nothing stored — see [`../concepts/telegram.md`](/concepts/telegram#running-a-command-yourself) and [`../guides/config-reference.md`](/guides/config-reference#os). ### `[run].protocol` — assistant stdout The optional `protocol` field declares the stdout shape used by an assistant dispatcher. The only supported value is `"assistant-v1"`: ```toml [run] command = "bash" args = ["./agent.sh"] protocol = "assistant-v1" ``` An `assistant-v1` agent emits one JSON object per stdout line with `type` set to `delta`, `reply`, or `session`. The gateway forwards every raw stdout line to a local client as `reply.delta`, and uses the last `reply` frame to shape the final `reply` event. `session` is agent bookkeeping only; dotagent does not own or persist the assistant transcript. The value is validated when the manifest loads. Unknown protocol values fail manifest validation instead of producing a run that the client cannot decode. `assistant-v1` is the one-shot assistant stdout protocol, not the persistent agent request/response protocol and not a version of the local Unix-socket API. See [Local Client API](/reference/local-api). ### `[[notifiers]]` — built-in drivers The `notifiers` array runs **in-process inside the daemon**, no subprocess. Supported `driver` values: `desktop`, `slack`, `ntfy`, `pushover`, `telegram`, `imessage` (macOS only — wraps `osascript`), and `plugin` (escape hatch to the legacy plugin protocol). See [`docs/concepts/notifications.md`](/concepts/notifications) for per-driver config schemas. Credential-bearing notifier fields accept `${VAR}`, resolved at send time against the daemon-loaded secrets file (see [`docs/concepts/secrets.md`](/concepts/secrets)), with `std::env` as fallback: `slack.webhook_url`, `ntfy.token` / `base_url` / `topic`, `pushover.token` / `user`, and `telegram.bot_token`. An unresolved reference **fails the send** — there is no fall-through to the literal `${…}` string. `telegram.chat_id` and `imessage.to` are not expanded. They are addresses, not credentials; write them literally. ### `[[on_failure]]` / `[[on_success]]` — legacy plugin hooks These two arrays still drive the plugin protocol (`dotagent-plugin-` binaries). They are reserved for **sink-style** hooks (persist output, publish to Roam, etc.). For notifications, prefer `[[notifiers]]` — it's faster, has fewer moving parts, and ships with the daemon. | Field | Type | Default | Notes | | ----------------- | -------------- | ------- | --------------------------------------------------------------------------------------------------------------- | | `plugin` | `string` | — | Short name; resolved to `dotagent-plugin-` via the plugin client. | | `config` | `object` | `{}` | Opaque JSON forwarded to the plugin's `invoke` verb. | | `events` | `string[]` | `[]` | Filter (empty = all). For `on_failure`: `attempt_failed`, `given_up`, `stale`, `recovered`. | | `timeout_seconds` | `integer` (>0) | `300` | Per-hook deadline. Supervisor kills the process group on overrun. Same field on `[[preflight]]` (default `30`). | ### Schedule types | Type | When a window is due | | ------------ | --------------------------------------------------------------------- | | `cron` | weekday matches AND `(hour:minute)` matches | | `interval` | every `interval_minutes`, anchored on the last success | | `expression` | free-form cron string (**not yet implemented** — parses, never fires) | A **single** daemon (`run.avelino.dotagent`) owns every schedule. There is no per-agent plist and no per-agent systemd timer: launchd / systemd start the daemon, and the daemon computes the next event across all schedules and sleeps until then. See [`daemon-lifecycle.md`](/guides/daemon-lifecycle). #### `on_battery` | Value | Effect | | ------- | ------------------------------------------------------ | | `run` | Dispatch regardless of power source. The default. | | `defer` | Hold the run until the machine is back on mains power. | Absent, the schedule inherits `[power] on_battery` from `config.toml` (itself `run` by default, so an untouched install is unaffected). Declared here, it wins — the cost of a run is a property of the schedule, so an agent can keep a cheap hourly check running unplugged while its expensive 15-minute sync waits for a charger. Deferring does not queue: the window that fires when the charger goes back in is the *current* one, not a backlog. The check runs after the staleness check, so a window that ages past `stale_after_minutes` while unplugged is dropped rather than run late. Full semantics — including the global `min_battery_percent` floor, which is not overridable per schedule — in [`config-reference.md`](/guides/config-reference). #### How `interval` windows advance An interval schedule is an arithmetic sequence of ticks anchored on the last **success** — `last_success`, `+ interval`, `+ 2 · interval`, … — and the window currently due is the greatest tick at or before now. Anchoring on success is deliberate: a run that succeeds re-phases the sequence, so a 90-minute agent that ran at 10:07 is next due at 11:37 rather than on some fixed wall-clock grid. The sequence keeps advancing **while runs fail**, exactly like the calendar keeps producing new cron windows. This matters more than it sounds. A frozen `last_success + interval` window is what deadlocked a failing interval agent: its single window aged past `stale_after_minutes`, so every tick was skipped, so `last_success` never advanced, so the window could never move — one observed agent sat dead for 55 days. A rolling window means the agent keeps being dispatched no matter how long it has been broken. Health is measured against a **different** window: the *first* one missed after the last success, not the rolling one. Judging staleness against a window that rolls forward would report a chronically failing agent as a fresh failure forever. So dispatch stays alive and `dotagent status` still says `stale`. `interval_minutes` must be greater than zero. A value of `0` is rejected during manifest validation instead of reaching the scheduler. ### Heartbeat slug dotagent derives a slug from the schedule's `args` to namespace state files: | `args` | slug | | ------------------------------ | --------------------- | | `[]` | `default` | | `["--period", "dia-anterior"]` | `period_dia-anterior` | | `["--mode", "unsubscribe"]` | `mode_unsubscribe` | Rules: strip leading dashes, lowercase, replace non-alphanumeric with `_`, collapse repeated `_`, trim trailing `_`. Empty input → `default`. Triggered runs use a source namespace instead of schedule args. Without a session, the slug keeps the existing `trigger-` form: | Source | No session | With `session_id = "chat-9_a"` | | ---------- | ------------------ | ------------------------------ | | `telegram` | `trigger-telegram` | `trigger-telegram-chat-9_a` | | `local` | `trigger-local` | `trigger-local-chat-9_a` | | `mcp` | `trigger-mcp` | `trigger-mcp-chat-9_a` | | `cli` | `trigger-cli` | `trigger-cli-chat-9_a` | The session part is sanitized before it reaches a filename. The local API uses the effective session `default` when `message.send` omits `session_id`, so a local API request without an explicit session is associated with `trigger-local-default`. ## Environment variables dotagent injects When dotagent invokes an agent, it sets these variables (in addition to the inherited parent environment, unless `env.inherit = false`): | Variable | Value | | ---------------------- | --------------------------------------------------- | | `AGENT_NAME` | manifest `agent.name` | | `AGENT_HOME` | absolute path to the manifest directory | | `AGENT_TMPDIR` | freshly created tempdir, auto-cleaned after the run | | `AGENT_DRY_RUN` | `"true"` or `"false"` | | `AGENT_SCHEDULE_ID` | which schedule is firing (`daily`, `every-90`, ...) | | `AGENT_SLUG` | derived heartbeat slug for this run | | `AGENT_START_EPOCH` | unix epoch seconds of `started_at` | | `AGENT_ARGV` | JSON array of the schedule's `args` | | `AGENT_HEARTBEAT_FILE` | path to the heartbeat file (empty if `dry_run`) | The agent's positional arguments are `args` from `[run]` followed by `args` from the schedule. ### `LANG` Set to a UTF-8 locale (`en_US.UTF-8` on macOS, `C.UTF-8` elsewhere) **only when neither `LANG` nor `LC_ALL` reaches the agent** — an inherited locale is never overridden, and `[env.extra]` wins over both. This is a default, not a policy. launchd and systemd start a daemon with no locale at all, and a process in the resulting `C` locale has `MB_CUR_MAX == 1`: it reads every **byte** of an environment variable as one character (Latin-1) and writes it back out as UTF-8. A `AGENT_TRIGGER_PAYLOAD` carrying `é` reaches such an agent as `é`, still valid UTF-8, so nothing errors and the agent simply acts on mangled text. Verified with fish 3.7.1, which round-trips `c3 a9` as `c3 83 c2 a9` under `C` and unchanged under `en_US.UTF-8`. Name your own if you need a different one: ```toml [env.extra] LANG = "pt_BR.UTF-8" ``` Runs started by a [trigger](/concepts/triggers) rather than a schedule get additional context, applied *before* the block above so a payload can never redefine `AGENT_NAME` or `AGENT_HEARTBEAT_FILE`: | Variable | Value | | ------------------------ | ------------------------------------------------ | | `AGENT_TRIGGER_SOURCE` | `telegram`, `local`, `mcp`, `cli` | | `AGENT_TRIGGER_ACTOR` | who asked, as the source can attest it | | `AGENT_TRIGGER_REPLY_TO` | opaque handle for the conversation to answer | | `AGENT_TRIGGER_PAYLOAD` | source-specific JSON body | | `AGENT_SESSION_ID` | opaque conversation id, when the trigger has one | `AGENT_SESSION_ID` is not a transcript or a Claude session id. It is an opaque key supplied by the trigger source. The local client API validates it against `^[A-Za-z0-9_-]{1,64}$`. Whether conversation state persists under that key is decided by the `[assistant]` harness (below) — without it, the agent remains responsible. See [Local Client API](/reference/local-api). Agents whose manifest declares `[assistant]` additionally receive, on triggered runs only: | Variable | Value | | --------------------------------- | ----------------------------------------------------------------------------------------------- | | `AGENT_ASSISTANT_SESSION` | recorded model session pointer, when one is valid for the chat | | `AGENT_ASSISTANT_MCP_CONFIG` | path to the assembled, content-addressed `mcp.json` | | `AGENT_ASSISTANT_TOOLKIT_HASH` | stable identity of the toolkit (changes ⇒ fresh session) | | `AGENT_ASSISTANT_MEMORY` | bounded block of recalled facts plus the topics already in use, when `[assistant].memory` is on | | `AGENT_ASSISTANT_CONTEXT_RETIRED` | `"true"` only on the first trigger after automatic transcript retirement | Each is omitted when it has nothing to say — no pointer recorded, no toolkit declared, no facts stored. A typical dispatcher resumes the model session with `claude --resume "$AGENT_ASSISTANT_SESSION"` and attaches the toolkit with `--mcp-config "$AGENT_ASSISTANT_MCP_CONFIG"`. `AGENT_ASSISTANT_CONTEXT_RETIRED` is a one-shot marker. The daemon persists its consumption before starting that process, so later triggers do not receive it. `/novo` clears the pointer without setting the marker; toolkit invalidation also starts fresh without it. ## Heartbeat & state dotagent writes a heartbeat file before and after every (non-dry-run) execution: ``` ~/.config/dotagent/state/agents/{name}/{slug}.heartbeat.json ``` Shape: ```jsonc { "name": "my-agent", "slug": "default", "args": [], "started_at": 1700000000, "started_at_iso": "2023-11-14T22:13:20+0000", "finished_at": 1700000100, "finished_at_iso": "2023-11-14T22:15:00+0000", "exit_code": 0, "duration_seconds": 100, "last_success_at": 1700000100, // preserved across runs; never overwritten on failure "last_success_at_iso": "2023-11-14T22:15:00+0000" } ``` This shape is intentionally compatible with the legacy `~/.local/state/agents/{name}/{slug}.heartbeat.json` written by the Fish-based `agent_init` so existing tools that read it keep working during the migration. Window state (one file per `(agent, schedule, expected_at)`): ``` ~/.config/dotagent/state/windows/{name}-{slug}-{YYYY-MM-DD-HHMM}.json ``` ## Health states For each `(agent, schedule)` dotagent computes one of: | State | Meaning | | ---------- | ---------------------------------------------------------------------------- | | `ok` | `last_success_at >= expected_at` and no retries needed in the current window | | `degraded` | the window succeeded, but only after at least one **failed** attempt | | `failing` | window passed without success; retrying or already given up | | `stale` | never ran, OR window older than `stale_after_minutes` | `attempts` in the window file counts **dispatches**, not retries — the daemon bumps it on every dispatch including the one that succeeded. So a window that worked on the first try lands on disk as `attempts: 1` with `last_attempt_exit_code: 0`, and that is `ok`, not `degraded`. Only attempts that actually failed make a recovered window `degraded`. For `stale`, an interval schedule is judged against the first window it missed after its last success — not the rolling window dispatch uses. See [How `interval` windows advance](#how-interval-windows-advance). ## `[security]` — schema-only in v0 Declares the agent's intended blast radius. dotagent **parses** these fields and `doctor` warns when an agent has no `[security]` block, but the runner does **NOT** yet enforce them. Sandbox integration (`sandbox-exec` / `bwrap` / `firejail`) lands as a follow-up — see [`docs/security/threat-model.md`](/security/threat-model). Declaring intent today still has value: it forces the agent author to think through the surface area, surfaces it for review, and gives `doctor` something to audit. | Field | Type | Default | Meaning | | --------------------- | --------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `allowed_commands` | `string[]` | `[]` (no whitelist) | Commands the agent is allowed to spawn. Empty = no enforcement. | | `allowed_plugins` | `string[]` | `[]` | Plugin names this agent may invoke. Empty = `[[preflight]]` / `[[on_*]]` plugins implicitly allowed. | | `network` | `"allow"` / `"deny"` / `string[]` | `"allow"` | Network policy. `"allow"` / `"deny"` are modes; an array of strings is a hostname allow-list. | | `filesystem_writable` | `string[]` | `[]` | Directories the agent may write to. Empty = unrestricted. `$AGENT_TMPDIR` and `$AGENT_HEARTBEAT_FILE` are **always** writable. | | `env_passthrough` | `string[]` | `[]` | Env vars to pass through. Empty = full inheritance (matches `EnvConfig::inherit = true`). | ### Examples **Minimal — document intent only**: ```toml [security] network = "allow" # explicit ``` This silences the `⚠ no [security] section` warning from `doctor` without changing behavior. **Tighter — hostname allow-list**: ```toml [security] allowed_commands = ["python3"] allowed_plugins = ["sink-roam", "preflight-warp"] network = ["api.github.com", "acme.sentry.io"] filesystem_writable = ["/Users/me/reports"] ``` When sandbox enforcement lands, this manifest forbids the agent from spawning anything other than `python3`, talking to any host outside the allow-list, writing outside `/Users/me/reports`, or invoking any plugin other than `sink-roam` / `preflight-warp`. **Network deny**: ```toml [security] network = "deny" ``` The agent will be denied any outbound network (when enforcement lands). Useful for pure local processors. ## Plugin events When dotagent fires `on_failure` / `on_success`, the `event` field is one of: * `attempt_failed` — a retry attempt failed; will retry again * `given_up` — retries exhausted; repeated on a rising ladder while it holds * `stale` — the schedule stopped running at all (window aged past `stale_after_minutes`, so nothing is even attempted). Delivered on the `given_up` channel when no entry lists `stale` explicitly * `recovered` — a previously-failing window succeeded * `success` — a normal successful run * `timed_out` — agent killed for exceeding `agent.timeout_seconds` * `preflight` — a preflight plugin returned `ok = false` and aborted the run * `daily_summary` — the daemon's daily health roll-up, delivered at `[daily_summary].time` (default 22:45 local; also surfaced via `dotagent daily-summary`) A plugin can filter via its manifest entry's `events` array — with one exception. `daily_summary` belongs to the daemon rather than to any agent, so it only ever reaches `[[daily_summary.notifiers]]` in [`config.toml`](/guides/config-reference#daily_summary); listing it in a manifest matches nothing. ## Manifest hash & drift detection dotagent caches `sha256(agent.toml)` for every loaded manifest at `state/known_manifests.json`. On the next load: * New `agent.name` not in the cache → `phantom_agent_detected` audit event + out-of-band notify (`Critical` severity). The agent still runs by default — see [V2 in the threat model](/security/threat-model#v2--phantom-agent). * Existing name but mismatched sha → `manifest_drift_detected` audit event + out-of-band notify (`Critical` severity). The agent uses the current on-disk manifest. Every `agent_run` audit entry records the manifest sha256 used, so forensic reconstruction can correlate runs with a specific manifest revision. ## What dotagent does NOT do * **Run the agent's business logic.** dotagent is a scheduler, supervisor, and trigger harness; the agent is an independent process. * **Own conversation state.** Trigger `session_id` values are opaque and are passed through as `AGENT_SESSION_ID`. Transcripts, LLM state, and persistence belong to the agent. * **Provide a network API.** The current local client API is a user-local Unix socket, not HTTP or TCP. It is enabled only when the configured dispatcher is discovered; see [Local Client API](/reference/local-api). * **Provide an SDK.** No client library is required. Read env vars, write stdout/stderr, exit with a code. That's it. * **Wait for the schedule.** The OS scheduler (launchd / systemd) fires the trigger; dotagent only computes "did the expected window succeed?" during `tick`. # Plugin protocol > **Note.** The common notifiers (`desktop`, `imessage`, `slack`, `ntfy`, `pushover`, `telegram`) are now **built into the daemon** — they do not use this protocol. See [`docs/concepts/notifications.md`](/concepts/notifications) for the `[[notifiers]]` shape. This protocol is the contract for **preflight checks**, **output sinks**, and **third-party notifiers** (Discord, Teams, custom relays) wired via `driver = "plugin"`. Plugins extend dotagent without recompiling it. They handle preflight checks, output sinks, and third-party notifiers. A plugin is **any binary** named `dotagent-plugin-` that speaks the protocol below — write it in Rust, Go, Python, Bash, whatever. ## Discovery dotagent searches for the binary in this order, returning the first match: 1. `$DOTAGENT_PLUGIN_PATH` (colon-separated list of directories) 2. `~/.config/dotagent/plugins/` 3. `/usr/local/lib/dotagent/plugins/` 4. `$PATH` ## CLI surface A plugin must accept exactly one positional argument: the **verb**. ``` dotagent-plugin- info dotagent-plugin- validate dotagent-plugin- invoke ``` For `validate` and `invoke`, the JSON payload arrives on **stdin**. The plugin must write its JSON response to **stdout** and exit with `0` for success, non-zero for failure. Human-readable logs go to **stderr**. ## Verbs ### `info` No stdin. Print self-describing metadata. ```jsonc { "name": "sink-roam", "version": "0.1.0", "kinds": ["sink"], // "notify" | "preflight" | "sink" "platforms": ["darwin", "linux"], // optional; empty means cross-platform "schema": { // JSON Schema describing accepted `config` "type": "object", "required": ["page"], "properties": { "page": { "type": "string" }, "marker_regex": { "type": "string" } } } } ``` ### `validate` Stdin: the same JSON object the manifest puts under `config`. ```jsonc // stdin { "page": "today", "marker_regex": "#daily" } ``` Stdout: `{"ok": true}` or `{"ok": false, "error": "..."}`. dotagent calls this when loading manifests so misconfigurations surface in `dotagent doctor` rather than at firing time. ### `invoke` Stdin: an `InvokePayload` (see schema below). ```jsonc { "kind": "sink", // matches plugin kind "agent": "finops-weekly", "schedule": "weekly", "event": "success", // "attempt_failed" | "given_up" | "stale" | "recovered" | "success" | "preflight" "message": "captured stdout from the agent...", "config": { "page": "today", "marker_regex": "#finops" } } ``` Stdout: minimum `{"ok": bool}`. Extra fields are forwarded into dotagent's logs and `--verbose` output; they do not change orchestrator behavior. ## Kind-specific contracts ### Notify * Receives `message` (string) and `event`. * Returns `ok=true` after delivery, `ok=false` otherwise. Best-effort — dotagent does not retry notifications. ### Preflight * Returns `ok=true` if the precondition holds, `ok=false` to abort the run. * May include `suggest` in the response (string) — dotagent forwards it into any failure notification so the user knows what to fix. * The agent will NOT be invoked if any preflight returns `ok=false`. dotagent emits an `attempt_failed` event with the suggestion attached. ### Sink * Persists the agent's output somewhere (file, Roam, Notion, etc.). * `message` carries the captured stdout. * Sinks run after a successful run; failure of a sink raises `attempt_failed` but does not roll back the agent (the run already succeeded). ## Convention: keep plugins single-purpose Cross-cutting concerns (auth refresh, retries with backoff for transient HTTP errors, etc.) belong in the plugin, not in dotagent. dotagent treats the plugin as a black box: "given this payload, did you succeed?". ## Testing a plugin ```bash echo '{"page":"today","marker_regex":"#daily"}' | dotagent-plugin-sink-roam validate echo '{"kind":"sink","agent":"x","schedule":"y","event":"success","message":"hi","config":{"page":"today","marker_regex":"#daily"}}' \ | dotagent-plugin-sink-roam invoke dotagent-plugin-sink-roam info | jq . ``` If your plugin passes the three commands above, it integrates with dotagent. # Persistent protocol > The contract for agents declaring `[lifecycle] mode = "persistent"`. A one-shot agent needs no protocol. stdout is the answer, the exit code is the verdict, and the process ending is the boundary between one answer and the next. A process that does not end has to say where an answer stops — and that is all this is. **One JSON object per line, both directions.** stdout is the channel; stderr is the log. Same split the [plugin protocol](/reference/plugin-protocol) already uses. For the *why*, see [Lifecycle](/concepts/lifecycle). A complete implementation in \~20 lines of bash is in [`examples/hello-persistent`](https://github.com/avelino/dotagent/tree/main/examples/hello-persistent/README.md). ## The exchange ```mermaid sequenceDiagram participant D as daemon participant A as your agent D->>A: spawn (stdin/stdout/stderr piped) D->>A: {"kind":"hello", …} A-->>D: {"kind":"ready","ok":true} Note over D,A: instance is live D->>A: {"kind":"request","id":"1", …} A-->>D: {"kind":"response","id":"1","output":"…"} D->>A: {"kind":"request","id":"2", …} A-->>D: {"kind":"response","id":"2","output":"…"} Note over D,A: idle past the window, or quota reached D-->>A: SIGTERM (process group) ``` ## `hello` → `ready` Written once, immediately after spawn. ```jsonc // stdin { "v": 1, "kind": "hello", "agent": "telegram-assistant", "key": "12345", "schedule": "trigger" } // stdout { "v": 1, "kind": "ready", "ok": true } ``` Answer `ready` only once whatever you needed to warm up is warm — that is what the handshake is for. dotagent sends no request until it arrives, and gives up after `[lifecycle] startup_timeout_seconds` (default 30). Refusing is legal and better than dying silently: ```jsonc { "v": 1, "kind": "ready", "ok": false, "error": "no API key in the environment" } ``` Either failure kills the instance and surfaces as a spawn error, rather than letting every message time out against a process that was never going to answer. ## `request` → `response` ```jsonc // stdin { "v": 1, "kind": "request", "id": "1", "agent": "telegram-assistant", "schedule": "trigger", "args": [], "deadline_seconds": 600, "trigger": { "source": "telegram", "session_id": "chat-9_a", "actor": "123456789", "reply_to": "123456789", "payload": { "text": "how are the disks?", "chat_id": 12345 } } } // stdout { "v": 1, "kind": "response", "id": "1", "ok": true, "output": "All under 70%." } ``` | Field | Meaning | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | Correlation handle. Echo it back. | | `deadline_seconds` | How long dotagent will wait. Bail first and say something useful — being killed mid-sentence says nothing. | | `trigger` | Present when a message or a tool call caused the run, absent when a clock did. When present, it has `source` and optional `session_id`, `actor`, `reply_to`, and `payload` fields. | | `output` | Goes back to whoever asked. | | `ok` | Optional, defaults to `true`. | | `exit_code` | Optional. Defaults to `0` when `ok`, `1` otherwise. Recorded in the heartbeat and the audit log exactly like a one-shot exit code. | | `error` | Used as the output when `output` is absent — a failure with something to say beats silence in the chat. | The minimum viable answer is `{"output":"hi"}`. ## Rules the reader follows **A line that does not parse is dropped.** A stray `echo` on stdout costs a debug line, not the conversation. Do not rely on it — put logs on stderr — but one mistake will not corrupt the stream. **A frame whose `id` is not the one in flight is dropped.** This is what stops a late answer from being handed to the next question. Without it, one slow reply would shift every subsequent answer by one and the bot would silently start replying to the previous message. **Omitting `id` is allowed.** There is only ever one request in flight per instance, so an answer with no `id` is taken as the answer to it. Echoing it back is still better: it is what makes the dropping rule work. **A timeout recycles the instance.** dotagent does not wait for a late answer and does not reuse the process — its state is now uncertain, and reciprocating that uncertainty is cheaper than reasoning about it. ## The environment The stable `AGENT_*` block is set at spawn, as always: `AGENT_NAME`, `AGENT_HOME`, `AGENT_TMPDIR`, `AGENT_SLUG`, `AGENT_SCHEDULE_ID`, `AGENT_HEARTBEAT_FILE`, `AGENT_ARGV`, plus `[env.extra]`. Two are specific to this mode: | Variable | Meaning | | ------------------- | ------------------------------------------------------------------------------------- | | `AGENT_LIFECYCLE` | `persistent`. Absent for a one-shot run, so one script can support both. | | `AGENT_PERSIST_KEY` | Which slice this instance answers for — the resolved `[lifecycle] key`, or `default`. | **`AGENT_TRIGGER_*` and `AGENT_SESSION_ID` are not defined in the persistent process environment.** Those values describe one message, and this process will see many; fixed at spawn, they would read as perfectly valid while being permanently stale. Trigger context, including the optional session id, arrives in the `trigger` field of each request instead. `AGENT_TMPDIR` belongs to the instance, not to a request — it survives between requests and is removed when the instance is recycled. ## Shutting down Recycling sends `SIGTERM` to the instance's process group, then `SIGKILL` after the grace window. stdin also closes. Either is a valid signal to flush and exit; a `while read` loop gets EOF and falls out of the loop on its own. Nothing is asked of a well-behaved agent here. If you hold something that must be persisted, persist it as you go — a recycle can happen between any two requests, and the reasons are listed in [Lifecycle](/concepts/lifecycle#when-an-instance-goes-away). ## Testing one by hand The protocol is plain text, so a pipe is enough: ```bash printf '%s\n%s\n' \ '{"v":1,"kind":"hello","agent":"x","key":"default"}' \ '{"v":1,"kind":"request","id":"1","deadline_seconds":30,"trigger":{"source":"cli","session_id":"manual-1","payload":{"text":"hi"}}}' \ | bash ./agent.sh ``` Two lines out, `ready` then a `response` carrying `id: "1"`, and your logs on stderr — that agent integrates. Through dotagent, with the real pool: ```bash DOTAGENT_ROOT=$PWD/examples dotagent run hello-persistent --schedule manual ``` ## See also * [Lifecycle](/concepts/lifecycle) — when to use this at all * [Agent spec](/reference/agent-spec#lifecycle--how-long-one-process-lives) — the `[lifecycle]` fields * [Plugin protocol](/reference/plugin-protocol) — the one-shot JSON-stdio contract * [Env vars](/reference/env-vars) — everything dotagent injects # MCP server > `dotagent mcp` — every agent on this machine, as a callable tool. ```bash dotagent mcp ``` Speaks JSON-RPC 2.0 over stdio, one object per line. Point any MCP client at it: ```json { "mcpServers": { "dotagent": { "command": "dotagent", "args": ["mcp"] } } } ``` The catalog is built from the manifests already on disk. Claude Code, Claude Desktop, an aggregating proxy, or a headless `claude -p --mcp-config` all see the same tools. ## Why this shape A model asked to run something has two options: compose a command, or pick from a list. Composing means somebody has to validate the result, and that check is exactly the one that gets forgotten. As tools, the list is the protocol. `tools/list` is the catalog, a name that is not in it is not callable, and "only run what the operator declared" stops being a rule anyone has to remember. ## Methods | Method | Behavior | | ------------ | ----------------------------------------------------------------------------------------- | | `initialize` | handshake; echoes the client's `protocolVersion` when it sent one, otherwise `2024-11-05` | | `tools/list` | one tool per discovered agent | | `tools/call` | run an agent, return its output | | `ping` | liveness, empty result | Requests without an `id` are notifications and get no response. Unparseable input answers with `-32700` and a null id. ## Introspection | Tool | Arguments | Behavior | | -------------------- | ---------------- | --------------------------------------------------------------------------- | | `dotagent-status` | — | Health of every scheduled agent, same as `dotagent status`. | | `dotagent-logs` | `agent`, `lines` | Tail an agent's captured output. Default 40 lines, capped at 400. | | `dotagent-inspect` | `agent` | Heartbeat, window state, manifest hash. | | `dotagent-doctor` | — | Manifest validation, plugin resolution, secrets / Telegram / memory status. | | `dotagent-next-runs` | — | What the scheduler would dispatch now. A preview; nothing runs. | Always present, regardless of `[memory]`. **Read-only by selection, not by accident.** The CLI has commands that write — `install`, `uninstall`, `reload`, `daily-summary` — and none are exposed. `bootstrap` is refused most deliberately: it marks every window as ok, which silences a failure rather than fixing it, and an assistant that "resolves" an alert by deleting the signal is worse than one that cannot help. `run` and `run-now` are absent because they already exist as the `run-*` tools. Agent names are resolved through discovery rather than concatenated into a path, so `../` in a name is refused rather than followed. These re-execute the `dotagent` binary instead of reimplementing each command. Rendering lives in the CLI, and a second copy would drift — an assistant quoting a status that no longer matches `dotagent status` is worse than a subprocess. ## Memory tools Five more tools sit beside the agent catalog when `[memory]` is enabled (the default). They are served in-process — nothing is spawned: | Tool | Arguments | Behavior | | ------------------ | --------------------------- | -------------------------------------------------------------------------------------------------------------------- | | `memory-remember` | `text`, `topics[]` | Store a durable fact, linked to its topics. Restating a fact already stored reinforces it instead of duplicating it. | | `memory-recall` | `query` or `topic`, `limit` | Rank facts by shared words then recency, or return everything linked to a topic. Each result leads with its id. | | `memory-topics` | — | List the subjects memory knows about. | | `memory-supersede` | `id`, `text`, `topics[]` | Replace a fact that stopped being true. The old one stays readable, stops being recalled. | | `memory-forget` | `id` | Delete a fact outright. | The ids in `memory-recall` output are what `memory-supersede` and `memory-forget` take — a model that finds a wrong fact can fix it in the next call instead of asking which one. Facts live in an [outl](https://github.com/avelino/outl) workspace you can open and edit. See [Memory](/concepts/memory) for what belongs in it and why ranking is lexical rather than semantic. The same verbs are available from a shell as [`dotagent memory`](/reference/cli#memory) — no daemon required, which is what lets a consolidation pass be an ordinary scheduled agent. Turn them off with `[memory] enabled = false` in `config.toml`. ## Remediation tools One tool per `[[preflight]] remediation` declared in a manifest, named `remediate--`: | Tool | Arguments | Behavior | | ---------------------------- | --------- | -------------------------------------------- | | `remediate--` | — | Run the declared command, return its output. | The point is an alert you can answer. "preflight aborted by plugin preflight-warp: warp-cli connect" names the fix, and until now the fix was something you got up and typed. **It takes no arguments**, deliberately: the command is fixed by the manifest, so the model chooses *which* remediation, never *what* runs. And it is declared by the operator rather than taken from the plugin's `suggest` string — running a command a plugin wrote, triggered from a chat, is arbitrary execution over an inbound path. See [threat model V12](/security/threat-model#v12--remediation-from-a-chat-message). Supervised with a 120-second deadline, audited as `remediation_invoked` at `Critical`, and it does **not** re-run the agent: clearing the check and dispatching a run stay two decisions. ## Skill tools One tool per installed skill, plus two verbs that reach inside one: | Tool | Arguments | Behavior | | -------------- | --------------------------- | ----------------------------------------------------------- | | `skill-` | — | Return the procedure, plus an index of the files beside it. | | `skill-read` | `skill`, `path` | Return one supporting file from that skill. | | `skill-run` | `skill`, `script`, `args[]` | Execute one `scripts/` entry, return its output. | Loading a skill returns **text**. The tool description says so explicitly, because these are usually written in the imperative ("Cut a new release") and a model could otherwise read the call as performing the procedure. `skill-read` exists because a skill is often more than one file. A procedure that says "see `references/glossary.md`" assumes a filesystem tool the caller may not have; without a way to fetch it, the model follows half a procedure and never notices. The index returned alongside the body is what makes the request possible. Skills come from `~/.config/dotagent/skills/` **and** `~/.claude/skills/` — the format is Anthropic's Agent Skills layout, so one written for Claude Code is already installed. See [Skills](/concepts/skills), including what does not port. A broken `SKILL.md` does not fail `tools/list` the way a broken manifest does. The consequences differ: an unparseable manifest means an agent silently never runs, while an unparseable skill means one procedure is missing. It is logged and reported by `doctor`; the catalog still serves. Turn them off with `[skills] enabled = false`. ## Command tools Two more when `[commands]` is enabled (the default) and at least one command is installed: | Tool | Arguments | Behavior | | -------------- | -------------- | ---------------------------------------------------------------------------- | | `command-get` | `name`, `args` | Resolve an invoked command into the prompt to follow, arguments substituted. | | `command-list` | — | Every installed command, with what it does and takes. | **Two tools, not one per command** — deliberately unlike `skill-*`. There, the catalog is a menu the model picks from. A command was already picked by a human before the model saw anything, so publishing N command tools would re-open that decision and let a model call the wrong one. Pass `command.name` and `command.args` from the trigger payload through unchanged. Note `args` is a **string** here and an array for `run-*`: an agent's arguments are argv, a command's are whatever the sender typed after the name. See [Commands](/concepts/commands). ## OS tools Present only when `[os]` is enabled **and** carries a non-empty `allow` list. With the section off, these two names are absent from `tools/list` — a client sees no such capability rather than one that refuses. | Tool | Arguments | Does | | --------- | ------------- | --------------------------------------------------------------------------------------- | | `os-list` | none | The allowed binaries, and for each whether the whole binary or only certain subcommands | | `os-run` | `bin`, `args` | Runs one allowed binary, returns its output | ### Named binaries Each `[[os.tool]]` in `config.toml` publishes one more entry, `os-` plus any fixed arguments (`os-outl`, `os-kubectl-get`), carrying the operator's description. They exist for discovery: `os-run` can already reach the same binary, but nothing in the catalog says what it is for. Their schema exposes only `args`, appended after the fixed ones. A model cannot replace what the operator pinned, which is what makes `kubectl get` a publishable read-only view of a binary that can also delete. `os-run` is the only tool here that lets the model supply arguments. The operator declares which binaries may run; the model composes the invocation within that. The allowlist is enforced before the process is spawned, so the schema is a hint to the model and never the control. Refusals name the machine's config rather than the assistant's judgment, so an assistant relaying one does not sound like it decided: ``` `sh` is not allowed here. Call os-list for what is, and tell whoever asked that it is the machine's config that says no — not you. ``` Output is capped at 8 KB. Invocations run under the supervisor with the configured deadline and are audited at `Critical`. See [`guides/config-reference.md`](/guides/config-reference#os) for the allowlist syntax and [`security/threat-model.md`](/security/threat-model) V17 for what it does and does not bound. ## Tool naming MCP restricts tool names to `[a-zA-Z0-9_-]`, which agent names are not bound by. Each agent becomes `run-`: | Agent | Tool | | ----------------- | --------------------- | | `disk-alert` | `run-disk-alert` | | `hn.digest` | `run-hn-digest` | | `ops/cost-report` | `run-ops-cost-report` | Skills use the same sanitizer under a different prefix, `skill-`, which is what keeps a skill from ever colliding with an agent. Two *skills* can still collide, and the same rule applies: first discovered wins, the loser is dropped with a warning, and `doctor` names both. The mapping is lossy — `a.b` and `a/b` both produce `run-a-b`. On a collision the first agent discovered wins and the later one is dropped from the catalog with a warning in the log. Shadowing silently would mean `tools/call` running an agent nobody chose. The reverse direction is never reconstructed: a tool name is resolved by rebuilding the catalog and matching, not by transforming the string back. ## Tool description `agent.description` from the manifest. Without one, the description falls back to the agent name plus its schedule ids, which still helps a model choose. Writing a real `description` is the single highest-leverage thing you can do for dispatch quality. ## Arguments ```json { "schedule": "daily", "args": ["--verbose"] } ``` Both optional. `schedule` picks which schedule's args to borrow, defaulting to the agent's first. `args` is appended after the manifest's own `run.args`. An agent with no schedules gets the synthetic id `trigger`. ## Results Success returns the agent's stdout as text content: ```json { "content": [{ "type": "text", "text": "..." }], "isError": false } ``` An agent that fails is **not** a JSON-RPC error. It comes back as a result with `isError: true` and the exit code plus stderr in the text, so the model can read the failure and react: ```json { "content": [{ "type": "text", "text": "disk-alert exited 1\n..." }], "isError": true } ``` JSON-RPC errors are reserved for protocol faults — unknown method, unknown tool, malformed params. The distinction matters: a model told "that tool does not exist" behaves differently from one told "the tool ran and failed". ## Process model Agents run **in the `dotagent mcp` process**, the same way `run-now` does. Not through the daemon. Consequences, all shared with `run-now`: * The subprocess tree does not appear in `dotagent status`, which reflects only the daemon's supervisor. * Nothing reaps it when the daemon stops. State is keyed off the `trigger-mcp` slug rather than the schedule's args, so an MCP-initiated run can never overwrite `last_success_at` for a cron window that never fired. See [Triggers](/concepts/triggers#what-a-triggered-run-keeps). That isolates MCP runs from the **daemon**, not from **each other**. The slug is a constant, so two MCP clients running the same agent at the same moment write the same heartbeat file. Writes are serialized — the state store holds an exclusive lock — so the file never tears, but the later finisher wins and the earlier run's `finished_at` is lost. Scheduling is unaffected (the scheduled slug is a different file); the visible effect is `dotagent inspect` showing one of the two runs. A single `dotagent mcp` process handles requests one at a time, so this needs two clients to happen at all. ## Streams stdout carries protocol only. All logging goes to stderr, so `RUST_LOG=debug dotagent mcp` stays safe to run under a client. ## Security Enabling an MCP client to reach dotagent means that client can run any installed agent. That is the point, and it is also the whole blast radius — worth stating plainly: * Over stdio the server inherits the trust of whatever spawned it. A local MCP client already runs as you. * The `dotagent mcp` subcommand itself uses stdio only. It does not open a port or provide a TCP/HTTP endpoint, and nothing from this MCP transport is reachable from another machine. * The catalog is the boundary. An agent that is not installed cannot be run, and arguments never reach a shell. This statement is scoped to the MCP subcommand. A daemon may separately expose the user-local Unix-socket [Local Client API](/reference/local-api) at `$DOTAGENT_HOME/api.sock`; that API is not an MCP transport and has its own limits and threat model. See the [threat model](/security/threat-model). ## Checking it ```bash printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | dotagent mcp ``` One tool per installed agent. To exercise a call: ```bash printf '%s\n' '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"run-hello-fish","arguments":{}}}' \ | dotagent mcp ``` A malformed manifest anywhere in the search path fails discovery, and `tools/list` answers with an error naming the parse failure rather than an empty catalog. A model told it has no tools would answer confidently and wrongly; an error is something you can see and fix. ## See also * [Triggers](/concepts/triggers) — the general concept * [Local Client API](/reference/local-api) — the daemon's Unix-socket trigger transport * [Telegram](/concepts/telegram) — a chat front end built on this * [CLI](/reference/cli#mcp) # Local client API This document specifies the current local client API served by the dotagent daemon. It is a transport for triggers and run output, not a conversation runtime. The daemon remains a harness: * It discovers the configured dispatcher, admits and orders triggers, supervises agent runs, and delivers output. * It does not own a chat session, transcript, model context, or LLM state. * `session_id` is opaque. The daemon passes it to the agent as `AGENT_SESSION_ID`; the agent decides whether and how to persist a conversation. ## Transport The API is a Unix domain socket using JSON Lines. It does not expose HTTP or a TCP listener. The socket path is: ``` $DOTAGENT_HOME/api.sock ``` The default is `~/.config/dotagent/api.sock`. The daemon starts the listener only when `telegram.dispatcher_agent` resolves to a discovered manifest. This condition is independent of Telegram polling: Telegram still requires its own bot token and non-empty allowlist, while a local client only needs the Unix socket. ### Raw CLI bridge `dotagent api` is the smallest built-in client for this socket. It connects to `$DOTAGENT_HOME/api.sock` by default, or to the path supplied with `--socket `: ```bash printf '%s\n' '{"id":"status","method":"status.get"}' | dotagent api ``` The command forwards stdin frames to the socket and socket frames to stdout without parsing, reformatting, or rendering them. It preserves the input and output bytes, including line endings, and writes diagnostics only to stderr. It is a raw JSONL bridge for scripts and TUI backends, not a TUI itself. After stdin EOF it half-closes the socket write side and keeps reading until the server closes the connection; it does not create a session or local state. ```mermaid sequenceDiagram participant C as local client participant S as api.sock participant D as daemon participant G as trigger gateway participant A as dispatcher agent C->>S: JSON request line S->>D: parse and enforce limits D->>G: admit trigger G->>A: supervised run + AGENT_* env A-->>G: stdout lines / exit status G-->>S: events and final reply S-->>C: JSON Lines ``` ### Startup, reload, and shutdown * At daemon boot, bind hygiene removes a stale socket left by a previous daemon when it is a socket owned by the current uid. * A live listener is never replaced. A live socket, a non-socket file, or a stale socket owned by another uid makes bind fail instead of unlinking an object the daemon does not own. * `dotagent reload` restarts the Telegram ingress so its allowlist and rate limit take effect. The local listener stays on the same socket and keeps one owner until the daemon restarts. A changed `dispatcher_agent` therefore takes effect for the local API only after restart. * A graceful daemon shutdown stops the listener and removes the socket. A crash can leave the path behind; the next bind applies the stale-socket checks above. ## Framing Both directions are one JSON object per line. A client must terminate each request with `\n`; responses and events are newline-terminated as well. Blank lines are ignored. ### Request envelope ```json { "id": "request-1", "method": "message.send", "params": { "session_id": "chat-9_a", "text": "status?" } } ``` | Field | Type | Meaning | | -------- | ---------------- | ------------------------------------------------------------------------------- | | `id` | string or number | Correlation token. The server normalizes it to a string and echoes that string. | | `method` | string | One of the methods below. | | `params` | object | Method parameters. Omit it when a method has no parameters. | The request id is not a session id and is not persisted. A malformed request gets an error response with the id recovered from the raw object when possible, or `""` otherwise. The connection remains usable after a parse error. ### Response envelope Exactly one of `result` or `error` is present: ```json {"id":"request-1","result":{"accepted":true}} ``` ```json {"id":"request-1","error":{"code":"invalid_request","message":"text must not be empty"}} ``` An accepted `message.send` response means that the trigger entered the gateway; it does not mean that the agent has finished. The reply is delivered later as server events on the same connection. ## Methods ### `message.send` Send one message to the configured dispatcher agent. ```json { "id": 1, "method": "message.send", "params": { "session_id": "chat-9_a", "text": "What changed today?" } } ``` `session_id` is optional. When omitted, the effective session is `default`. The text must be non-empty after trimming and no larger than 32 KiB in UTF-8 bytes. A session id must match `^[A-Za-z0-9_-]{1,64}$`. The successful result is always: ```json {"accepted":true} ``` The request does not select an arbitrary agent. It always targets the discovered `telegram.dispatcher_agent` configured for this daemon. ### `message.send` with a `!` prefix Text beginning with `!` is not forwarded to the dispatcher. It runs the named binary and answers with a single `reply` event — no `run.started`, no `typing`, no `reply.delta`, because there is no run and nothing streams. ```json {"id":"1","method":"message.send","params":{"text":"!rg TODO src"}} ``` A command on the `confirm` list is parked instead of run: the reply quotes it and asks for `!!`, which arrives as another `message.send` on the same `session_id`. The session is what keys it, so a `!!` cannot release what a different session parked. Requires `[os]` to be configured, and obeys the same allowlist as `os-run`. A binary the list does not admit comes back as an ordinary `reply` naming the config, not as a protocol error: nothing about the request was malformed. ### `commands.list` Return the discovered command catalog shaped for Telegram/local dispatch. It does not execute a command. ```json {"id":"commands","method":"commands.list"} ``` The result is an array whose entries have this shape: ```json [ { "name": "standup", "telegram_name": "standup", "description": "Post a standup", "argument_hint": "..." } ] ``` Invalid command files are omitted and logged by the daemon. ### `status.get` Return the local API handler's current daemon/gateway snapshot: ```json {"id":"status","method":"status.get"} ``` ```json { "id": "status", "result": { "daemon": "ok", "gateway": "ok", "dispatcher_agent": "telegram-assistant" } } ``` ## Events Events are server-initiated JSON lines. They do not carry a request id; use `session_id` to correlate them with a `message.send`. Different sessions may run concurrently, so a client must not assume that events for different sessions are globally ordered. Requests for the same gateway conversation are FIFO. | Event | Fields | Meaning | | ------------- | --------------------- | --------------------------------------------------------------------------------------------------- | | `typing` | `session_id` | The dispatcher run is in progress. Long runs may emit this more than once. | | `run.started` | `session_id`, `agent` | The gateway admitted and started a run for the dispatcher agent. | | `reply.delta` | `session_id`, `line` | One raw stdout line, in arrival order. Assistant protocol frames are not decoded or rewritten here. | | `reply` | `session_id`, `text` | The shaped final reply for that trigger. | For an agent using `assistant-v1`, the final `reply` uses the last assistant `reply` frame. For a plain agent it falls back to the captured stdout tail, or an explicit no-output message. The local API receives raw streaming deltas; Telegram deliberately ignores deltas and sends only the final reply because the Telegram Bot API path is final-only. ## Error codes The wire error object contains a stable `code` and a human-readable `message`. Clients should branch on `code`, not on message text. | Code | Meaning | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `invalid_request` | Invalid JSON/request shape, unknown method, invalid params, empty text, an oversized text field, or an oversized request line. | | `rate_limited` | The per-connection local API budget or the gateway's local-trigger budget was exceeded. | | `session_id_invalid` | The effective session id failed the allowed charset/length check. | | `too_many_connections` | The 16-connection server cap was reached. The rejected connection is closed. | | `internal` | The gateway is unavailable, the conversation cap/queue rejected the trigger, or the handler failed. The message may identify the admission reason. | Socket bind failures happen before the wire exists and therefore have no JSON error response. ## Limits and backpressure These are the default limits in the current implementation: | Limit | Default | Scope / behavior | | ----------------------- | -------------------: | ----------------------------------------------------------------------------------------------------------------------------- | | Live socket connections | 16 | An over-cap connection receives `too_many_connections` and closes. | | Requests | 30 per minute | Sliding window per connection. Gateway local-trigger admission also defaults to 30 per minute per attested actor/session key. | | `message.send` text | 32 KiB | Measured in UTF-8 bytes; invalid input is rejected before dispatch. | | Request line | 64 KiB | An overlong JSON line is rejected without unbounded buffering. | | Gateway conversations | 4 | New `(source, session)` conversations over the cap are rejected; existing conversations keep their FIFO queue. | | Per-conversation queue | 64 jobs | A full queue is rejected instead of blocking unrelated conversations. | | Pending event bytes | 1 MiB per connection | A slow client is disconnected when the budget is exceeded. A 512-frame channel is an additional backstop. | | One socket write | 10 seconds | A client that stops reading is disconnected rather than pinning a writer task. | The event queue is intentionally not an infinite buffer. A local client that cannot consume `reply.delta` quickly enough loses that connection and must retry at the application layer. ## Harness boundary and assistant-v1 An assistant dispatcher opts into the one-shot stdout protocol in its manifest: ```toml [run] command = "bash" args = ["./agent.sh"] protocol = "assistant-v1" ``` The agent emits one JSON object per stdout line: ```json {"type":"delta","text":"partial answer"} {"type":"reply","text":"final answer"} {"type":"session","claude_session":"opaque-agent-session","transcript_bytes":1234} ``` The `delta` and `reply` fields are the assistant-v1 contract. `session` is optional bookkeeping supplied by the agent. By default the daemon ignores `session` for conversation ownership and persistence; it only uses `reply` when shaping the final delivery, while forwarding every raw stdout line as `reply.delta` to a local client. When the manifest also declares `[assistant]`, the daemon stops ignoring `session`: the frame's `claude_session` pointer and `transcript_bytes` are recorded in the conversation registry (pointers, never transcript content), and the next trigger for the same session receives the pointer back as `AGENT_ASSISTANT_SESSION`. A `transcript_bytes` past the configured ceiling retires the pointer — the next trigger starts a fresh session and receives `AGENT_ASSISTANT_CONTEXT_RETIRED=true` exactly once. `/novo` clears the pointer without that marker. Trailing `MEMO:` capture lines are stripped from the delivered reply and flushed to the memory workspace. See [agent-spec](/reference/agent-spec#assistant-conversational-harness-opt-in). `assistant-v1` is the only currently supported value for `[run].protocol`. Unknown values are rejected while loading the manifest. The protocol does not require Claude, a particular model, or a particular language. A flag such as `claude --include-partial-messages` is an implementation detail of the example, not part of the dotagent contract. Persistent agents use the separate persistent JSON-lines protocol. This local API document does not merge that process protocol with `assistant-v1`. ## Security model The socket is created with mode `0600`. When the kernel provides peer credentials, the daemon records the peer uid and pid in the trigger actor used for audit attribution. Missing kernel credentials degrade to the actor `local`; they are never invented. The socket permission is the access boundary, not an authentication system. There is no additional token, handshake, or user database. Any process able to open this user-local socket can ask the configured dispatcher to run installed agents and can read the replies it requested. A same-user attacker is already outside dotagent's privilege boundary; see threat model V15. The API does not write message text to the audit log. The daemon audits trigger admission/rejection and agent execution, while the message body and streamed reply remain data delivered to the client and agent. ## Versioning The current local API has no version field, handshake, or capability negotiation. `assistant-v1` names the agent stdout protocol; it is not a version of the Unix-socket API. Clients should correlate by `id`/`session_id`, tolerate unknown object fields, and treat an unknown event or error code as an unsupported/failed operation rather than inferring semantics from the socket path. Any incompatible local wire revision must be introduced explicitly; there is no second transport or versioned endpoint in the current implementation. ## See also * [Agent spec](/reference/agent-spec) - manifest and injected environment contract * [Environment variables](/reference/env-vars) - `AGENT_SESSION_ID` and trigger context * [Triggers](/concepts/triggers) - admission, ordering, and trigger slugs * [Lifecycle](/concepts/lifecycle) - one-shot versus persistent agents * [Threat model](/security/threat-model#v15--local-unix-socket-api) # Environment variables Two categories: 1. [**Injected into the agent**](#injected-into-the-agent-subprocess) — what your script sees when dotagent invokes it. 2. [**Read by dotagent itself**](#read-by-dotagent-itself) — overrides for paths, verbosity, OTel headers. *** ## Injected into the agent subprocess When dotagent spawns your agent, these `AGENT_*` variables are set on top of the inherited environment (unless `env.inherit = false` in the manifest). They are the **only API surface** you depend on — there's no SDK to import. | Variable | Type | Example | When set | | --------------------------------- | -------------------- | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AGENT_NAME` | string | `finops-weekly` | Always. | | `AGENT_HOME` | abs path | `/Users/me/.config/dotagent/agents/finops-weekly` | Always — the manifest directory. | | `AGENT_TMPDIR` | abs path | `/var/folders/.../tmpAbCdEf` | Always — fresh per run, auto-cleaned on exit. | | `AGENT_DRY_RUN` | `"true"` / `"false"` | `"false"` | Always. | | `AGENT_SCHEDULE_ID` | string | `daily` | Always — matches `[[schedules]].id`. | | `AGENT_SLUG` | string | `period_dia-anterior` | Always — derived from the schedule's `args`. | | `AGENT_START_EPOCH` | int | `1700000000` | Always — unix epoch of `started_at`. | | `AGENT_ARGV` | JSON array | `["--period","dia-anterior"]` | Always — the schedule's `args` as JSON. | | `AGENT_HEARTBEAT_FILE` | abs path | `~/.config/dotagent/state/agents/.../slug.heartbeat.json` | Set when NOT dry-run. | | `AGENT_LIFECYCLE` | string | `persistent` | Only when `[lifecycle] mode = "persistent"`. Absent otherwise, so one script can support both shapes. | | `AGENT_PERSIST_KEY` | string | `12345` | Persistent runs only — which slice this instance answers for (the resolved `[lifecycle] key`, or `default`). | | `AGENT_TRIGGER_SOURCE` | string | `telegram` | One-shot [triggered](/concepts/triggers) runs only. One of `telegram`, `local`, `mcp`, `cli`. **Never set in persistent mode** — see below. | | `AGENT_TRIGGER_ACTOR` | string | `123456789` | One-shot triggered runs, when the source can attest an identity. Telegram: numeric user id. | | `AGENT_TRIGGER_REPLY_TO` | string | `123456789` | One-shot triggered runs, when the source can be answered. Telegram: chat id. | | `AGENT_TRIGGER_PAYLOAD` | JSON object | `{"text":"/standup x","chat_id":1,"user_id":2,"command":{"name":"standup","args":"x"}}` | One-shot triggered runs. Body travels here, never in argv. `command` is present only when the sender invoked one — see [Commands](/concepts/commands#the-payload). `reply_to_run` is present when the sender replied to a notification dotagent sent — see below. | | `AGENT_SESSION_ID` | string | `chat-9_a` | One-shot triggered runs only, when the source has a conversation id. Persistent agents receive this as `trigger.session_id` in each request frame instead. | | `AGENT_ASSISTANT_CONTEXT_RETIRED` | `"true"` | `"true"` | Only on the first triggered `[assistant]` run after automatic transcript retirement. Absent after `/novo` and otherwise. | | `LANG` | string | `en_US.UTF-8` | Only when neither `LANG` nor `LC_ALL` was inherited — see below. | ### `LANG`, and why a daemon has to name one launchd and systemd start a daemon with **no locale at all**, and every agent inherits that gap. A process in the resulting `C` locale has `MB_CUR_MAX == 1`: it reads each **byte** of an environment variable as one character (Latin-1) and writes it back out as UTF-8. `AGENT_TRIGGER_PAYLOAD` carrying `é` (`c3 a9`) reaches the agent as `é` (`c3 83 c2 a9`) — still valid UTF-8, so nothing errors and the agent just acts on mangled text. So dotagent fills the gap: `en_US.UTF-8` on macOS, `C.UTF-8` elsewhere. Only when the value is missing — an inherited `LANG` or `LC_ALL` is never overridden, and `[env.extra]` wins over both: ```toml [env.extra] LANG = "pt_BR.UTF-8" ``` Reproduced with fish 3.7.1, whose command-substitution output is *not* affected — which is why an agent could log a mangled prompt beside a clean answer and look like a model problem. #### `reply_to_run` When the message answers a notification dotagent sent, the payload carries which run it came from: ```jsonc { "text": "por que falhou?", "reply_to_text": "🚨 disk-alert/every-15min gave up after 3 attempts (exit 1)…", "reply_to_run": { "agent": "disk-alert", "schedule": "every-15min", "event": "given_up" } } ``` Resolved from the replied-to message id within the inbound chat, not from the text. The wording is not a stable interface — one event names `agent/schedule` and another says only `preflight aborted by plugin preflight-warp` — and two agents can fail identically. Absent when the reply is to something else, when the notification is older than the few hundred kept in `state/notify/telegram/sent.json`. #### Persistent agents get no per-message trigger environment An environment is fixed at spawn. A persistent process is spawned once and answers many different messages, so those per-message variables would freeze the first one and keep serving it — and stale trigger context reads as perfectly valid, which is worse than absent. In `[lifecycle] mode = "persistent"` the same information arrives in the `trigger` field of each request frame: ```jsonc { "kind": "request", "id": "1", "deadline_seconds": 600, "trigger": { "source": "telegram", "session_id": "chat-9_a", "actor": "123", "reply_to": "123", "payload": { "text": "…", "chat_id": 12345 } } } ``` The request's `trigger` object carries `source`, optional `session_id`, `actor`, `reply_to`, and `payload`. `session_id` is per-trigger context, not a process-wide conversation store. The local one-shot API validates local session ids against `^[A-Za-z0-9_-]{1,64}$` and defaults an omitted id to `default`. See [Local Client API](/reference/local-api). `AGENT_TMPDIR` also changes lifetime: it belongs to the instance rather than to one request, so it survives between them and is removed when the instance is recycled. See [the persistent protocol](/reference/persistent-protocol). The positional `argv` of your process is `[run].command` + `[run].args` * schedule's `args`, so most scripts don't actually need `AGENT_ARGV` unless they want JSON-shaped access. ### Slug derivation `AGENT_SLUG` is computed from the schedule's `args`: | `args` | slug | | ------------------------------ | --------------------- | | `[]` | `default` | | `["--period", "dia-anterior"]` | `period_dia-anterior` | | `["--mode", "unsubscribe"]` | `mode_unsubscribe` | | `["foo bar"]` | `foo_bar` | Rules: strip leading dashes, lowercase, replace non-alphanumeric with `_`, collapse repeated `_`, trim trailing `_`. Empty input → `default`. For triggered runs, `AGENT_SLUG` is the source slug rather than the schedule slug. Without a session it is `trigger-`; with a session it is `trigger--`. The local API uses its effective `default` session when the request omits `session_id`. ### Reading these vars **Fish**: ```fish echo "I am $AGENT_NAME running on schedule $AGENT_SCHEDULE_ID" test "$AGENT_DRY_RUN" = "true"; and exit 0 cd $AGENT_TMPDIR ``` **Python**: ```python import os, json name = os.environ["AGENT_NAME"] argv = json.loads(os.environ.get("AGENT_ARGV", "[]")) if os.environ.get("AGENT_DRY_RUN") == "true": return ``` **Go**: ```go name := os.Getenv("AGENT_NAME") var argv []string json.Unmarshal([]byte(os.Getenv("AGENT_ARGV")), &argv) ``` **Bash**: ```bash : "${AGENT_NAME:?AGENT_NAME not set — running outside dotagent?}" cd "$AGENT_TMPDIR" || exit ``` ### Extra variables you declare `[env].extra` in the manifest is merged on top. Standard idiom for agent-tuning constants: ```toml [env] inherit = true # default [env.extra] LOG_LEVEL = "info" PYTHONUNBUFFERED = "1" DISK_FREE_MIN_PCT = "20" ``` Set `inherit = false` if you want a hermetic environment (no parent env leaks in — careful, that removes `$PATH` too unless you re-add it under `extra`). *** ## Injected into a skill script A [skill](/concepts/skills) may package executables under `scripts/`, which `skill-run` executes. Those get a **different, much smaller** set of variables — a skill script is not an agent run: it has no schedule, no heartbeat and no manifest. | Variable | Type | Example | When | | ------------ | -------- | ------------------------------------------ | ------------------------------------ | | `SKILL_NAME` | string | `triage` | Always. | | `SKILL_DIR` | abs path | `/Users/me/.config/dotagent/skills/triage` | Always — also the working directory. | Everything else in the environment is inherited from whatever spawned `dotagent mcp`. Arguments arrive through argv, never a shell string. ```bash #!/usr/bin/env bash # scripts/report.sh — reach a sibling file without assuming the cwd. set -euo pipefail cat "$SKILL_DIR/references/template.md" ``` *** ## Read by dotagent itself dotagent reads these to override defaults. None are required — everything works out-of-the-box. | Variable | Purpose | Default | | ---------------------------- | ------------------------------------------------------------------------------------------------ | ---------------------------- | | `DOTAGENT_HOME` | Override the root directory. | `~/.config/dotagent` | | `DOTAGENT_ROOT` | Extra (colon-separated) directories to scan for manifests, prepended to the default search list. | (empty) | | `DOTAGENT_PLUGIN_PATH` | Extra (colon-separated) directories to search for plugin binaries. | (empty) | | `RUST_LOG` | Tracing filter — overrides `[logging].level` in `config.toml`. | `info` | | `DOTAGENT_LOG_STDERR` | Force the stderr mirror layer on (`1`/`true`/`yes`/`on`) or off (`0`/`false`/`no`/`off`). | (unset — mirror follows TTY) | | `NO_COLOR` | Set to any value to suppress ANSI escapes in the daemon and every subcommand. | (unset) | | `OTEL_EXPORTER_OTLP_HEADERS` | OTLP auth headers (comma-separated `k=v`). Used when `[telemetry].otlp_endpoint` is set. | (empty) | ### `DOTAGENT_HOME` Moves everything dotagent owns (manifests, state, logs, audit, config) under a different root. ```bash DOTAGENT_HOME=/var/lib/dotagent dotagent doctor DOTAGENT_HOME=/var/lib/dotagent dotagent daemon ``` Both the daemon and CLI must agree — if you start the daemon with `DOTAGENT_HOME=A` and then run `dotagent reload` with `DOTAGENT_HOME=B`, the reload reads the **wrong** PID file (the daemon under A's `state/daemon.pid`). When set in a launchd plist: ```xml EnvironmentVariables DOTAGENT_HOME /var/lib/dotagent ``` When set for a systemd unit: ```ini [Service] Environment=DOTAGENT_HOME=/var/lib/dotagent ``` ### `DOTAGENT_ROOT` Adds **search roots** for manifest discovery, ahead of the defaults. Useful for CI / testing without touching `~/.config/dotagent/`: ```bash DOTAGENT_ROOT=$PWD/examples dotagent doctor DOTAGENT_ROOT=$PWD/examples dotagent run hello-fish --schedule manual ``` The full discovery order with `DOTAGENT_ROOT` set: 1. Every directory in `$DOTAGENT_ROOT` 2. `$DOTAGENT_HOME/agents/` 3. `$CWD/agents/` 4. `$CWD` Each direct subdirectory of these roots that contains an `agent.toml` becomes one agent. Duplicates resolve to first-found by `agent.name`. ### `DOTAGENT_PLUGIN_PATH` Adds search directories for plugin binary resolution, ahead of the defaults: ```bash DOTAGENT_PLUGIN_PATH=$PWD/target/release dotagent plugin list ``` Full plugin discovery order: 1. Every directory in `$DOTAGENT_PLUGIN_PATH` 2. `$DOTAGENT_HOME/plugins/` 3. `/usr/local/lib/dotagent/plugins/` 4. `$PATH` First match wins. `dotagent plugin list` shows the resolved path. > **Daemon gotcha**: when launchd starts the daemon, your interactive shell's `$DOTAGENT_PLUGIN_PATH` is NOT inherited. Set it in the plist (`EnvironmentVariables`), or move the plugin into `$DOTAGENT_HOME/plugins/`. ### `RUST_LOG` Overrides `[logging].level` from `config.toml`. Same `EnvFilter` syntax as the rest of the Rust ecosystem — per-target filters supported: ```bash # Globally chatty RUST_LOG=debug dotagent daemon # Just the runner RUST_LOG=info,dotagent_runner=trace dotagent daemon # Quiet down a noisy crate RUST_LOG=info,h2=warn,hyper_util=warn dotagent daemon ``` The `RUST_LOG` env var only affects the **CLI subcommand it's set for**. To make it sticky for the daemon, put it in the launchd plist / systemd unit `Environment=`. ### `DOTAGENT_LOG_STDERR` Whether the daemon mirrors its `tracing` stream to stderr in addition to `logs/daemon/dotagent.log`. The default is **only when stderr is a terminal**. Run `dotagent daemon` by hand and you get the familiar compact stream; run it under launchd or systemd and you do not — there stderr is an appended plain file (`run.avelino.dotagent-error.log`) that no rotation policy covers, so the mirror would duplicate an already-rotated log into one that grows forever. What still reaches that file is what has nowhere else to go: panics, and startup failures that happen before logging is up. ```bash # Unit rewired to journald, which does rotate — take the stream back. DOTAGENT_LOG_STDERR=1 dotagent daemon # Interactive run, but you only want the JSON file. DOTAGENT_LOG_STDERR=0 dotagent daemon ``` Accepted: `1`, `true`, `yes`, `on` / `0`, `false`, `no`, `off`. Anything else is ignored and the TTY default applies. ### `NO_COLOR` Set to any value (including empty) and dotagent emits no ANSI escapes — in the daemon's stderr mirror and in every subcommand's output. Colour is otherwise enabled only when the stream is a terminal, so redirecting to a file already gives you plain text. Follows [no-color.org](https://no-color.org). ### `OTEL_EXPORTER_OTLP_HEADERS` Vendor-specific authentication for the OTLP exporter. Format is comma-separated `k=v`. Used only when `[telemetry].otlp_endpoint` is non-empty in `config.toml`. ```bash # Honeycomb export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_API_KEY" # Grafana Cloud (base64-encoded basic auth) export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(printf %s "$STACK_ID:$API_TOKEN" | base64)" ``` Headers can also be declared in `[telemetry.headers]` of `config.toml` — the env var wins when both are set. See [`observability.md`](/guides/observability#vendor-recipes) for full vendor recipes. *** ## What dotagent does NOT honor Some env vars you might expect from other tools — dotagent ignores them on purpose: | Variable | Why dotagent ignores it | | ----------------- | ------------------------------------------------------------------------------------- | | `XDG_CONFIG_HOME` | dotagent uses `~/.config/dotagent` directly. Override via `DOTAGENT_HOME`. | | `XDG_DATA_HOME` | dotagent doesn't separate config/data/cache — everything lives under `DOTAGENT_HOME`. | | `EDITOR` | dotagent has no interactive editing. | | `LAUNCH_AGENT` | The daemon is the launchd-managed unit; agents themselves never touch launchd. | *** ## Quick reference ```bash # Agent script — runtime env it sees AGENT_NAME finops-weekly AGENT_HOME ~/.config/dotagent/agents/finops-weekly AGENT_TMPDIR /var/folders/.../tmpXXXXXX AGENT_DRY_RUN false AGENT_SCHEDULE_ID weekly AGENT_SLUG default AGENT_START_EPOCH 1700000000 AGENT_ARGV [] AGENT_HEARTBEAT_FILE ~/.config/dotagent/state/agents/finops-weekly/default.heartbeat.json LANG en_US.UTF-8 (only if none was inherited) # Caller — overrides for the dotagent CLI / daemon DOTAGENT_HOME /var/lib/dotagent (default: ~/.config/dotagent) DOTAGENT_ROOT /tmp/test-agents (prepended to manifest search) DOTAGENT_PLUGIN_PATH $PWD/target/release (prepended to plugin search) RUST_LOG info,dotagent_runner=debug DOTAGENT_LOG_STDERR 1 (default: mirror only on a TTY) NO_COLOR 1 (default: colour only on a TTY) OTEL_EXPORTER_OTLP_HEADERS x-honeycomb-team=KEY ``` *** ## Related * [`paths.md`](/reference/paths) — where every file actually lives * [`agent-spec.md`](/reference/agent-spec) — `[env]` block in the manifest * [`config-reference.md`](/guides/config-reference) — `config.toml` options that env vars can override * [`observability.md`](/guides/observability) — `RUST_LOG` and OTel headers in context # Filesystem layout > Every file dotagent reads or writes, where it lives, and who owns it. dotagent picks **convergence over XDG**: everything lives under a single root. The trade-off is documented in [`crates/dotagent-state/src/paths.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-state/src/paths.rs): finding / inspecting / wiping is easier when one `$DOTAGENT_HOME` holds state, logs, config, and the audit log together. If you religiously follow the XDG Base Directory Spec, set `DOTAGENT_HOME` to your preferred path. *** ## The root ``` $DOTAGENT_HOME/ # default: ~/.config/dotagent ``` Resolution order: 1. `$DOTAGENT_HOME` env var (if set, absolute path) 2. `$HOME/.config/dotagent` 3. `./.dotagent` (last-resort sentinel if `$HOME` isn't set) Everything below is relative to this root. *** ## Top-level ``` $DOTAGENT_HOME/ ├── agents/ # YOUR manifests (or symlinks to them) ├── skills/ # YOUR procedures (optional; also read from ~/.claude/skills) ├── commands/ # YOUR commands (optional; one .md each) ├── plugins/ # YOUR custom plugin binaries (optional) ├── config.toml # global config (optional) ├── secrets.env # daemon-loaded KEY=VALUE secrets (optional, 0600) ├── api.sock # local client API socket (daemon, 0600) ├── state/ # daemon state (read-write, machine-managed) ├── logs/ # operational logs (rotated) ├── audit.log # append-only hash-chained event log └── audit.log.20260806T101500 # rotated segment (kept forever) ``` | Path | Who writes | Who reads | Notes | | ------------- | ------------------ | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `agents/` | **you** | daemon, CLI | Manifests OR symlinks to manifests living elsewhere (e.g., dotfiles). | | `skills/` | **you** (optional) | `dotagent mcp` | One directory per procedure. `~/.claude/skills/` is searched too, so a skill written for Claude Code needs no copy. See [`concepts/skills.md`](/concepts/skills). | | `commands/` | **you** (optional) | daemon, `dotagent mcp` | One `.md` per command, subdirectories namespaced with `:`. Published as the Telegram menu. `~/.claude/commands/` is **not** searched unless `[commands] claude_commands = true`. See [`concepts/commands.md`](/concepts/commands). | | `plugins/` | **you** | `PluginClient` | Per-user plugin binaries. Skip if you install plugins via brew / cargo. | | `config.toml` | **you** (optional) | daemon | Schema in [`config-reference.md`](/guides/config-reference). | | `secrets.env` | **you** (optional) | daemon | KEY=VALUE secrets for `${VAR}` interpolation in notifier configs. **Must be mode 0600.** See [`concepts/secrets.md`](/concepts/secrets). | | `api.sock` | daemon | local clients | Unix-socket JSON-lines API, mode 0600. Created only when `telegram.dispatcher_agent` resolves to a discovered agent. See [`local-api.md`](/reference/local-api). | | `state/` | daemon, runner | daemon, CLI | **Don't edit by hand.** | | `logs/` | daemon | you (via `dotagent logs` / `tail`) | Rotated daily, gzipped after 1d, deleted after retention horizon. | | `audit.log` | daemon | you (`tail`, `jq`) | Append-only, hash-chained. Rotates at 32MB into `audit.log.` segments, which are **never deleted automatically**. | *** ## `agents//` Each direct subdirectory containing an `agent.toml` is an agent. ``` agents// ├── agent.toml # REQUIRED — manifest ├── agent.fish # (or agent.py / agent.go / a built binary) ├── prompt.md # optional — LLM prompt ├── config.json # optional — static data ├── CLAUDE.md # optional — doc-for-LLMs └── README.md # optional — doc-for-humans ``` **Discovery roots** (a manifest is picked up from the first match): 1. Every directory in `$DOTAGENT_ROOT` (colon-separated; for one-off overrides / CI). 2. `$DOTAGENT_HOME/agents/` ← typical production 3. `$CWD/agents/` 4. `$CWD` Each direct subdirectory of a search root that has an `agent.toml` becomes one agent. dotagent indexes by `agent.name` from the manifest, not by directory name — duplicates resolve to first-found. *** ## `skills//` Each direct subdirectory containing a `SKILL.md` is a skill — a procedure an assistant loads on demand, not something that runs on a schedule. ``` skills// ├── SKILL.md # REQUIRED — frontmatter (name, description) + procedure ├── scripts/ # optional — the only files `skill-run` will execute └── references/ # optional — fetched one at a time with `skill-read` ``` **Discovery roots** (first match wins a name): 1. `[skills] paths` from `config.toml` 2. `$DOTAGENT_ROOT` and `$DOTAGENT_ROOT/skills` 3. `$DOTAGENT_HOME/skills/` ← typical 4. `~/.claude/skills/` and `$CWD/.claude/skills/` — unless `[skills] claude_skills = false` 5. `$CWD/skills/` Not scaffolded: an empty catalog is a valid state, and step 4 means a skill you already wrote for Claude Code is reachable without being copied. Symlinks work here the same way they do under `agents/`. See [`concepts/skills.md`](/concepts/skills). *** ## `commands/` Each `.md` file is one command — a prompt **you** invoke by name from the Telegram menu. Not a directory, unlike a skill: a command is text and nothing else, so there is nothing to package alongside it. ``` commands/ ├── standup.md # → /standup └── git/ └── status.md # → /git_status (namespaced `git:status`) ``` **Discovery roots** (first match wins a name): 1. `[commands] paths` from `config.toml` 2. `$DOTAGENT_ROOT/commands` 3. `$DOTAGENT_HOME/commands/` ← typical 4. `~/.claude/commands/` and `$CWD/.claude/commands/` — **only** when `[commands] claude_commands = true` 5. `$CWD/commands/` Step 4 is opt-in, the reverse of skills: a command becomes a published menu entry, and a Claude Code catalog usually assumes a shell it will not have. Nesting goes three levels deep. Not scaffolded, same as `skills/`. See [`concepts/commands.md`](/concepts/commands). *** ## `plugins/` Custom plugin binaries you install yourself (vs. brew / cargo install, which drop binaries into `$PATH` already). ``` plugins/ ├── dotagent-plugin-notify-discord ├── dotagent-plugin-sink-notion └── ... ``` **Discovery order** (`dotagent-plugin-` is resolved against, in sequence): 1. Every directory in `$DOTAGENT_PLUGIN_PATH` (colon-separated) 2. `$DOTAGENT_HOME/plugins/` 3. `/usr/local/lib/dotagent/plugins/` 4. `$PATH` First match wins. See [`plugin-protocol.md`](/reference/plugin-protocol#discovery). *** ## `config.toml` Optional global config. dotagent works with **zero config** — defaults are baked into the binary. ``` config.toml ``` When present, fields you set override defaults; missing fields fall back. Full schema: [`config-reference.md`](/guides/config-reference). *** ## `state/` ``` state/ ├── agents//.heartbeat.json ├── windows/--.json ├── plugins//.json ├── notify//.json ├── notify/alerts.json # re-notification ladder for ongoing alerts ├── known_manifests.json ├── supervisor.json # live subprocess registry (daemon → CLI) └── daemon.pid ``` ### `state/supervisor.json` Snapshot of every subprocess the daemon is supervising right now (agents * plugin invocations). Out-of-process consumers (`dotagent status`, `dotagent doctor`) read it to surface live processes with age vs deadline. Cleared on daemon exit. The daemon refreshes it every 2s *while something is supervised*. Once the registry drains it publishes the empty snapshot once and then stops writing until the next spawn — so an unchanging mtime is the normal resting state, not a sign of trouble. ```jsonc [ { "id": 42, "pid": 3980, "kind": "sink", "owner": { "agent": "hn-digest", "schedule": "weekday-morning", "hook_event": "success", "plugin": "sink-file" }, "label": "sink-file.invoke", "started_at": "2026-05-21T15:00:00.123456-03:00", "deadline_seconds": 300, "age_seconds": 47, "deadline_pct": 15, "pgid": 3980, "command": "dotagent-plugin-sink-file invoke" } ] ``` `pgid` and `command` are not for display — `dotagent status` shows neither. They exist so a **later** daemon can prove that a pid it read off disk is still the process that was recorded, before it signals anything. Both are optional on read: a snapshot written by an older build has neither, and such a record must deserialize into one that is refused rather than fail the whole file. Missing file ⇒ daemon not running. **File age is not a liveness signal**: an idle daemon deliberately stops rewriting it, so an old mtime says nothing about whether the daemon is alive. What the file's mere *existence* means is that the previous daemon did not run its shutdown path (`SIGKILL`, panic, `launchctl kickstart -k`), because that path deletes this file. The next daemon treats the leftover as a record of possible orphans: it reaps the entries whose identity it can confirm, leaves every other entry alone, and then removes the file. An unreadable or unparseable snapshot signals nothing and is left on disk as evidence. See [Boot orphan reap](/guides/daemon-lifecycle#boot-orphan-reap). > **Scope**: `supervisor.json` reflects ONLY the daemon's supervisor. `dotagent run-now` / `dotagent run` invoked standalone instantiate their own short-lived supervisor and do NOT publish to this file — they aren't visible in `dotagent status`. Follow-up to issue #36 will either emit per-PID snapshot files (`supervisor-.json`) or move to a real IPC channel. ### `state/agents//.heartbeat.json` One file per `(agent, slug)` pair. Written before AND after each non-dry-run execution. The shape is intentionally compatible with the legacy Fish framework's heartbeat (see [`migrating-from-fish.md`](/guides/migrating-from-fish)). ```jsonc { "name": "finops-weekly", "slug": "period_dia-anterior", "args": ["--period", "dia-anterior"], "started_at": 1700000000, "started_at_iso": "2023-11-14T22:13:20+0000", "finished_at": 1700000100, "finished_at_iso": "2023-11-14T22:15:00+0000", "exit_code": 0, "duration_seconds": 100, "last_success_at": 1700000100, // preserved across runs; never zeroed on failure "last_success_at_iso": "2023-11-14T22:15:00+0000" } ``` **Slug derivation** (from the schedule's `args`): | `args` | slug | | ------------------------------ | --------------------- | | `[]` | `default` | | `["--period", "dia-anterior"]` | `period_dia-anterior` | | `["--mode", "unsubscribe"]` | `mode_unsubscribe` | Rules: strip leading dashes, lowercase, non-alphanumeric → `_`, collapse `_`, trim trailing `_`. Empty → `default`. ### `state/windows/--.json` One file per `(agent, schedule, expected_at)`, plus a `.lock` beside it. Tracks whether the expected window has been satisfied — drives the retry policy and health computation. The only state directory with a retention horizon: a schedule never revisits a window once it passes, so the directory would otherwise grow forever. The daily 03:00 sweep deletes each pair older than [`[state] window_retention_days`](/guides/config-reference#state) (default 30). ```jsonc { "agent": "hn-digest", "schedule_id": "weekday-morning", "expected_at": 1700000000, "attempts": 1, "last_attempt_at": 1700000000, "last_attempt_exit_code": 0, "last_attempt_stderr": null, "given_up": false, "given_up_at": null } ``` `attempts` counts **dispatches**, not retries — the one that finally succeeded is in there too. So a window that worked on the first try reads `attempts: 1` with `last_attempt_exit_code: 0`, and that is `ok`, not `degraded`. ### `state/plugins//.json` Per-plugin scratch. Format is plugin-defined — dotagent doesn't read this directory itself. Convention from the in-tree plugins (e.g., `sink-roam`): `` is a stable identifier the plugin picks (slug, hash, etc.). ### `state/notify//.json` Built-in notifier rate-limit state. Each driver decides what to write here. The `imessage` driver, for example, persists the last-send timestamp per `(agent, slug)` so `rate_limit_minutes` works across daemon restarts. ### `.lock` Empty file beside every JSON state file, held with `flock` while the file is being changed. Most are held for the write alone, because the write is the whole update. `notify/alerts.lock` and `audit.log.lock` are held longer — across the entire read-modify-write cycle — because both files are read, edited, and written back, and two processes really do run those cycles (the daemon and `dotagent tick`). A lock scoped to the write alone would let one of them save a table that never saw the other's change. It is **not** removed afterwards, on purpose. Unlinking it while still holding the lock lets the next writer open a fresh inode and take its own "exclusive" lock, so two processes end up writing the same file believing they have it to themselves. Safe to delete when nothing is running; it reappears on the next write. ### `outl/` Long-term agent memory, as an [outl](https://github.com/avelino/outl) workspace. Scaffolded when the daemon starts. ``` outl/ ├── journals/YYYY-MM-DD.md # the facts, one block each ├── journals/YYYY-MM-DD.outl # block identity sidecar ├── ops/ # op log — source of truth └── pages/ templates/ assets/ ``` A normal outl workspace, on purpose: open it in the desktop app, read what an agent remembered, fix a wrong memory, delete a page. Relocate with `[memory] workspace` in `config.toml`. See [Memory](/concepts/memory). ### `state//` Whatever an agent needs to keep between runs, under a directory named after it. dotagent writes nothing here; the agent owns it. [`examples/telegram-assistant`](https://github.com/avelino/dotagent/tree/main/examples/telegram-assistant/README.md) uses `state/telegram-assistant/` for which `claude` session belongs to which chat, and how many times that session has been retired for length: ``` state/telegram-assistant/ ├── .started # session id, written only after a run that created it └── .gen # generation counter, bumped on each retirement ``` An agent that keeps state here should declare the path in `[security] filesystem_writable`, so `doctor` can audit what it writes. ### `state/notify/alerts.json` When each ongoing alert last spoke, so a condition that stays true does not notify on every tick. ```jsonc { "episodes": { "disk-alert/every-15min/stale": { "first_notified_at": 1785925367, "last_notified_at": 1786011767, "count": 2 } } } ``` Keyed `agent/schedule/event`. An episode starts at the first notification and is **deleted** when that schedule succeeds again, so the next failure alerts immediately instead of inheriting the previous episode's spacing. Re-notify ladder: on entry, then 1h, 6h, and daily thereafter. Losing the file costs at most one duplicate alert — the opposite failure (losing the alert) is the expensive one, so a missing or corrupt table is read as "nothing has been notified yet". See [notifications](/concepts/notifications#stale-and-why-alerts-repeat). ### `state/notify/telegram/sent.json` What each outbound notification was about, so a reply to one resolves to the run that caused it. ```jsonc { "entries": { "5:-1001:4821": { "chat_id": "-1001", "agent": "disk-alert", "schedule": "every-15min", "event": "given_up", "at": 1785925367 } } } ``` The key is scoped by Telegram chat, because `message_id` is only unique within that chat. Older entries may keep the numeric key and omit `chat_id`; they stay available to legacy callers but are not used for scoped Telegram correlation. Capped at the most recent 500, oldest dropped first. It is a lookup table for alerts you might still answer, not a history — and losing it costs correlation on old messages, never delivery. ### `state/notify/telegram/threads.json` Which conversation each recent group message belongs to, so a Telegram reply inherits the thread it answers instead of mixing subjects into one session. ```jsonc { "entries": { "6:-1001:4821": { "chat_id": "-1001", "session": "-1001-r4821", "at": 1785925367 } } } ``` Both ends of an exchange are bound: the inbound message when it is screened, and the bot's reply when it is delivered — so replying to either continues the thread. The key is scoped by chat, like `sent.json`, because Telegram message ids are only unique within a chat. Capped at the most recent 1000, oldest dropped first. Losing an entry costs thread continuity on that one reply (it roots a new conversation), never a delivered message. Direct chats never touch this file — their session is the chat id. See [Telegram](/concepts/telegram#conversations-and-threads). ### `state/notify/telegram/offset.json` Last acknowledged Telegram `update_id`, written tmp-then-rename. ```jsonc { "offset": 481923771 } ``` Telegram redelivers every update until a higher `offset` acknowledges it. Without this file a daemon restart replays the backlog — and for a bot that runs agents, replay means re-running whatever the last messages asked for. Delivery is at-most-once on purpose. See [Telegram](/concepts/telegram). ### `state/known_manifests.json` Cache of `sha256(agent.toml)` for every loaded manifest. Drives [manifest drift detection](/security/threat-model): ```jsonc { "entries": { "finops-weekly": { "sha256": "a3f9...", "path": "/Users/me/.config/dotagent/agents/finops-weekly/agent.toml", "first_seen_at_iso": "2026-05-19T14:00:00-0300" } } } ``` On each daemon load: * New name in `agents/` not in the cache → `PhantomAgentDetected` (critical, notify). * Existing name but mismatched sha → `ManifestDriftDetected` (critical, notify). ### `state/daemon.pid` The running daemon's PID. Used by `dotagent reload` (sends SIGHUP) and removed on graceful exit via a `Drop` guard. Stale pidfile (no live process at that PID) means the daemon crashed without cleanup — restart it. *** ## `logs/` ``` logs/ ├── daemon/ │ ├── dotagent.log # structured JSON, daily rotation │ ├── dotagent.log.2026-05-19 # yesterday's rolled file │ ├── dotagent.log.2026-05-18.gz # older, gzipped │ ├── run.avelino.dotagent.log # launchd / systemd stdout capture │ └── run.avelino.dotagent-error.log # launchd / systemd stderr — crashes only ├── agents/ │ └── / │ ├── .log # raw stdout+stderr from the agent │ ├── .log.2026-05-19 # rolled │ └── .log.2026-05-18.gz # gzipped └── plugins/ └── / # (currently unused; reserved) ``` | File | Format | Rotation | Retention default | | --------------------------------------- | ------------------ | --------------------------- | ----------------- | | `daemon/dotagent.log` | NDJSON (`tracing`) | daily | 30 days | | `daemon/run.avelino.dotagent.log` | Raw text | (launchd / systemd appends) | (managed by OS) | | `daemon/run.avelino.dotagent-error.log` | Raw text | (launchd / systemd appends) | (managed by OS) | | `agents//.log` | Raw stdout+stderr | daily | 14 days | Compression: rotated files older than `compress_after_days` (default 1) get gzipped in-place. Deletion: files older than the retention horizon are removed by the 03:00 sweep. The **active** (non-rotated) file is never deleted, whatever the horizon says: launchd and systemd hold an open fd on it, so unlinking it would strand every subsequent write in a file with no name. ### The stderr file is a crash channel, not an activity log `run.avelino.dotagent-error.log` receives panics and pre-logging startup failures. It does **not** receive the daemon's `tracing` stream. The daemon installs its stderr mirror layer only when stderr is a terminal. Under launchd / systemd stderr is an appended plain file that no rotation policy covers, so mirroring there would duplicate `dotagent.log` into a file that grows without bound. On a healthy daemon this file stays empty — that is the intended state, not a symptom. | `DOTAGENT_LOG_STDERR` | Effect | | ---------------------------- | ---------------------------------------------------------------------- | | unset (default) | mirror on **only** if stderr is a TTY | | `1` / `true` / `yes` / `on` | force the mirror on — for units rewired to journald, which does rotate | | `0` / `false` / `no` / `off` | force it off, even in a terminal | | anything else | ignored; falls back to the TTY default | ANSI colour follows the same rule and additionally honours `NO_COLOR`: set it to any value and escapes are suppressed in the daemon and in every subcommand. The full schema + jq examples are in [`guides/observability.md`](/guides/observability). *** ## `audit.log` ``` audit.log ``` **Hash-chained, append-only. Rotates by size; segments are never deleted automatically.** One JSON object per line. Each line carries `prev_hash = sha256(previous line's full JSON)`. The very first line ever written has `prev_hash = "GENESIS"`. On startup the daemon verifies the chain; if it breaks, an `AuditChainBroken` entry is appended (which itself becomes a chained entry — anchoring the new chain to the broken position). ### Rotation and the seam Past 32MB the live file is renamed to `audit.log.` and a fresh `audit.log` opens with a **seam** as its first line: ```jsonc { "ts": "2026-08-06T10:15:00-0300", "severity": "notice", "event": { "event_type": "audit_log_rotated", "rotated_to": "audit.log.20260806T101500", "entries": 38513, "first_ts": "2026-05-19T08:31:07-0300", "tail_hash": "c8d2..." }, "prev_hash": "c8d2..." // == tail_hash: the chain crosses the rename } ``` The seam's `prev_hash` **is** the rotated segment's tail hash, so the chain has no gap at the rename. And because the seam names what left, deleting an old segment is legible: verification reports *"intact since `first_ts`; 38513 earlier entries lived in `audit.log.20260806T101500`, which is gone"* rather than *"chain broken"*. Cutting lines off the head of the live file takes the seam with them, and the orphaned `prev_hash` that remains is accounted for by nothing — which is the case that fires `audit_chain_broken`. Segments are **never removed by dotagent**. The log rotation sweeper in `[logs]` does not touch them; this is the only forensic artifact dotagent keeps and pruning it is the operator's call. `rm` them when you mean to. Verification has two scopes: | Scope | What it walks | Used by | | --------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | current segment | the live `audit.log` only | daemon at boot, and `dotagent audit verify` — the live file is the only one that changes | | full | follows seams backwards through every segment still present | `dotagent audit verify --full`, when you want the guarantee to reach `GENESIS` | `dotagent audit verify` prints the verdict and exits non-zero on a break or an unexplained truncation; `--json` emits one machine-parseable line. See [`cli.md`](/reference/cli#audit-verify). Example line (pretty-printed): ```jsonc { "ts": "2026-05-19T14:30:01-0300", "severity": "info", "event": { "event_type": "agent_run", "agent": "finops-weekly", "schedule": "weekly", "slug": "default", "manifest_sha256": "a3f9...", "exit_code": 0, "duration_seconds": 47, "timed_out": false }, "prev_hash": "c8d2..." } ``` Audit events emitted by dotagent: | `event_type` | When | Severity | | ------------------------- | ------------------------------------------------------------------------------------------------------------------ | --------------- | | `daemon_started` | Daemon process boots | info | | `daemon_stopped` | Daemon receives SIGTERM / SIGINT | info | | `tick_started` | **No longer emitted.** Parsed so existing logs stay readable — see below | info | | `tick_completed` | **No longer emitted.** Parsed so existing logs stay readable — see below | info | | `agent_run` | An agent completed (success or failure) | info / critical | | `agent_recovered` | A previously-failing schedule passed | notice | | `agent_given_up` | All retries exhausted for a window | critical | | `preflight_failed` | A preflight plugin blocked the run | critical | | `plugin_invoked` | Any plugin / notifier invocation | info / notice | | `manifest_loaded` | Manifest read on daemon start / SIGHUP | info | | `manifest_drift_detected` | sha256(manifest) doesn't match cache | critical | | `phantom_agent_detected` | Discovered agent not in `known_manifests.json` | critical | | `audit_chain_broken` | Hash chain verification failed at line N, or the head of the log was removed with nothing accounting for it | critical | | `audit_log_rotated` | The log passed 32MB; the seam naming the segment it came from and that segment's tail hash | notice | | `config_reloaded` | SIGHUP picked up changes to `config.toml` | notice | | `secrets_loaded` | Daemon read `secrets.env`; payload has `path`, `key_count`, `unresolved_references` (no values, no `op://` paths) | notice | | `secrets_refused` | Daemon rejected `secrets.env` (insecure mode, parse, or IO) | critical | | `manifest_invalid` | An `agent.toml` exists but failed to parse or validate; that agent is skipped, the rest keep running | critical | | `trigger_received` | Inbound message accepted; records `actor` and `reply_to`, never the text | notice | | `trigger_rejected` | Inbound message refused (allowlist, rate limit) before anything ran | critical | | `agent_triggered` | A run started from a trigger rather than a schedule window | notice | | `command_dispatched` | A `/name` invocation was resolved; records the name and whether it was known, never the arguments | notice | | `skill_invoked` | A `scripts/` executable inside a skill ran — code outside any manifest, which `agent_run` would not record | notice | | `skill_invalid` | A `SKILL.md` exists but failed to parse; that procedure is missing from the catalog, the rest keep working | notice | | `remediation_invoked` | A declared `[[preflight]] remediation` ran. The one event where a chat message changed the machine | critical | | `orphan_reaped` | A process a previous daemon left running was killed at boot, after its identity was confirmed against the snapshot | critical | `Critical` severity drives out-of-band notifier dispatch. Defined in [`crates/dotagent-core/src/audit.rs`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-core/src/audit.rs). ### Retired: `tick_started` / `tick_completed` The daemon no longer writes these. A tick is telemetry, not an auditable event — "woke up and looked at 17 agents" tells a forensic reader nothing, and it dominated the file: on one real 38,510-entry log the two variants were 64% of every line. Because each append re-reads the log to find the tail hash, that noise also made every event worth recording more expensive to write. The daemon emits `tracing` at `debug` level instead, which already rotates — see [observability](/guides/observability). The enum variants are **kept** so existing logs stay parseable. Removing them would make `dotagent audit verify` and chain verification fail the moment they reached a historic tick entry, and a chain you can no longer verify is worse than one carrying dead weight. *** ## Platform-specific paths (outside `$DOTAGENT_HOME`) These are written by `dotagent install`, not by the daemon at runtime: ### macOS ``` ~/Library/LaunchAgents/run.avelino.dotagent.plist ``` Template: [`crates/dotagent-unit-gen/templates/daemon.plist`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-unit-gen/templates/daemon.plist). Rendered properties: * `Label` = `run.avelino.dotagent` * `ProgramArguments` = `["", "daemon"]` * `RunAtLoad` = true * `KeepAlive` = true * `ProcessType` = `Background` * `ThrottleInterval` = 10 * `StandardOutPath` = `$DOTAGENT_HOME/logs/daemon/run.avelino.dotagent.log` * `StandardErrorPath` = `$DOTAGENT_HOME/logs/daemon/run.avelino.dotagent-error.log` ### Linux ``` ~/.config/systemd/user/run.avelino.dotagent.service ``` Template: [`crates/dotagent-unit-gen/templates/daemon.service`](https://github.com/avelino/dotagent/tree/main/crates/dotagent-unit-gen/templates/daemon.service). Rendered properties: * `[Service] Type=simple` * `ExecStart= daemon` * `Restart=always` * `RestartSec=10` * `StandardOutput=append:` * `StandardError=append:` * `[Install] WantedBy=default.target` *** ## Permissions dotagent runs as **your user** (no daemon root, no setuid). Files are written with your umask (`0644` for regular files, `0755` for directories on a typical user shell). Override by setting umask before launching the daemon: ```bash # In your launchd / systemd unit override, or shell rc: umask 027 # group-readable only ``` macOS Full Disk Access: if your agent or `sink-file` writes into `~/Documents` / `~/Downloads` / `~/Desktop`, the daemon binary needs the **Full Disk Access** entitlement under System Settings → Privacy & Security. *** ## What dotagent **does NOT** put under `$DOTAGENT_HOME` * launchd plist / systemd unit (platform-specific, see above). * The `dotagent` binary itself (lives in `~/.cargo/bin/`, `/opt/homebrew/bin/`, `/usr/local/bin/`, etc.). * Plugin binaries installed via `cargo install` / `brew` (those go to `~/.cargo/bin/` / Homebrew prefix). * Your agent scripts' working data, unless your script writes there deliberately. dotagent gives each run a fresh `$AGENT_TMPDIR` that auto-cleans on exit. *** ## Related * [`env-vars.md`](/reference/env-vars) — `DOTAGENT_HOME`, `DOTAGENT_ROOT`, `DOTAGENT_PLUGIN_PATH`, etc. * [`config-reference.md`](/guides/config-reference) — `config.toml` schema * [`observability.md`](/guides/observability) — log format + jq recipes * [`threat-model.md`](/security/threat-model) — audit log's forensic role # Overview dotagent ships with first-party plugins, installed alongside the `dotagent` binary by the Homebrew formula. Each has its own page below with full config schema, behavior details, examples, and troubleshooting. For the **concept** (what a plugin is, how to use one, how to write one), see [`docs/plugins.md`](/concepts/plugins). For the **protocol spec** (verbs, JSON shape, exit codes), see [`docs/reference/plugin-protocol.md`](/reference/plugin-protocol). > **Note on notifications.** Notifications **used to be plugins** (`notify-imessage`, `notify-desktop`, etc). They are now **built into the daemon** under the `[[notifiers]]` array — no plugin subprocess, native APIs (NSUserNotification on macOS, D-Bus on Linux, HTTPS for Slack/ntfy/Pushover/Telegram). See [`docs/concepts/notifications.md`](/concepts/notifications) for the new shape. ## Preflight plugins (`kind = "preflight"`) Run BEFORE the agent. If any returns `ok=false`, the agent is aborted and `on_failure` fires with `event = "preflight"`. | Plugin | Platforms | When to pick | | -------------------------------------------------------------- | ------------- | ----------------------------------------------------------------- | | [`preflight-warp`](/built-in-plugins/preflight/preflight-warp) | macOS + Linux | Agent depends on a corporate VPN routed via Cloudflare WARP. | | [`preflight-cmd`](/built-in-plugins/preflight/preflight-cmd) | macOS + Linux | Generic: run any shell command, gate on exit code + stdout match. | ## Sink plugins (`kind = "sink"`) Run AFTER a successful agent run. Persist the captured stdout somewhere. | Plugin | Platforms | When to pick | | ----------------------------------------------- | ------------- | ------------------------------------------------------------------------ | | [`sink-roam`](/built-in-plugins/sink/sink-roam) | macOS + Linux | Output is hierarchical markdown destined for Roam Research. | | [`sink-outl`](/built-in-plugins/sink/sink-outl) | macOS + Linux | Output is hierarchical markdown destined for Outl (single batched call). | | [`sink-file`](/built-in-plugins/sink/sink-file) | macOS + Linux | Output is a single file (overwrite or append). | ## Discovery dotagent finds these binaries via `$PATH` (Homebrew installs them in `/opt/homebrew/bin/` on Apple silicon / `/usr/local/bin/` on Intel / Linux). Custom plugins go in `~/.config/dotagent/plugins/` or anywhere listed in `$DOTAGENT_PLUGIN_PATH`. Verify what's available: ```bash dotagent plugin list dotagent doctor # validates every reference in your manifests dotagent-plugin- info | jq . # raw metadata for any plugin ``` # Preflight # preflight-cmd > Generic preflight: run any command, check its exit code (and optionally match a substring in stdout/stderr). If the check fails, the agent run is aborted. | Property | Value | | ------------- | ------------------------------- | | Kind | `preflight` | | Platforms | `darwin`, `linux` | | Binary | `dotagent-plugin-preflight-cmd` | | External deps | whatever command you configure | ## What it does Spawns the configured command, captures stdout + stderr + exit code, and compares against the expected values. Returns `ok: true` when both: 1. Exit code equals `expect_exit` (default `0`) 2. If `expect_contains` is set, that substring appears in either stdout or stderr ## When to use * You need a preflight gate but no specialized plugin exists yet. * The check is a one-off command that doesn't justify a dedicated plugin (`gh auth status`, `aws sts get-caller-identity`, `ping -c1 ...`). * You're prototyping — get the gate working, then upstream a specialized plugin later if it sticks. ## Config schema | Field | Type | Required | Default | Description | | ----------------- | ------- | -------- | ------- | -------------------------------------------------------- | | `command` | string | **yes** | — | Binary to execute (resolved via `$PATH` unless absolute) | | `args` | array | no | `[]` | Arguments to pass | | `expect_exit` | integer | no | `0` | Expected exit code | | `expect_contains` | string | no | (none) | Substring required in stdout OR stderr | Verify schema at runtime: ```bash dotagent-plugin-preflight-cmd info | jq .schema ``` ## Examples ### gh auth check before an agent that uses the GitHub CLI ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "gh", args = ["auth", "status"], expect_exit = 0 } ``` ### AWS credentials valid before a FinOps agent ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "aws", args = ["sts", "get-caller-identity"], expect_exit = 0 } ``` ### Network reachability check ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "ping", args = ["-c", "1", "-W", "2", "api.github.com"], expect_exit = 0 } ``` ### Combined exit + content check Verify `gws` (Google Workspace CLI) is logged in by checking specific text in the output: ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "gws", args = ["gmail", "users", "labels", "list", "--params", "{\"userId\":\"me\"}"], expect_exit = 0, expect_contains = "labels" } ``` If `gws` is logged out, exit will be 0 but the output won't contain "labels" — `expect_contains` catches the silent-fail case. ### Custom non-zero "success" Some tools use non-zero for valid states (`grep` returns 1 on no-match): ```toml [[preflight]] plugin = "preflight-cmd" config = { command = "grep", args = ["-q", "ServerAliveInterval", "~/.ssh/config"], expect_exit = 0 # 0 = match found = ssh keepalive configured = ok } ``` ## Response shape ### Passed ```json { "ok": true, "exit_code": 0 } ``` ### Failed (wrong exit code) ```json { "ok": false, "error": "preflight command failed", "exit_code": 1 } ``` ### Failed (substring not found) ```json { "ok": false, "error": "preflight command failed", "exit_code": 0 } ``` (Exit was OK but `expect_contains` didn't match. The `error` message is generic — check the agent log for the `dotagent doctor` / daemon log to see which condition failed.) ## Behavior details * **No shell.** `command` is executed directly with `args` as argv. No pipes, no globs, no env-var expansion. If you need shell features, invoke `/bin/sh -c "..."` explicitly: ```toml config = { command = "/bin/sh", args = ["-c", "test -f /tmp/ready && grep -q ok /tmp/ready"] } ``` * **Substring match is case-sensitive.** Use the exact string you expect. * **Stdout AND stderr are searched** for `expect_contains` — some tools log to stderr (e.g., `gh auth status` writes to stderr by design). * **No timeout inside the plugin.** If your command hangs, the daemon's `agent.timeout_seconds` doesn't apply (that's the agent's timeout, not the preflight's). Keep preflight commands fast. ## External dependencies Whatever your `command` is. The plugin itself has no runtime deps beyond the OS. ## Manual testing ```bash # 1) Info dotagent-plugin-preflight-cmd info | jq . # 2) Validate echo '{"command":"gh","args":["auth","status"]}' \ | dotagent-plugin-preflight-cmd validate # 3) Real invoke echo '{ "kind": "preflight", "agent": "test", "schedule": "test", "event": "preflight", "config": {"command":"gh","args":["auth","status"],"expect_exit":0} }' | dotagent-plugin-preflight-cmd invoke # 4) Negative test — make it fail echo '{ "config": {"command":"false"} }' | dotagent-plugin-preflight-cmd invoke # → {"ok":false,"error":"preflight command failed","exit_code":1} ``` ## Troubleshooting ### Plugin always returns `exit_code: -1` The command couldn't be spawned (binary missing, permission denied). The real error is on stderr: ```bash echo '{"config":{"command":"nonexistent"}}' \ | dotagent-plugin-preflight-cmd invoke 2>&1 ``` ### `expect_contains` not matching even though I see the text * Case sensitivity: `Logged in` ≠ `logged in`. * ANSI escape codes (some CLIs color their output): pipe through `--no-color` or `2>&1 | cat` to verify the raw bytes. * The substring must be in stdout OR stderr **of the spawned process** — not in YOUR terminal after shell processing. ### I need conditional logic (run only on weekdays etc.) Wrong tool. Conditional logic lives in the schedule (`weekdays`) or in the agent script. Preflight is a binary pass/fail gate. ## See also * [Concept guide](/concepts/plugins) * [`preflight-warp`](/built-in-plugins/preflight/preflight-warp) — specialized version for WARP * Source: [`plugins/preflight-cmd/`](https://github.com/avelino/dotagent/blob/main/plugins/preflight-cmd/README.md) # preflight-warp > Gate an agent's run on Cloudflare WARP being connected. Returns `ok: false` (and optionally a `suggest` hint) when WARP is not in `Connected` state, which aborts the run and fires `on_failure` with `event = "preflight"`. | Property | Value | | ------------- | ------------------------------------- | | Kind | `preflight` | | Platforms | `darwin`, `linux` | | Binary | `dotagent-plugin-preflight-warp` | | External deps | `warp-cli` (Cloudflare's WARP client) | ## What it does Runs `warp-cli status`, parses stdout for the literal substring `Connected`, and returns `ok: true` only on a match. On failure, the response includes: * `error` — short summary (`WARP not connected`) * `suggest` — the manifest's configured `connect_command` (so a notification can show the user *how* to fix it) * `status` — the raw `warp-cli status` stdout for context dotagent aborts the run when the response is `ok: false`. The agent script never starts. ## When to use * Your agent depends on internal services routed through a corporate VPN (Cloudflare's WARP / Zero Trust). * The first symptom of WARP being off is a long sequence of timeouts / 401s from inside the agent — much louder to fail fast at preflight. * You want the failure notification to *tell you* the exact command to reconnect, so you don't have to remember. ## Config schema | Field | Type | Required | Default | Description | | ----------------- | ------ | -------- | ---------- | ------------------------------------------------------------------ | | `binary` | string | no | `warp-cli` | Override the binary name (e.g., `/usr/local/bin/warp-cli`) | | `connect_command` | string | no | — | Hint reported in the response so a notification plugin can echo it | Both fields are optional — the plugin works with zero config when `warp-cli` is on `$PATH`. Verify schema at runtime: ```bash dotagent-plugin-preflight-warp info | jq .schema ``` ## Examples ### Minimal — abort the run unless WARP is connected ```toml [[preflight]] plugin = "preflight-warp" ``` ### With reconnect hint ```toml [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } ``` When this fails and `on_failure` fires, the message includes `preflight aborted by plugin preflight-warp: warp-cli connect` — so the iMessage / Slack alert hands you the command to run. ### Combined with retry to wait for the user ```toml [agent] name = "team-standup" timeout_seconds = 1200 [defaults] max_retries = 20 # tries every backoff window… retry_backoff_minutes = [30] # …every 30 minutes… stale_after_minutes = 720 # …until the window is 12 hours old. [[preflight]] plugin = "preflight-warp" config = { connect_command = "warp-cli connect" } [[notifiers]] driver = "imessage" to = "+5511..." rate_limit_minutes = 60 events = ["preflight"] ``` This is the production pattern from the legacy `team-standup`: the agent keeps retrying every 30 min while WARP is down, but the iMessage alert is rate-limited to one per hour so you don't get spammed during a long disconnection. ## Response shape ### WARP connected ```json { "ok": true, "status": "Status update: Connected\nSuccess" } ``` ### WARP disconnected ```json { "ok": false, "error": "WARP not connected", "suggest": "warp-cli connect", "status": "Status update: Disconnected\nSuccess" } ``` ### `warp-cli` not installed / not on PATH ```json { "ok": false, "error": "could not run warp-cli", "suggest": "warp-cli connect" } ``` ## Behavior details * **Match is substring `Connected`.** The plugin doesn't parse the full status structure — `warp-cli status` reliably emits `Status update: Connected` when the tunnel is up. * **No retry inside the plugin.** Retrying is dotagent's job — the manifest controls how often the daemon re-fires the agent. * **Exit code is always 0** (the plugin spoke successfully), but `ok: false` blocks the agent. dotagent's audit log differentiates them. ## External dependencies * **Cloudflare WARP client** with `warp-cli`: * macOS: download from * Linux: `apt install cloudflare-warp` or Cloudflare's RPM repo * The user must have already run `warp-cli registration new` once. ## Manual testing ```bash # 1) Info dotagent-plugin-preflight-warp info | jq . # 2) Validate (no config needed) echo '{}' | dotagent-plugin-preflight-warp validate # 3) Real invoke echo '{ "kind": "preflight", "agent": "test", "schedule": "test", "event": "preflight", "config": {"connect_command":"warp-cli connect"} }' | dotagent-plugin-preflight-warp invoke # 4) Confirm WARP state directly warp-cli status ``` Force a `false` response by disconnecting: ```bash warp-cli disconnect echo '{"config":{"connect_command":"warp-cli connect"}}' \ | dotagent-plugin-preflight-warp invoke # → {"ok":false,"error":"WARP not connected","suggest":"warp-cli connect","status":"..."} warp-cli connect ``` ## Troubleshooting ### Plugin returns `ok: false` even when WARP is connected `warp-cli status` output varies by version. Confirm what your version prints: ```bash warp-cli status | head -2 ``` If it doesn't contain the literal word `Connected`, file an issue — we'll widen the match. ### `could not run warp-cli` Either: * `warp-cli` isn't installed: install Cloudflare WARP. * It's installed but not on the daemon's `$PATH`: pass the full path: ```toml [[preflight]] plugin = "preflight-warp" config = { binary = "/usr/local/bin/warp-cli", connect_command = "warp-cli connect" } ``` ### Agent runs even with WARP off Confirm the `[[preflight]]` block is in the manifest: ```bash dotagent doctor | grep preflight ``` If the plugin is unresolved (`plugin preflight-warp not found`), the preflight is silently skipped. Install with: ```bash brew install dotagent # bundles all first-party plugins ``` ## See also * [Concept guide](/concepts/plugins) * [`preflight-cmd`](/built-in-plugins/preflight/preflight-cmd) — generic version (any command + exit check) * Source: [`plugins/preflight-warp/`](https://github.com/avelino/dotagent/blob/main/plugins/preflight-warp/README.md) * Cloudflare WARP: # Sink # sink-file > Persist an agent's stdout to a file. Two modes: overwrite (default) or append. Creates parent directories on demand. | Property | Value | | ------------- | --------------------------- | | Kind | `sink` | | Platforms | `darwin`, `linux` | | Binary | `dotagent-plugin-sink-file` | | External deps | none | ## What it does Writes `payload.message` to the configured `path`. With `mode = "overwrite"` (default) the file is replaced. With `mode = "append"` the message is appended to the end (no implicit newline beyond whatever's in the message). Parent directories are created with `mkdir -p` semantics. ## When to use * You want the agent's output **in a file you `grep`/`cat` later**, not pushed to a service. * You're feeding another tool (Hugo, mkdocs, your local search) that watches a directory. * You want a quick audit trail without standing up Roam / Notion / etc. ## Config schema | Field | Type | Required | Default | Description | | ------ | ------ | -------- | ----------- | ---------------------------------------------------------- | | `path` | string | **yes** | — | Destination file path (absolute or relative to daemon CWD) | | `mode` | string | no | `overwrite` | `overwrite` (default) or `append` | Verify schema at runtime: ```bash dotagent-plugin-sink-file info | jq .schema ``` ## Examples ### Daily snapshot (overwritten each run) ```toml [[on_success]] plugin = "sink-file" config = { path = "/Users/me/reports/hn-today.md" } ``` After today's run, the file contains exactly today's output. No accumulation. ### Append-only log (one entry per run) ```toml [[on_success]] plugin = "sink-file" config = { path = "/Users/me/dotagent-history/standup.log", mode = "append" } ``` Each run grows the file. Useful for "did this agent ever generate the phrase X?" type retrospective greps. ### Date-stamped filename via shell `sink-file` doesn't template `path`. To get a per-day filename, have the agent itself build the path and use a different sink. Simplest is to use both `sink-file` AND let the agent write its own dated copy: ```fish # In agent.fish — write the dated copy yourself. set -l dated /Users/me/reports/(date +%Y-%m-%d)-summary.md cp $AGENT_TMPDIR/output.txt $dated ``` And use `sink-file` for the "latest" alias: ```toml [[on_success]] plugin = "sink-file" config = { path = "/Users/me/reports/latest-summary.md" } ``` ## Response shape ### Success ```json { "ok": true, "written_to": "/Users/me/reports/hn-today.md" } ``` ### Failed validation ```json { "ok": false, "error": "path is required" } ``` ### Runtime failure Exit non-zero. Common causes: * Parent directory exists but isn't writable (permission denied) * `path` collides with an existing directory of the same name * Disk full Stderr carries the raw `io::Error`. ## Behavior details * **Newlines**: the plugin writes exactly `payload.message` bytes. If your message ends without a trailing newline, the file doesn't get one either. Tools that expect newline-terminated lines should append `\n` to the agent's stdout. * **Atomic-ish**: `overwrite` mode uses `std::fs::write` which truncates * writes. There's a brief window where the file is empty if the daemon crashes mid-write. Use `mode = "append"` if you need stricter durability (each entry is an `O_APPEND` write). * **Encoding**: bytes-through. If your message is UTF-8 (almost always is), the file is UTF-8. No BOM, no transformation. * **Permissions**: written with the daemon's umask (`0644` by default). Change your shell's umask before launching the daemon if you need tighter perms. ## External dependencies None. Pure stdlib I/O. ## Manual testing ```bash # 1) Info dotagent-plugin-sink-file info | jq . # 2) Validate echo '{"path":"/tmp/test.txt"}' | dotagent-plugin-sink-file validate # 3) Overwrite echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "hello\nworld\n", "config": {"path":"/tmp/dotagent-sink-test.txt"} }' | dotagent-plugin-sink-file invoke cat /tmp/dotagent-sink-test.txt # hello # world # 4) Append echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "second line\n", "config": {"path":"/tmp/dotagent-sink-test.txt","mode":"append"} }' | dotagent-plugin-sink-file invoke cat /tmp/dotagent-sink-test.txt # hello # world # second line ``` ## Troubleshooting ### File doesn't appear * Check the daemon's audit log to confirm `sink-file` was invoked: ```bash tail ~/.config/dotagent/audit.log \ | jq 'select(.event.event_type == "plugin_invoked" and .event.plugin == "sink-file")' ``` * Was `payload.message` empty? `dotagent run ` first; the message is the captured stdout. ### Permission denied Two flavors: 1. **Parent dir not writable** — fix with `chmod` or pick a different path. 2. **macOS TCC** — if `path` is under `~/Documents`, `~/Downloads`, `~/Desktop`, the daemon needs Full Disk Access. Settings → Privacy → Full Disk Access → add the dotagent daemon binary. ### Append mode silently overwrote my file It didn't — but it might have inherited the prior overwrite if you changed `mode` after one overwrite-mode run. Read the file's actual length before / after: ```bash wc -c /tmp/dotagent-sink-test.txt ``` If it grew, append worked. ### Concurrent runs of the same agent corrupt the file dotagent ensures only one run per `(agent, slug)` at a time, so this shouldn't happen organically. If you're invoking the plugin manually in parallel, use `mode = "append"` — `O_APPEND` is atomic on POSIX up to PIPE\_BUF (4KiB on Linux, 512B on macOS). For larger writes, you'll need a different strategy (write to tmp + rename). ## See also * [Concept guide](/concepts/plugins) * [`sink-roam`](/built-in-plugins/sink/sink-roam) — for hierarchical / Roam destinations * Source: [`plugins/sink-file/`](https://github.com/avelino/dotagent/tree/main/plugins/sink-file/README.md) # sink-roam > Publish an agent's stdout to Roam Research as a hierarchical block tree (root + L1 sections + L2 children). Idempotent: re-running a schedule replaces the previous block instead of duplicating. | Property | Value | | ------------- | ---------------------------------------------- | | Kind | `sink` | | Platforms | `darwin`, `linux` | | Binary | `dotagent-plugin-sink-roam` | | External deps | `mcp` CLI with `roam` server configured | | Network | Whatever the `mcp roam` server hits internally | ## What it does This plugin is a faithful port of the legacy `lib/roam.fish` from the Fish framework. The pipeline: ```mermaid flowchart LR in[agent stdout] --> sanitize["sanitize
(strip fences,
normalize indent)"] sanitize --> parse["parse hierarchy
(L1/L2)"] parse --> resolve["resolve page_uid
(today / named / namespaced)"] resolve --> replace["replace_block
(idempotent delete)"] replace --> write["write_tree
(root + N sections)"] ``` Each step calls the `mcp` CLI under `mcp roam ` — `get_daily_note`, `get_page`, `create_page`, `create_block_with_children`, `delete_block`. ## When to use * Your agent generates **hierarchical markdown** (not arbitrary prose) destined for Roam. * You want re-runs to be idempotent — yesterday's block is replaced, not appended to. * You use Roam as the human-facing dashboard and want this agent's output in a specific page/daily note. ## Config schema | Field | Type | Required | Default | Description | | -------------- | ------ | -------- | ------------------ | ----------------------------------------------------------------------- | | `page` | string | **yes** | — | Target page reference. `today` / `April 22nd, 2026` / `acme/tech/aws/X` | | `marker_regex` | string | no | (matches nothing) | Regex used to find and delete an existing block (idempotency) | | `mcp_binary` | string | no | `~/.cargo/bin/mcp` | Override the `mcp` CLI path | Verify schema at runtime: ```bash dotagent-plugin-sink-roam info | jq .schema ``` ### `page` reference forms | Form | Resolves to | | ----------------------- | -------------------------------------------------------------- | | `today` | The current daily note (computed by the Roam MCP server) | | `April 22nd, 2026` | A specific daily note (Roam native ordinal format — note `nd`) | | `acme/tech/infra/aws/X` | A namespaced page (created if missing) | > Reminder on Roam's date format: ordinal suffixes are mandatory — `1st`, `2nd`, `3rd`, `21st`, etc. Use `[[April 22nd, 2026]]` style in `page`, not ISO `2026-04-22` (which creates an orphan page outside the daily-note graph). ### Content hierarchy expected on stdin The plugin's `parse_hierarchy` enforces this convention (mirrors the Fish framework): ``` #TAG First line is the ROOT (no indentation). L1 header — direct child of root (indent ≤ 3 spaces) L2 child of the previous L1 (indent > 3 spaces) Another L2 of the same L1 Another L1 header L2 of that L1 ``` Code fences (` ``` `) are stripped. Everything after a `---` separator is cut. Leading `-` per line is removed (Roam adds bullets itself). ## Examples ### DORA standup in today's daily note ```toml [agent] name = "team-standup" [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#DORA.*dia-anterior" } ``` Agent's stdout must look like: ``` #DORA dia-anterior 2026-05-19 📋 TL;DR 24 PRs merged today Sentry: 3 new issues, none P0 🎯 Por pessoa [[@ana]]: 4 PRs, 1 release [[@bruno]]: 6 PRs, 2 reviews --- Notes after the separator are cut. ``` The marker regex `#DORA.*dia-anterior` matches the root block; on the next run the plugin deletes it and re-writes — no duplicates. ### Post draft in today's daily, replacing yesterday's ```toml [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#post-draft.*\\d{4}-\\d{2}-\\d{2}" } ``` ### Namespaced page (created on first run) ```toml [[on_success]] plugin = "sink-roam" config = { page = "acme/tech/infra/aws/finops-2026", marker_regex = "#weekly-report" } ``` ### Custom mcp binary path ```toml [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#summary", mcp_binary = "/opt/dev/mcp-staging" } ``` ## Response shape ### Success ```json { "ok": true, "root_uid": "abc123XYZ" } ``` `root_uid` is the UID of the new root block — useful for chasing the result manually with `mcp roam get_page '{"uid":"abc123XYZ"}'`. ### Validation failed ```json { "ok": false, "error": "page is required" } ``` ### Runtime failure ```json { "ok": false, "error": "" } ``` Possible causes: * `mcp` CLI not found (set `mcp_binary`) * Roam MCP server misconfigured (`mcp roam get_daily_note '{}'` fails) * Stdin was empty / unparseable * Page reference couldn't be resolved or created ## Behavior details ### Idempotency When `marker_regex` matches an existing child of the target page, the plugin deletes that child *before* writing the new tree. If no match, the plugin writes a new root regardless — running an un-markered config twice WILL produce two roots. Always set `marker_regex` for production. ### Sanitization The plugin tolerates Claude's habits: * Strips opening/closing ` ``` ` code fences. * Cuts at the first `---` line — Claude often adds notes / metadata after a separator. Those go away. * Removes `-` literals at the start of lines (Roam auto-renders bullets; the literal would produce `- - foo`). * Normalizes whitespace. ### Hierarchy threshold Indent ≤ 3 spaces → L1. Indent > 3 spaces → L2. This forgives small indentation variations from the LLM. Orphan L2 lines (no preceding L1) are silently dropped — matches the legacy Fish parser. If you see content disappear, your output probably has L2 indentation without an L1 header above it. ### Resolution order for `page` 1. If `page == "today"` → call `mcp roam get_daily_note '{}'` and use `.":block/uid"`. 2. Otherwise, try `mcp roam get_page '{"title":""}'`. If it returns a UID, use that. 3. Otherwise, `mcp roam create_page '{"title":""}'` and use the new UID. This means namespaced pages are auto-created (matching the legacy behavior). Named dailies (`April 22nd, 2026`) are NOT auto-created if Roam refuses — it usually finds them. ## External dependencies * **`mcp` CLI** (in Rust, separate project): * **Roam server configured** under the mcp client config (`~/.config/mcp/`). Test with: ```bash mcp roam get_daily_note '{}' | head ``` You should see JSON containing `:block/uid` for today's daily note. ## Manual testing ```bash # 1) Info dotagent-plugin-sink-roam info | jq . # 2) Validate echo '{"page":"today"}' | dotagent-plugin-sink-roam validate # 3) Real invoke echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "#TEST root from sink-roam\n L1 header\n L2 child", "config": {"page":"today","marker_regex":"#TEST"} }' | dotagent-plugin-sink-roam invoke ``` After this runs, open Roam — today's daily should have a `#TEST root from sink-roam` block with `L1 header` and `L2 child` nested. Run the same invoke again: the previous block is deleted and replaced (one block, not two). ## Troubleshooting ### `mcp roam get_daily_note '{}'` fails The dotagent plugin can't work without the `mcp` CLI's roam server. Fix the upstream config: ```bash ls ~/.config/mcp/ # roam.json should be here mcp roam get_daily_note '{}' 2>&1 | head ``` ### "no JSON in mcp output" The mcp CLI is printing tracing INFO lines before the JSON envelope. The plugin already filters from the first `{` or `[`, but extremely verbose log levels can corrupt the output. Lower verbosity: ```bash RUST_LOG=warn mcp roam get_daily_note '{}' ``` Then in dotagent's environment ensure `RUST_LOG` is at most `info`. ### Block keeps duplicating after re-runs Your `marker_regex` doesn't match the root block's text. Test the regex against your actual output: ```bash # What's currently in the daily? mcp roam get_daily_note '{}' \ | jq -r '.content[0].text' \ | jq -r '.":block/children"[]? | .":block/string"' ``` The regex is matched via Rust's `regex` crate (PCRE-ish syntax). Anchor with `^`/`$` if needed, escape `.`/`+`/`*`. ### Orphan L2 lines disappear Make sure every L2 line has an L1 line above it. The parser drops L2 without context. ### Content was cut at `---` That's intentional. Either remove the separator from your prompt's output, or move the kept content above it. ## See also * [Concept guide](/concepts/plugins) * [`sink-file`](/built-in-plugins/sink/sink-file) — for non-hierarchical / flat output * [Roam date format guidance](https://github.com/avelino/dotagent/tree/main/CLAUDE.md) — native ordinal format is mandatory for daily-note resolution * Source: [`plugins/sink-roam/`](https://github.com/avelino/dotagent/tree/main/plugins/sink-roam/README.md) * Legacy reference: `lib/roam.fish` in the avelino/dotfiles repo # sink-outl > Publish an agent's stdout to [Outl](https://github.com/avelino/outl) as a hierarchical block tree (root + L1 sections + L2 children). Idempotent: re-running a schedule replaces the previous block via a single batched call instead of duplicating. | Property | Value | | ------------- | ---------------------------------------------- | | Kind | `sink` | | Platforms | `darwin`, `linux` | | Binary | `dotagent-plugin-sink-outl` | | External deps | `mcp` CLI with `outl` server configured | | Network | Whatever the `mcp outl` server hits internally | ## What it does Twin of [`sink-roam`](/built-in-plugins/sink/sink-roam). Same config schema (`page`, `marker_regex`, `mcp_binary`) so a single agent can ship the same payload to both backends by declaring two `[[on_success]]` blocks. The pipeline: ```mermaid flowchart LR in[agent stdout] --> sanitize["sanitize
(strip fences,
normalize indent)"] sanitize --> parse["parse hierarchy
(L1/L2)"] parse --> resolve["resolve target
(daily / page / namespaced)"] resolve --> batch["outl_batch:
N delete + 1 append_tree"] ``` Unlike `sink-roam` (which makes several round-trips to Roam's API), Outl exposes `outl_batch` — a sequence of ops applied in one call. The plugin composes one batch with: 1. `block_delete` for every existing block matching `marker_regex` (idempotency). 2. One `block_append_tree` writing the new root + sections in a single call. Each MCP call is `mcp outl ''`. ## When to use * Your agent generates **hierarchical markdown** (not arbitrary prose) destined for Outl. * You want re-runs to be idempotent — yesterday's block is replaced, not appended to — and atomic (delete + write happens server-side in one batch). * You use Outl as the human-facing dashboard or want to mirror an agent output to Outl alongside Roam. ## Config schema | Field | Type | Required | Default | Description | | -------------- | ------ | -------- | ------------------ | ------------------------------------------------------------------------------------ | | `page` | string | **yes** | — | Target page reference. `today` / `April 22nd, 2026` / `2026-06-01` / `acme/tech/aws` | | `marker_regex` | string | no | (matches nothing) | Regex used to find and delete an existing block (idempotency) | | `mcp_binary` | string | no | `~/.cargo/bin/mcp` | Override the `mcp` CLI path | Verify the schema at runtime: ```bash dotagent-plugin-sink-outl info | jq .schema ``` ### `page` reference forms | Form | Resolves to | | ------------------------ | ------------------------------------------------------------------- | | `today` | Local-clock daily journal (ISO slug, e.g. `2026-06-01`) | | `yesterday` / `tomorrow` | Daily journal offset by one day from local clock | | `2026-06-01` | A specific daily journal (ISO `YYYY-MM-DD`) | | `April 22nd, 2026` | A specific daily journal (Roam-native ordinal — note `nd`) | | `acme/tech/infra/aws/X` | A namespaced page (lowercased, auto-created via `outl_page_create`) | > Unlike Roam, Outl's daily slug is **ISO** (`2026-06-01`). The plugin accepts the Roam-native ordinal form too and normalizes it to ISO before talking to Outl — so the same `agent.toml` works against both backends. ### Content hierarchy expected on stdin The parser is shared with `sink-roam`: ``` #TAG First line is the ROOT (no indentation). L1 header — direct child of root (indent ≤ 3 spaces) L2 child of the previous L1 (indent > 3 spaces) Another L2 of the same L1 Another L1 header L2 of that L1 ``` Code fences (` ``` `) are stripped. Everything after a `---` separator is cut. Leading `-` per line is removed (Outl renders bullets itself). ## Examples ### Mirror an agent to both Roam and Outl ```toml [agent] name = "team-standup" # Roam (legacy) [[on_success]] plugin = "sink-roam" config = { page = "today", marker_regex = "#DORA.*dia-anterior" } # Outl (new) — identical config shape, just swap the plugin name [[on_success]] plugin = "sink-outl" config = { page = "today", marker_regex = "#DORA.*dia-anterior" } ``` The same stdout payload reaches both backends; idempotency holds on each side independently. ### Namespaced page (created on first run) ```toml [[on_success]] plugin = "sink-outl" config = { page = "acme/tech/infra/aws/finops-2026", marker_regex = "#weekly-report" } ``` ### Specific past daily ```toml [[on_success]] plugin = "sink-outl" config = { page = "2026-05-27", marker_regex = "#postmortem" } ``` ### Custom mcp binary path ```toml [[on_success]] plugin = "sink-outl" config = { page = "today", marker_regex = "#summary", mcp_binary = "/opt/dev/mcp-staging" } ``` ## Response shape ### Success ```json { "ok": true, "root_id": "abc123XYZ" } ``` `root_id` is the id of the newly appended root block — useful for chasing the result manually with `mcp outl outl_block_get '{"id":"abc123XYZ"}'`. ### Validation failed ```json { "ok": false, "error": "page is required" } ``` ### Runtime failure ```json { "ok": false, "error": "" } ``` Possible causes: * `mcp` CLI not found (set `mcp_binary`) * Outl MCP server misconfigured (`mcp outl outl_daily_today '{}'` fails) * Stdin was empty / unparseable * `outl_batch` returned an `error` envelope (the plugin propagates it verbatim along with `failed_at`) ## Behavior details ### Single round-trip via `outl_batch` The whole publish is **one** `mcp outl outl_batch '{"ops":[...]}'` call. The ops are emitted in order: 1. N × `block_delete` (one per match found by `marker_regex`). 2. 1 × `block_append_tree` (the new root + sections). If any op fails server-side, the batch stops; the response carries `error` + `failed_at` and the plugin returns `ok=false` with that detail. Earlier `block_delete` ops that already ran are **not** rolled back — Outl batches are best-effort sequential. ### Idempotency When `marker_regex` matches existing children of the target page, the plugin deletes them *before* writing the new tree. If no match, the plugin writes a new root regardless — running an un-markered config twice WILL produce two roots. **Always set `marker_regex` for production.** The marker search uses `outl_search` with a cheap FTS probe (first contiguous run of `#`/alphanumeric/`_`/`-` from the pattern), then re-filters client-side with the full regex. ### Sanitization Same convention as `sink-roam`: * Strips opening/closing ` ``` ` code fences. * Cuts at the first `---` line. * Removes `-` literals at the start of lines. * Normalizes whitespace. ### Hierarchy threshold Indent ≤ 3 spaces → L1. Indent > 3 spaces → L2. Orphan L2 lines (no preceding L1) are silently dropped. ### Resolution order for `page` 1. `today` / `yesterday` / `tomorrow` → ISO slug from local clock. 2. ISO `YYYY-MM-DD` → daily target as-is. 3. Roam-native ordinal `Month , YYYY` → converted to ISO. 4. Everything else → namespaced page (lowercased, `/` preserved). The plugin issues an `outl_page_create` first (idempotent in Outl). ## External dependencies * **`mcp` CLI** (in Rust, separate project): * **Outl server configured** under the mcp client config (`~/.config/mcp/`). Test with: ```bash mcp outl outl_daily_today '{}' | head ``` You should see JSON containing today's daily journal id. ## Manual testing ```bash # 1) Info dotagent-plugin-sink-outl info | jq . # 2) Validate echo '{"page":"today"}' | dotagent-plugin-sink-outl validate # 3) Real invoke echo '{ "kind": "sink", "agent": "test", "schedule": "test", "event": "success", "message": "#TEST root from sink-outl\n L1 header\n L2 child", "config": {"page":"today","marker_regex":"#TEST"} }' | dotagent-plugin-sink-outl invoke ``` Run the same invoke twice — the previous block is deleted and replaced in the same batch, so you end with one block, not two. ## Troubleshooting ### `mcp outl outl_daily_today '{}'` fails Fix the upstream config: ```bash ls ~/.config/mcp/ # outl.json should be here mcp outl outl_daily_today '{}' 2>&1 | head ``` ### "no JSON in mcp output" The mcp CLI is printing tracing INFO lines before the JSON envelope. The plugin already filters from the first `{` or `[`, but extremely verbose log levels can corrupt the output. Lower verbosity: ```bash RUST_LOG=warn mcp outl outl_daily_today '{}' ``` ### `outl_batch failed at op N` A `block_delete` or `block_append_tree` op was rejected by the server. `failed_at` is the 0-indexed position in the `ops` list. The first `block_append_tree` is always last; numbers below it are deletes. Common causes: stale block id (someone else deleted it between search and batch), or the target page slug doesn't exist and the `outl_page_create` step failed silently. ### Block keeps duplicating after re-runs Your `marker_regex` doesn't match the root block's text. The plugin uses Rust's `regex` crate (PCRE-ish syntax). Anchor with `^`/`$` if needed, escape `.`/`+`/`*`. Inspect what's there: ```bash mcp outl outl_search '{"query":"#TAG","in":"blocks","limit":10}' ``` ### Orphan L2 lines disappear Every L2 line needs an L1 line above it. ### Content was cut at `---` Intentional. Either remove the separator from your prompt's output, or move the kept content above it. ## See also * [Concept guide](/concepts/plugins) * [`sink-roam`](/built-in-plugins/sink/sink-roam) — twin plugin for Roam Research * [`sink-file`](/built-in-plugins/sink/sink-file) — flat output to a single file * Source: [`plugins/sink-outl/`](https://github.com/avelino/dotagent/blob/main/plugins/sink-outl/README.md) * Upstream: [outl](https://github.com/avelino/outl), [mcp](https://github.com/avelino/mcp) # Threat model > What we defend against, what we don't, and why. This document captures the security posture of dotagent — the assumptions, the attack surface, the defenses we ship, and the defenses we explicitly chose to defer. Read it before proposing security features; the goal is detection and auditability, not impossible-to-break sandboxing. ## Premise dotagent is a daemon running as the user that spawns arbitrary external commands declared in `agent.toml` files on disk. Any attacker with the same user privileges that the daemon has — write access to `~/.config`, `~/Library/LaunchAgents`, the daemon binary path — can already do what dotagent does, bypassing it entirely. **We do not try to defend against a local attacker with user-equivalent capability**. Doing so would require a privilege boundary dotagent cannot provide (TPM / Secure Enclave / hardware key, kernel-level mandatory access control, etc.) and would degrade the developer-experience that justifies dotagent's existence. What we DO target: 1. **Detectability**: every consequential action emits an event in an append-only, hash-chained audit log. 2. **Out-of-band notification**: critical events fire a notification plugin (iMessage / Pushover / etc.) so the user sees attacks from a different device. 3. **Documented intent**: manifests declare a `[security]` section so even if enforcement is partial, the *intent* is captured and reviewable. ## Threat vectors ### V1 — Manifest hijack An attacker rewrites `~/.config/dotagent/agents/innocent/agent.toml` so that `run.command` exfiltrates data the next time the agent fires. **Mitigations:** * **Manifest drift detection**: daemon caches sha256 of every loaded manifest in `~/.config/dotagent/state/known_manifests.json`. On the next load, compares hashes. Drift → audit event `manifest_drift_detected` → notify. * **Audit log**: every `agent_run` records the manifest hash used. Forensic reconstruction of "what was the agent at time T". * **Not mitigated**: if the attacker edits both `agent.toml` and the cache file before the next daemon tick, the change is invisible to dotagent. Sophisticated attacker → eventually detected only via out-of-band review. ### V2 — Phantom agent An attacker drops a new directory `~/.config/dotagent/agents/spyware/` with its own `agent.toml`. The daemon discovers it on the next tick and runs it. **Mitigations:** * **First-seen detection**: any new agent name not present in `known_manifests.json` triggers `phantom_agent_detected` → notify. The agent **still runs by default** because we chose not to require explicit registration (would block legitimate `git pull` workflows). * **Not mitigated**: we don't gate execution on prior approval. This is a deliberate UX trade-off — the user gets a notification immediately, but the attacker gets one execution before the user can react. > **Rationale for not implementing a registry**: a registry only works if the attacker respects it. An attacker with rwx on `~/.config` writes the registry along with the manifest. Defense is theatre. ### V3 — Daemon binary swap An attacker replaces `/usr/local/bin/dotagent` with a trojan binary (same CLI, additional behavior). **Mitigations (planned, not in v0):** * Self-hash check on daemon startup against `~/.config/dotagent/daemon.sha256`. Mismatch → notify, daemon refuses to start. * Optional macOS code signing: notarized binary, gatekeeper verifies. ### V4 — Plugin swap An attacker replaces a plugin binary (`dotagent-plugin-sink-roam`) with one that exfiltrates payloads. **Mitigations (planned, not in v0):** * Plugin hash registry analogous to manifest tracking. * `doctor` reports plugin resolution path; user can spot oddities. ### V5 — Input flowing into spawn An attacker controls input that ends up in `Command::new(...).arg(...)` without sanitization. **Mitigations:** * `RunConfig::args` is treated as token list, never passed through a shell. We do not `Command::new("sh").arg("-c").arg(...)` user-controlled strings. * `EnvConfig::extra` keys/values are typed `String`, not interpreted. * Plugin invocation uses subprocess with JSON stdin — plugin output is parsed as JSON, not eval'd. ### V6 — Secrets leak via stdout A plugin logs its config (containing secrets) to stdout, which dotagent captures in audit log entries. **Mitigations (partial):** * Convention: plugins MUST NOT log config to stdout. Only the JSON response goes to stdout; human logs go to stderr. * Audit log policy: stderr tails (5 lines) appear in `attempt_failed` / `given_up` events. Plugins are responsible for not putting secrets in stderr either. * **Not mitigated**: a misbehaving plugin can leak via the response JSON. Audit log will contain it. User is responsible for plugin trust. ### V7 — Resource exhaustion / fork bomb A malicious manifest sets `max_retries = 1000000` with `retry_backoff_minutes = [0]`. **Mitigations (planned, not in v0):** * Hard clamp `max_retries <= 32` in the runtime. * Hard clamp `retry_backoff_minutes[i] >= 1` in the runtime. * Concurrent run cap (default 10 daemon-wide, 1 per agent). ### V8 — Untrusted inbound message causes a local run **This vector is different in kind from V1–V7, and it is worth saying so plainly.** Every other threat here assumes an attacker who already has user-equivalent access to the machine. Inbound Telegram ([`concepts/telegram.md`](/concepts/telegram)) is the first path where someone with **no access at all** can cause a local process to run. The premise above still holds for everything else; this section is the exception carved out of it. The exposure exists only when `[telegram]` is configured. It is off by default, and dotagent opens no inbound path you did not ask for. **What an attacker needs:** the bot token (leaked from `secrets.env`, a backup, a screenshot) or the ability to message a bot they discovered. Telegram bots are enumerable, so assume discovery is free. **Mitigations:** * **Numeric allowlist.** `allowed_user_ids` gates every message. `@username` is changeable and therefore never used for authorization. * **Open chats are operator-listed, not discovered.** `open_chat_ids` lets any member of a *listed* group talk to the dispatcher. The boundary is the operator's choice of group: anyone can add the bot to a group they control, and an unlisted group changes nothing. Typed commands (`!`/`!!`) and direct messages remain `allowed_user_ids`-only even in an open chat, so an open chat grants conversation, never binary execution. * **Empty means nobody.** A token configured with an empty allowlist leaves the ingress off and logs why. Reading empty as "no restriction" would turn one forgotten line into an open remote-execution endpoint. * **The message never supplies a command.** It selects among agents already installed on disk. An agent name that does not resolve is refused, and nothing from a trigger reaches a shell — the message body travels in `AGENT_TRIGGER_PAYLOAD`, not argv. * **Trigger env cannot shadow runner env.** Per-invocation variables are applied before the `AGENT_*` block, so a payload cannot redefine `AGENT_NAME` or `AGENT_HEARTBEAT_FILE`. * **Rate limit per sender**, default 10/minute, so one sender cannot occupy the daemon indefinitely. * **Audit.** Accepted messages emit `trigger_received` then `agent_triggered`; refusals emit `trigger_rejected` at `Critical`, which fires out-of-band notification. An unlisted sender means somebody found your bot, and you should hear about it from a different device. * **No message text in the audit log.** It records the sender id and chat id. Bodies are content, not attribution, and a chat can contain anything you pasted into it. **Not mitigated:** * An allowlisted account that is itself compromised (stolen Telegram session) can run any installed agent. The allowlist authenticates an account, not a person. * A dispatcher agent that ignores the catalog and interprets message text itself re-opens everything this section closes. The [`telegram-assistant` example](https://github.com/avelino/dotagent/tree/main/examples/telegram-assistant/README.md) routes only through MCP tools for exactly this reason. ### V9 — MCP client reaches the agent catalog `dotagent mcp` ([`reference/mcp.md`](/reference/mcp)) exposes every installed agent as a callable tool. **Mitigations:** * stdio transport only. No port, no listener, nothing reachable from another machine. The server inherits the trust of whatever spawned it, and a local MCP client already runs as you. * The catalog is the boundary: an agent that is not installed cannot be run. * Arguments are a token list, never a shell string — same rule as V5. **Not mitigated:** any local process that can spawn `dotagent mcp` can run any installed agent. That is user-equivalent capability, which the premise above already places outside the boundary. ### V10 — Skill script execution A skill ([`concepts/skills.md`](/concepts/skills)) may package executables under `scripts/`, and `skill-run` executes one on request from an MCP client. Two things are new. Code runs **outside any manifest**, so nothing in `agent.toml` describes or bounds it. And the search path includes `~/.claude/skills/`, a directory that exists for another tool and that the operator did not create for dotagent. **Mitigations:** * **Path containment.** Absolute paths and `..` are refused before touching the disk; the canonicalized target must still sit under the canonicalized skill directory, which is what catches a symlink pointing outside. * **`scripts/` only, executable bit required.** A `references/` document is not runnable, and dropping a helper beside an entry point does not silently make it callable — the author names entry points by chmod. * **Supervised.** Every execution goes through `dotagent-supervisor`: enforced deadline (`timeout_seconds`, default 300) and kill-tree on expiry, so a script that spawns children cannot leave orphans — and the guarantee now survives the daemon itself. A daemon killed mid-run used to leave its children with nobody holding their deadline; the next boot re-reads the supervisor snapshot and kill-trees what it can prove was left behind. See [Boot orphan reap](/guides/daemon-lifecycle#boot-orphan-reap). The reap is **deliberately incomplete**, and that is the security position. Killing the wrong process is a worse outcome than an orphan surviving, so a record is only signalled when group leadership, start time and command line all still match what was recorded. A recycled pid fails at least one of them and is refused. Unreadable snapshot, missing identity fields, another daemon alive: all abort without signalling. Bounding an escaped script is a best-effort mitigation here, never a licence to signal a pid read off disk. * **Arguments are a token list**, never a shell string — same rule as V5. * **Audited.** Each run appends `skill_invoked` (skill, script, exit code, whether it timed out). Without it, "what ran on this machine" would have a hole exactly where a manifest cannot answer. **Not mitigated:** anyone who can write to a skill directory can put a script there and have it run. That is the same statement V1 makes about manifests, and it now applies to `~/.claude/skills/` too — a directory whose contents may have been installed for a different tool with different expectations. Narrow the search path if that matters: ```toml [skills] claude_skills = false paths = ["/Users/me/.config/dotagent/skills"] ``` Turning skills off entirely (`[skills] enabled = false`) removes both the tools and the execution path. ### V11 — Commands A command ([`concepts/commands.md`](/concepts/commands)) is a prompt with no `scripts/` equivalent. Installing one grants **no execution**: everything it can cause is already reachable through the tools the dispatcher holds. That is the whole difference from V10, and it is why this entry is short. What remains is **influence** — a command file is text a model is told to follow, so anyone who can write the directory can steer an assistant that has real tools. Two smaller edges come from Telegram itself. **Mitigations:** * **No new execution vector.** The daemon resolves nothing; `command-get` returns text. Arguments are substituted textually and never reach a shell. * **`~/.claude/commands/` is off by default**, the reverse of `claude_skills`. A directory maintained for another tool does not become a published menu because dotagent happened to find it. * **Per-chat menu scope.** `setMyCommands` is registered against each allowlisted chat rather than `BotCommandScopeDefault`, so command names and descriptions are not published to anyone who finds the bot. The allowlist already gates execution; this keeps the *catalog* from leaking too. * **Screening order.** The allowlist and rate limit are checked before any command is parsed or answered, so an unlisted sender cannot enumerate the catalog with `/help`, and a sender looping `/typo` spends the same budget as one looping prose. * **Audited by name only.** `command_dispatched` records the command and whether it was `known`; arguments are content, the same reason `trigger_received` never records a message body. Repeated `known: false` from one sender is what probing looks like. **Not mitigated:** a command file is trusted input to the model, exactly as a skill body is. Write access to the commands directory is write access to what the assistant is told. `[commands] enabled = false` removes the menu and both tools. ### V12 — Remediation from a chat message A `[[preflight]] remediation` ([`concepts/agents.md`](/concepts/agents)) is a command an assistant can run, and the request to run it arrives over Telegram. That is V8's shape with a sharper edge: the point is to change the machine, not to read from it. The rejected design is worth naming, because it is the obvious one. A preflight plugin already returns `suggest` — the command that would clear the check — and letting an assistant run that string is one line of code. It is also arbitrary execution: the string is written by a plugin, and any plugin could then have anything run by asking an assistant nicely. **Mitigations:** * **Declared, not suggested.** The command comes from the operator's manifest. `suggest` is never executable; it stays a message. * **The catalog is the boundary**, as with agents: `tools/list` publishes one entry per declared remediation, a name that is not there is not callable, and the tool takes no arguments — so the model chooses *which*, never *what*. * **argv, not a shell.** `remediation = "x && curl … | sh"` runs a program named `x` with those literal arguments and fails. There is no shell to interpret the operators. * **Supervised**, with a 120-second deadline and kill-tree on expiry. * **Audited** as `remediation_invoked` at `Critical`, recording the command as declared. It is the one event where a chat message changed the machine. * **It does not re-run the agent.** Clearing the check and dispatching a run stay two decisions. **Not mitigated:** an operator who declares a dangerous command has declared it. This moves the trust to the manifest, where V1 already puts it — someone who can write your `agent.toml` can already set `run.command`. ### V13 — State carried between senders by a persistent agent `[lifecycle] mode = "persistent"` ([`concepts/lifecycle.md`](/concepts/lifecycle)) keeps an agent process alive between runs. That process holds whatever it holds — a conversation, a cached credential, a decision someone authorized — and every later request lands in the same memory. The failure is not a bug in the pool. It is a manifest that declares `mode = "persistent"` without a `key` on an agent that serves more than one person: every conversation shares one process, so what one sender said is context for the next one's answer. Nothing about it looks wrong until two people use the bot, which is exactly the shape of a leak that ships. **Mitigations:** * **`key` shards by an attested field.** `key = "chat_id"` gives one process per conversation. The value is a *selector over the payload the daemon already attested* — the sender names nothing. * **The key never reaches a label raw.** It is reduced to `[A-Za-z0-9_-]`, and anything else — path separators, whitespace, 4 KB of chat text — becomes a stable digest instead. Two distinct values never collapse to one process, so sanitizing cannot merge two conversations. * **`doctor` warns** when the Telegram `dispatcher_agent` is persistent with no key, because that is the configuration where this actually bites. * **`max_invocations` bounds how long any one process accumulates**, and `max_instances` bounds how many exist. Both default to finite values. * **Recycling is audited** as `persistent_agent_recycled` with the reason, so "why did it forget" has an answer. * **One pool, inside the daemon.** There is no second instance to race it, which is what an external pool needs `flock` for. **Not mitigated:** an agent that writes its own state to disk, keyed however it likes. dotagent isolates the *process*, not the filesystem — `[security] filesystem_writable` is still schema-only (V-deferred, below). ### V16 — Assistant harness: registry and memory as new state `[assistant]` moves conversation bookkeeping into the daemon, which makes two new files interesting to an attacker with user-level access: **The registry** (`state/assistant/-.json`, 0600, atomic writes) holds pointers only — model session id, generation, toolkit hash, transcript size. It deliberately has no field that can hold chat text, so reading it yields "which session served chat X", never what was said. The deeper risk is *writing* it: a poisoned pointer makes the next reply resume an attacker-chosen model session. That is a same-user attack surface the premise already accepts (the attacker could edit the manifest and get arbitrary execution directly); the registry adds no privilege. **The memory workspace** (`outl/`) accumulates `MEMO:` facts — flushed from assistant replies, and from the stdout of any agent whose manifest declares `[memory]`. Trust boundary: the facts are *agent-produced text*, not attested input — a compromised or prompt-injected agent can poison its own memory with false facts that later color future replies. It cannot write another conversation's registry, and `memory-recall` output is bounded (2 KiB per turn), but "the assistant believes wrong things" is a real failure mode. Mitigations: the workspace is a plain outl graph the user can open, read and edit (a bad memory is visible and correctable by hand); every fact records which agent, source and session wrote it as block properties, so one poisoned run's output can be found and removed as a group; `dotagent memory forget` and `memory-supersede` make that correction a command rather than a hand-edit; and recall failure degrades to an empty block, never to a blocked conversation. Capture from a plain agent is **opt-in per manifest and successful runs only**. Both bounds are about trust rather than tidiness: an agent that never declared `[memory]` cannot write facts at all, and a failing run — the state an attacker is most likely to be able to force — cannot file its output as durable truth. **Not mitigated:** prompt injection through message content that ends up in a `MEMO:` line verbatim. The memory stores what the agent claimed. Recorded provenance narrows *which* run to distrust, and editing or forgetting the fact is the correction path — but nothing validates a claim at write time. ### V14 — Notifier credentials written to the daemon's own log V6 is about a *plugin* leaking secrets outward. This is the inverse, and it does not need a misbehaving anything: dotagent leaking its own notifier credentials into a file it writes itself. `reqwest::Error`'s `Display` appends `for url (…)`. For two built-in drivers the URL **is** the credential — `hooks.slack.com/services/T…/B…/` and `api.telegram.org/bot/sendMessage`. Any `?` that converted such an error into a `NotifyError` reached `warn!(driver, error = %e, "notifier failed")` and landed in `~/.config/dotagent/logs/daemon/dotagent.log.*` — live credentials, in a file with a 30-day retention window, from a failure as mundane as a DNS blip. Nothing was compromised to cause it; the notifier just had a bad day. **Mitigations:** * **The leak is unrepresentable, not merely unwritten.** `NotifyError` has no `From` — the variant was **removed from the enum**, so `?` on a `reqwest` call does not compile. A driver has to route transport failures through `redact::sanitize_reqwest_err`, which keeps only the failure kind and the HTTP status (`slack transport error (timeout, status 503)`). Deleting the conversion is what stops the next driver from reintroducing this by accident; fixing the log line would not have. * **API error bodies are scrubbed before logging.** Telegram's own error text happily echoes the request URL back, so the token is blanked in the description — both as a whole string and as its trailing high-entropy segment, since a truncated echo can carry the half that matters. * **ntfy `base_url` userinfo is redacted**, because a self-hosted URL may legitimately embed HTTP basic credentials. * **Credentials do not have to be in the manifest at all.** Every credential-bearing field takes `${VAR}`, resolved at send time from `secrets.env` — see [`concepts/secrets.md`](/concepts/secrets). The expansion error names the field and the variable only; it never echoes a resolved value or the literal input, because the input is a credential template and the store holds sibling credentials. **Not mitigated:** an operator who writes a literal credential into `agent.toml` and commits it. dotagent cannot un-publish a git history. The `${VAR}` path exists so this never has to happen; `doctor` does not currently flag literals that look like credentials. ### V17 — Installed binaries reachable from a chat message `[os]` lets an assistant run binaries that are already on the machine. It is the widest inbound path in this document, and the only one where the operator declares what *may* run while a model chooses the arguments. Everything else here either fixes the whole command in a manifest (V12) or ships the code being run inside the repo (V10). The honest framing: an allowlist of whole binaries is not a sandbox. `git log` cannot write to the repo, but `git` with the right flags can run a pager, and a binary that takes a `--exec` style flag will do what that flag says. What the allowlist buys is that the set of programs is finite, declared, and auditable — not that every invocation inside it is harmless. **Mitigations:** * **Off by default, and empty by default when on.** `enabled = true` with no `allow` list runs nothing. Reaching "on" by accident is a state that should do nothing rather than everything. * **The catalog is the boundary**, as everywhere else: with `[os]` off, the `os-run` and `os-list` tools are absent from `tools/list`, so a client sees no such capability rather than one that refuses. * **Whole-token matching.** `kubectl get` admits `kubectl get pods` and refuses `kubectl delete` and `kubectl getsecrets`. Granularity is per entry, so a binary that only reads can be listed bare and one that can change production can be pinned to its safe subcommands. * **A name, never a path.** `/bin/sh`, `./sh` and anything carrying `..` are refused before the list is consulted, so a path cannot impersonate a listed name. Resolution goes through `PATH`. * **argv, not a shell.** Arguments reach the program as a token list. `|`, `&&` and `$(…)` in an argument are literal characters, the same rule as V5. * **Supervised**, with the configured deadline and kill-tree on expiry. * **Audited** as `os_command_invoked` at `Critical`, recording the binary and **the full argument list** — the half that no manifest declared. **The `!` prefix.** A message starting with `!` is run directly: `!rg foo` spawns `rg` with one argument and never reaches the dispatcher. It is the same policy through a different door, and it is *narrower* than the model's door in the way that matters — a typed line cannot be steered by content, so the prompt-injection surface that makes `os-run` risky does not exist here. * **Screened first.** The prefix is read after the Telegram allowlist and the rate limit, never before. Reading it earlier would make `!` a way past both. * **The same `[os] allow` list.** A typed line is not trusted further than a chosen one, because the allowlist authenticates an account and not a person (V8): a stolen session types `!` just as well. Without this, one compromised session would go from "runs what the operator allowed" to "unrestricted shell". * **No session, no model, no memory.** Nothing is stored and no reply is paraphrased. Errors come back raw, which is the point of typing it yourself. * **Not a shell.** Quotes group an argument; `|`, `&&`, `;` and `$(…)` are ordinary characters in an argument list. `!ls; rm x` asks for a binary literally named `ls;` and fails to find it. **`deny` and `confirm`.** `deny` refuses always and beats `allow`, `*` included — a list that could be widened past its own refusals would not be one. `confirm` runs only after a person answers `!!` in the same conversation, and it defaults to a non-empty list precisely because `allow = ["*"]` with no brake is the configuration people reach for first. Matching is per binary rather than per pattern. `confirm = ["rm"]` covers `rm -rf /`, `rm -r -f /`, `rm -fr /` and `rm --recursive --force /`; a textual `"rm -rf"` would cover the first and miss three. The shells are on the default list for the same reason: with `*`, `sh -c` reaches every binary a binary-name guard would otherwise catch. **A model may not confirm.** `os-run` refuses anything on the `confirm` list outright and tells the caller to have the person type it. The confirmation exists so a human decides; a tool that could both request and grant it would be theatre. This is the one asymmetry between the two doors, and it is the point of having two. Confirmations are held in memory, one slot per conversation, with a TTL. A restart forgets them, which fails closed: the cost is retyping a command, not a `!!` from yesterday landing on a machine whose operator has moved on. **`allow = ["*"]`** admits every binary on `PATH`. It is a supported configuration and it is the widest one: with a shell in reach, the mitigations above stop bounding *what* runs and bound only *who can ask* and *what gets recorded*. The inbound channel's own gate (Telegram's `allowed_user_ids`, the socket's uid check) becomes the whole boundary, and `doctor` says so on every run rather than letting it be configured once and forgotten. **Residual risk, stated plainly:** an allowlisted binary is trusted with whatever that binary can do. Listing `kubectl` bare on a machine with production credentials means a chat message can reach production. That is a choice the operator makes per entry, which is why the granularity exists and why the default is an empty list. ### V15 — Local Unix-socket API The daemon can expose a local client API at `$DOTAGENT_HOME/api.sock` when `telegram.dispatcher_agent` resolves to a discovered manifest. A local process can send messages to that socket and cause the configured dispatcher agent to run. This is intentionally a same-machine transport, not a network endpoint. **Mitigations:** * **Socket permissions.** The listener is created with mode `0600`, so another uid cannot open it through the normal Unix-socket permission check. * **Bind hygiene.** The daemon uses `lstat`-style metadata checks, refuses a non-socket file or symlink at the path, refuses a live listener, and removes a stale socket only when its uid is the current uid. A stale socket owned by another uid is left in place and reported as an error. * **Peer attribution.** Linux `SO_PEERCRED` and Darwin local peer credential APIs provide uid and, when available, pid. The daemon carries that identity into the trigger actor used for audit entries. If the kernel cannot provide it, the actor is honestly recorded as `local` rather than guessed. * **Bounded input and work.** The API caps connections at 16, requests at 30 per minute per connection, text at 32 KiB, session ids at 64 ASCII identifier characters, gateway conversations at 4, and each conversation queue at 64. A pending event queue is capped at 1 MiB per connection; a slow client is disconnected instead of pinning daemon memory. * **Audit without transcript capture.** Gateway admission, rejection, and agent execution are auditable, with actor/session attribution. Message text is not copied into the audit log because it is content, not identity. * **No public listener.** The current implementation has no HTTP or TCP mode. Mobile clients, remote clients, and token-based authentication are future transport decisions, not capabilities of this socket. **Trade-offs and not mitigated:** * `0600` is an OS-user boundary, not authentication. A process already running as the dotagent user can open the socket, submit arbitrary dispatcher input, and read the replies it requested. That is user-equivalent capability, which this threat model explicitly does not claim to prevent. * Rate limits and gateway caps reduce accidental floods and resource pressure; they do not distinguish a trusted process from another process running as the same user. `session_id` is an opaque routing key, not an access-control credential. * Local clients receive raw streamed stdout lines. A client with socket access can therefore see the output of the work it triggered; agents that persist transcripts or other sensitive state must enforce their own data boundary. * A daemon crash can leave `api.sock` behind. The next daemon applies ownership and liveness checks before removing it, rather than blindly unlinking a path. ## Defenses shipped in v0 (with the daemon engine) | Defense | Status | Scope | | ---------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Audit log (hash-chained, append-only) | ✅ v0 | All `agent_run`, `agent_failed`, `agent_recovered`, `manifest_*`, `plugin_*` events. Rotates at 32MB across a hash **seam**; segments are never deleted automatically. See [what the chain guarantees](#what-the-hash-chain-guarantees-and-what-it-does-not). | | Out-of-band notification on critical events | ✅ v0 | `given_up`, `phantom_agent_detected`, `manifest_drift_detected`, `audit_chain_broken` (which now also covers a log whose head was removed with no seam accounting for it) | | `[security]` schema in manifest | ✅ v0 schema-only | Parses + `doctor` warns on inconsistency. **Enforcement is post-v0** — see below. | | Manifest drift detection | ✅ v0 | sha256 cache + notify on mismatch | | Phantom agent detection | ✅ v0 | first-seen detection + notify | | Broken manifest does not hide healthy agents | ✅ | A failed parse is skipped and audited as `manifest_invalid` (Critical). Previously one bad file aborted the whole scan, leaving the daemon with zero agents and nothing but a log line. | | Trigger env cannot shadow runner env | ✅ | Per-invocation variables are applied before the `AGENT_*` block, so an untrusted payload cannot redefine `AGENT_NAME` or `AGENT_HEARTBEAT_FILE`. | | Notifier credentials cannot reach `tracing` | ✅ | `NotifyError` has no `From`, so a `?` that would log a webhook URL does not compile. Transport errors are reduced to kind + status; API error bodies are token-scrubbed. See [V14](#v14--notifier-credentials-written-to-the-daemons-own-log). | | `${VAR}` for every credential-bearing notifier field | ✅ | `slack.webhook_url`, `ntfy.token`/`base_url`/`topic`, `pushover.token`/`user`, `telegram.bot_token`. Resolved at send time from `secrets.env` (0600-enforced), env as fallback. Unresolved = failed send, never the literal placeholder. | | Local client API socket hygiene and backpressure | ✅ v0 | Unix socket `0600`, stale-path ownership checks, peer uid/pid attribution when available, connection/request/text/session/event-queue limits. This is not same-user authentication. See [V15](#v15--local-unix-socket-api). | | Alerts that repeat while a failure holds | ✅ | `stale` and `given_up` re-notify on a rising ladder (entry, 1h, 6h, daily) rather than once. A monitoring channel that goes quiet while the failure persists is indistinguishable from one where nothing is wrong. | ### The `[security]` gap, stated plainly `[security]` is **declared intent, not enforcement**. `allowed_commands`, `filesystem_writable`, `network` and `env_passthrough` are parsed, reported by `doctor`, and otherwise ignored by the runner. An agent that declares `allowed_commands = ["jq"]` and then runs `curl` is not stopped. That was a reasonable trade while every agent was code you wrote for yourself. It reads differently since **V8**: there is now a path where a message from the internet chooses *which* declared agent runs. The choice is still confined to what you installed — that part is real — but the blast radius of the agent it picks is bounded by nothing except what that agent's own code does. Concretely, `examples/telegram-assistant` declares `allowed_commands = ["bash", "jq", "claude", "dotagent", "perl"]`. Today that list is a comment. Treat it as documentation of what the author intended to need, not as a control that holds. Until enforcement lands, the honest mitigations are the ones outside dotagent: outbound firewall, disk encryption, and not installing an agent you have not read. ## What the hash chain guarantees (and what it does not) Worth stating plainly, because rotation made the question concrete and the answer is easy to overstate. **The chain detects partial edits. It has never detected a total rewrite.** Each line carries `prev_hash = sha256(previous line)`. Change one line in the middle and every hash downstream stops reproducing, so verification names the position. Delete a line and the same thing happens. That is the guarantee, and it is a real one: it turns "somebody edited the log" from invisible into loud. But an attacker with write access to `audit.log` can recompute the whole file — strip the entries they dislike, rewrite every `prev_hash` forward, and hand back a file that verifies from `GENESIS` to the end. Nothing in dotagent stops that, and nothing could without a key the attacker cannot reach (see *Defenses deferred*: TPM / Secure Enclave anchoring). Per the premise at the top of this document, an attacker with user-equivalent capability is out of scope, and this is one of the places that shows. ### Rotation preserves exactly that property — no more, no less Rotation renames the live file to `audit.log.` and starts a new one whose first line is a **seam** (`audit_log_rotated`) recording the segment name, its entry count, and its tail hash. The seam's own `prev_hash` equals that tail hash, so the chain crosses the rename with no gap. The seam is what makes retention legible: | On disk | Verification says | Why | | -------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | | all segments present | intact from `GENESIS` | every link checked | | old segment deleted, seam present | **intact since ``**, naming the missing segment and its entry count | the seam survived in the current file, still covered by the chain, and explains the orphan | | head of the live file cut off | **unexplained truncation** → `audit_chain_broken`, critical | the seam went with the deleted lines; nothing accounts for the remaining `prev_hash` | | a line edited anywhere present | **broken at position N**, naming the segment | hashes stop reproducing | | segment truncated at its end | **broken**, seam's `tail_hash` vs. actual | the seam pinned the tail before the segment left | | a line nobody can parse | **broken at position N**, naming the segment | an entry that will not deserialize is as much a hole as one that will not hash | | a seam graph that loops back on itself | **broken**, naming the segment | rotation only ever writes seams forward; a cycle was assembled | You read that table with: ```bash dotagent audit verify --full # exits 1 on the rows that say "broken" dotagent audit verify --json # same verdict, one machine-parseable line ``` The daemon runs the same check at boot, but only over the live file and only for yes/no — after the first rotation it never re-reads a rotated segment. Everything below the first two rows needs `--full` to be seen. Flags and output shapes: [`reference/cli.md`](/reference/cli#audit-verify). Could an attacker forge a seam — write an `audit_log_rotated` entry pointing at a segment that never existed, to explain away entries they deleted? Yes. But that costs them exactly what a total rewrite already costs: write access plus recomputing the chain forward. **Rotation does not move the boundary between what the chain catches and what it doesn't.** It only adds a way to say "history was pruned here, on purpose" that is as trustworthy as everything else in the file — no more, and importantly no less. Two smaller hardening notes, since the seam is a value read off disk and disk is attacker-writable: * `rotated_to` is validated as a bare sibling filename matching `.[-N]`. A seam naming `../../etc/passwd` is not followed; it is treated as an orphan, i.e. suspicious. * A seam whose `prev_hash` disagrees with its own declared `tail_hash` is not a seam. It is an entry shaped like one, which is what a clumsy forgery looks like, and it reads as unexplained truncation. ## Defenses deferred (with rationale) | Defense | Why deferred | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | Manifest signing (minisign / GPG / age) | Key rotation UX is hard pre-1.0. Will be opt-in. | | Agent registry with explicit approval | Defense is theatre — attacker with rwx edits registry too. UX cost is high. | | Real sandbox (sandbox-exec / bwrap / firejail) | Cross-platform sandboxing is its own product. The `[security]` schema lands first; enforcement lands as a follow-up tracked by issue. | | Daemon binary self-hash | Useful but low-impact pre-1.0. Adds it when there's distribution channel beyond `cargo install`. | | Plugin signing | Same reasoning as manifest signing. | | TPM / Secure Enclave-backed signing | Out of scope for v0. Architectural note: the audit log + `dotagent approve` flow could anchor to a Secure Enclave key later. | | Hard resource limits (max\_retries clamp etc.) | Will land before 1.0. Currently we trust the manifest author. | ## What the user is expected to handle dotagent is **one layer** of defense, not the only one. We assume the user runs: * **Disk encryption** (FileVault / LUKS) — kills offline manifest tampering. * **SSH key passphrase** + 1Password/secrets manager — kills credential theft via filesystem. * **Outbound firewall** (Little Snitch on macOS / nftables on Linux) — kills exfiltration even if dotagent is hijacked. * **Regular backups / snapshots** — recovery, not prevention. * **macOS TCC permissions reviewed yearly** — Full Disk Access creep. dotagent's `doctor` command will eventually warn when these are not in place (best-effort detection). ## Convention for adding new event types When introducing a new audit event, decide: 1. **Severity**: `info` (run-of-the-mill), `notice` (worth grep'ing later), `critical` (notify out-of-band). 2. **Schema**: event fields are typed. Use existing fields where they fit (`agent`, `schedule`, `manifest_sha256`) before inventing new ones. 3. **`security-reviewer` agent**: run it against the change. The agent in `.claude/agents/security-reviewer.md` knows the threat model and will flag if the new event widens attack surface. ## Reporting issues Security issues: open a GitHub issue with `security` label, or email the maintainer privately. Do not include reproduction code in public issues before discussion. # FAQ Quick answers to recurring questions. If yours isn't here, try [Troubleshooting](/guides/troubleshooting) first. *** ## Installation & support ### Does dotagent run on Windows? **No.** dotagent uses `kill(2)`, `osascript`, launchd, systemd, and Unix signal handling. WSL works (Linux paths apply). Native Windows is out of scope. ### Does it need root / sudo? **No.** dotagent runs as your user. The daemon is registered as a **user agent** (launchd `gui/` domain) / **systemd user unit**. There's no system-wide install, no setuid binary, no privileged helper. The only step that might surface a prompt is macOS Full Disk Access / Automation entitlement when a `sink-file` writes into protected dirs (`~/Documents`, `~/Downloads`) or the `imessage` notifier sends through Messages.app. Those are per-user TCC grants, not root. ### Is there a stable release? **Pre-release** as of this writing. The CHANGELOG / GitHub Releases page is authoritative. What's stable: * The `agent.toml` manifest schema (additive changes only) * The plugin protocol (verbs + JSON payload shape) * The heartbeat file shape (compatible with the legacy Fish framework) What might still move: * `config.toml` may grow new sections. * A couple of CLI subcommands are stubs (`bootstrap`, `plugin invoke`). ### Will it run on a server (headless)? **Yes**, with caveats: * `desktop` notifier needs a graphical session (`NSUserNotification` on macOS, D-Bus on Linux). Headless? Skip that driver — `slack`, `ntfy`, `pushover` are pure HTTPS and work fine. * `imessage` is macOS GUI-only. * On Linux, enable lingering so the systemd user session survives logout: `loginctl enable-linger $USER`. ### Multi-user / multi-tenant? dotagent runs **per-user**. The state directory is `$DOTAGENT_HOME` (default `~/.config/dotagent`). Two users on the same machine have completely independent daemons, manifests, audits — none shared. If you need cross-user orchestration, run one daemon per user OR build a thin shared layer (queue, shared filesystem). dotagent isn't a multi-tenant scheduler. *** ## Concepts ### Why a daemon and not just cron? `cron` runs an entry. `dotagent daemon` runs **and monitors** an entry. The difference is whether you can answer "did the 08:30 run succeed?" without writing your own bookkeeping. Specifically, cron can't: * Retry on failure with backoff. * Tell you a window passed without being satisfied. * Notify on `attempt_failed` / `given_up` / `stale` / `recovered` semantics — including telling you an entry **stopped running at all**, and saying so again tomorrow if you haven't fixed it. * Detect manifest tampering or phantom agents. * Run preflight checks before the body. * Spawn one process per agent without forking the scheduler. If your task is "run this thing every Monday at 9am, fire-and-forget, don't care what happens" — `cron` is enough. If you care whether it *actually ran*, you wanted dotagent (or kubernetes cronjobs, or nomad jobs — same family). ### Why not launchd / systemd timers directly? Same answer as cron, plus a deeper one: dotagent uses launchd / systemd to keep the **daemon** alive, then does adaptive scheduling itself. That's intentional: * Generates **one** unit, not one-per-agent. Adding an agent is a `.toml` drop, not a unit-install + reload + boot dance. * Adaptive sleep — wake at the next event, not "tick every minute". 100 schedules cost the same as 10. * Per-(agent, schedule) retry policy. launchd has `RetryDelay` for the whole unit; systemd has `Restart=on-failure` for the whole unit. No notion of "this window failed". * Cross-platform — `agent.toml` is identical on macOS + Linux. The launchd plist / systemd unit is a generated artifact, not your source of truth. ### Why is this not just kubernetes cronjobs? It is, ish. If you're already on a k8s cluster, k8s cronjobs are fine. dotagent is for the **laptop / single-user / on-device** version of the same problem — orchestrating personal automations without running k8s on your laptop. Different deployment target, similar shape. ### Why is the agent a subprocess and not a library? * **Crash isolation.** A panic in your agent doesn't take down the daemon. * **Polyglot.** Fish, Python, Go, Rust, bash, a compiled binary — all first-class. No SDK to import. * **No ABI.** No `cdylib` versioning headaches, no FFI breakage. Trade-off: fork+exec per run (\~5-10ms on Apple silicon). dotagent fires agents on **discrete events** (a schedule window), not in hot loops, so the cost is invisible. ### Why is each notifier in-process but plugins are subprocess? Different fire frequency: * **Notifiers** fire on **every** failure attempt. A 20-retry agent fires the notifier 20 times. fork+exec × 20 + JSON marshal/unmarshal × 20 was real overhead. In-process is faster and removes the dependency on extra binaries on `$PATH`. * **Sinks / preflight** fire **once per run**. The fork+exec cost is invisible. The plugin protocol stays alive for third-party notifiers (Discord, Teams, etc.) via `driver = "plugin"` — escape hatch when the built-in drivers don't cover your case. ### Why isn't there an LLM SDK? dotagent runs scripts. The script decides whether and when to call an LLM. This is intentional: * Most "agent" loops don't need an LLM 80% of the time. Pay tokens only when judgment is actually required. * Picking the model / prompt / cost / retry shape is **your call**, not dotagent's. Lock-in to a vendor SDK is not what you want from an orchestrator. See [`concepts/agents.md`](/concepts/agents#the-dotagent-definition) for the philosophy. ### Does dotagent speak MCP? It **serves** MCP; it does not **consume** it. `dotagent mcp` exposes every installed agent as a callable tool over JSON-RPC on stdio, so Claude Code, Claude Desktop or any MCP proxy can run your agents. The point is that a model picks from a closed catalog instead of composing a command — a tool name that does not exist is not callable, so "only run what the operator declared" is a property of the protocol rather than a check someone has to remember. In the other direction nothing changed: dotagent has no MCP client. An agent that wants to reach an MCP server spawns one, the same way it spawns `curl`. See [`reference/mcp.md`](/reference/mcp). ### Can a message start a run? Yes, with inbound Telegram configured. It is off by default. A message reaches a dispatcher agent you name, and that agent's stdout goes back to the chat. dotagent itself interprets nothing — there is no model, provider or prompt in the daemon. What the dispatcher does with the message is its business: hand it to `claude -p`, or match `/disk` with a `case` statement and spend no tokens at all. This is the one path where someone with no access to your machine can cause a local process to run, so read [V8 in the threat model](/security/threat-model) before enabling it. See [`concepts/telegram.md`](/concepts/telegram). *** ## Workflow ### Can I debug an agent without the daemon? Yes — three options: ```bash # 1. Foreground run, no plugins fire. dotagent run --schedule # 2. Foreground run, plugins fire (preflight / sink / notify). dotagent run-now # 3. Direct invocation (no dotagent involvement at all). cd ~/.config/dotagent/agents/ fish ./agent.fish # or python3 ./agent.py, etc. ``` For (3) you won't get the `AGENT_*` env vars — write your script to handle empty / unset gracefully if you want this to work. ### How do I trigger an agent immediately? ```bash dotagent run-now [--schedule ] ``` This fires the agent now, ignoring schedule windows. Preflight, sinks, and notifiers all run as usual. From an MCP client (Claude Code, Claude Desktop) the same thing is a tool call against `dotagent mcp`. From a chat, it is a message — see [`concepts/triggers.md`](/concepts/triggers) for how those differ from a scheduled run. ### How do I see what the daemon is about to do? ```bash dotagent tick --dry-run # → (dry-run) scanned 4 agent(s); would dispatch 1; next event: 2026-05-19T08:30:00-0300 ``` That `next event` timestamp is the next **agent** window. The daemon also wakes for the daily summary and for a 30-minute safety cap, whichever comes first — see [What wakes the daemon](/guides/daemon-lifecycle#what-wakes-the-daemon). ### How do I tail logs? ```bash dotagent logs # last 50 lines, one agent dotagent logs --follow # tail -F dotagent logs --follow # tail -F across every agent at once dotagent logs -n 200 # last 200 lines ``` For the daemon's own structured logs: ```bash tail -F ~/.config/dotagent/logs/daemon/dotagent.log | jq . ``` ### How do I clear state for a single agent? ```bash rm -rf ~/.config/dotagent/state/agents// rm ~/.config/dotagent/state/windows/-* ``` dotagent will re-run from scratch on the next dispatch. Your manifests and agent scripts aren't touched. ### How do I version-control my agents? Two patterns: **Pattern A — agents in dotfiles**: ```bash mkdir -p ~/dotfiles/agents/ # … develop there, version with git … ln -s ~/dotfiles/agents/ ~/.config/dotagent/agents/ ``` dotagent follows the symlink. Your agent stays under git; only the symlink lives in `~/.config/dotagent/`. **Pattern B — agents in their own repo**: ```bash git clone git@github.com:you/my-agents.git ~/agents # … one directory per agent, each with an agent.toml … ln -s ~/agents/ ~/.config/dotagent/agents/ ``` If you use Nix / home-manager, there's a reference module in [avelino/dotfiles](https://github.com/avelino/dotfiles) that handles discovery + symlinking automatically. See [`migrating-from-fish.md`](/guides/migrating-from-fish). *** ## Security ### Is dotagent sandboxed? **Not in v0.** The `[security]` block in `agent.toml` is parsed and `dotagent doctor` warns on inconsistency, but the runner doesn't enforce `allowed_commands` / `filesystem_writable` / network policy. Sandbox integration (sandbox-exec on macOS / bwrap / firejail on Linux) is on the roadmap. See [`security/threat-model.md`](/security/threat-model) for the full posture and what *is* defended in v0 (audit log, drift detection, phantom-agent detection, out-of-band notify). ### Does dotagent need disk encryption / outbound firewall / etc.? **Yes** — see [the user-expected layers in the threat model](/security/threat-model#what-the-user-is-expected-to-handle). dotagent is **one layer** of defense; we assume FileVault / LUKS, Little Snitch / nftables, and a passworded SSH key are in place. Without those, dotagent's audit log lets you reconstruct what happened, but doesn't prevent the attacker from doing it. ### What gets sent to third parties? By default: **nothing**. * Slack / ntfy / Pushover / Telegram notifiers: **only** when configured per agent. Each delivery POSTs the agent name + schedule + event + message body to the configured webhook. * OpenTelemetry export: opt-in via `[telemetry] otlp_endpoint`. The configured endpoint receives span metadata only (no log bodies yet). * iMessage notifier: shells out to `osascript` — Apple's privacy policy applies. * `mcp` CLI (if your agent uses it): whatever your `mcp` config says. There's no telemetry pinging an Anthropic/Avelino-owned endpoint. No update-check call-home. No analytics. ### Can I run plugins I didn't write? Treat them like any binary you put on `$PATH` — same trust assumptions. The audit log records every invocation with `ok` / `error` and the resolved binary path, so misbehaving plugins are at least **detectable** after the fact. See [V4 in the threat model](/security/threat-model#v4--plugin-swap). *** ## Limits ### How many agents can I run? Soft answer: hundreds, easily, on a modern laptop. The scheduler is adaptive — sleep until the next event, not poll. The bottleneck is fork+exec, which happens **only** when an agent fires, not on every tick. Hard answer: there's no internal cap. dotagent has been tested with 9 production agents firing 14+ times/day each. ### Can I run a sub-second schedule? No, and you shouldn't. `interval_minutes` minimum is 1 minute, and spawn overhead is real (5-10ms × language + runtime). If you need high-frequency work, write a long-running service and don't use dotagent. ### What's the max agent timeout? `timeout_seconds: u64` — practical limit is "however long you're willing to wait". Default is 1800 (30 min). Set higher for slow agents (some FinOps reports take 20+ min). dotagent sends SIGTERM, waits 5 seconds, then SIGKILL. *** ## Trivia ### Why "dotagent"? Two reasons: 1. "dotfiles for agents" — the same convention that turned shell config into version-controllable text turns scheduled automations into version-controllable text. 2. `.` (dot) is the smallest unit of file naming. The whole point of dotagent is staying out of the way: agent.toml is small, the binary is one, the daemon is one process. Minimal noise. ### Why launchd label "run.avelino.dotagent"? Reverse-DNS label, same convention Apple uses internally for system agents. `avelino` because the project was originally personal to one user; the label may genericize before 1.0. The daemon's behavior is unchanged regardless. ### Will dotagent run my LLM agents? It'll run **scripts that call** your LLM agents. There's no LLM in dotagent itself. See [`concepts/agents.md`](/concepts/agents) for the "script controls, AI analyzes" philosophy. ### Is there a UI? No. `dotagent status` is the textual dashboard. The structured logs are JSON — pipe to whatever (Grafana / Honeycomb / your dashboard tool). A web UI is **not** on the roadmap. *** ## Related * [Installation](/getting-started/installation) — install paths * [First agent](/getting-started/first-agent) — zero → daemon * [CLI reference](/reference/cli) — every command * [Troubleshooting](/guides/troubleshooting) — sintoma → fix * [Threat model](/security/threat-model) — what's defended, what isn't # LLM digest (llms.txt) — single-page reference for LLM agents writing dotagent manifests