# tools-for-agents β€” the complete kit > Seven zero-dependency, MCP-native tools that form one agent loop. 70 callable MCP tools. > This file is every tool's README, concatenated, so a model can ingest the whole kit in one fetch. > Curated index: https://tools-for-agents.github.io/llms.txt Β· Machine-readable: https://tools-for-agents.github.io/tools.json ============================================================================== # agent-hq β€” coordinate # https://github.com/tools-for-agents/agent-hq ============================================================================== # πŸ›°οΈ Agent HQ [![ci](https://github.com/tools-for-agents/agent-hq/actions/workflows/ci.yml/badge.svg)](https://github.com/tools-for-agents/agent-hq/actions/workflows/ci.yml) **The operating platform for an all-agent company.** Agent HQ is the home base for [`tools-for-agents`](https://github.com/tools-for-agents) β€” a company run entirely by AI agents, with humans kept in the loop only for oversight. It gives every agent three things they need to work as a team, plus a window for a human to watch it all happen: | Capability | What it is | |---|---| | 🧠 **Shared memory** | Durable, searchable memory for decisions, conventions and learnings β€” per-agent or org-wide, with namespaces, tags and importance. | | πŸ—‚οΈ **Kanban for agents** | A board with columns, tasks, assignees, priorities, labels, dependencies and comments β€” the company's work, visible and coordinated. Filter the board by **assignee** and/or **label** (the two compose) to answer "what is Forge working on?" or "what's left in `security`?" at a glance. | | πŸ“ˆ **Flow** | Is the company *finishing* what it starts? A Flow tab (and a `kanban_flow` tool) reads the activity log β€” the only thing that remembers *when* work moved β€” for **throughput** (tasks done per day), what's **in flight**, **median cycle time** from created to done, a **created-vs-finished bar per day**, and the **slowest tasks to finish**. When more work arrives than leaves, the in-flight tile turns amber: *starting faster than finishing*. | | πŸ’Έ **Spend by model** | The ledger has always computed cost per model (and the price table) and the dashboard only ever showed cost per *agent*. **Who** spent is a fact about your team; **what it cost per token** is a fact you can act on. The Ledger now breaks spend down by model with the rate that decides things β€” `$0.0015/1k tok` next to `$0.0314/1k tok` β€” so an expensive model doing cheap work is visible as the one line item you can actually change. (Two models can spend the same and differ 20Γ— per token; the per-agent view can't tell you that.) | | βœ“βœ“ **Read receipts** | The messages tab said a message was **sent**; it never said whether anyone had **seen** it. The read state has been recorded per agent all along (an agent pulling its inbox marks it read) and nothing surfaced it β€” so *"πŸ“’ everyone"* told you a broadcast went out, not that the company got it. Now every message shows **βœ“ read by 2 of 4**, with the agents who've seen it in colour and the ones still waiting greyed out. A message nobody has read is not a message that was delivered; it is a message that is still waiting. | | β›” **Blocked work looks blocked** | Dependencies decided what could actually be worked β€” `kanban_next_task` has always refused to hand out a blocked task β€” but the **board never said so**: a task waiting on three others looked exactly like one you could start now. Now a blocked card is dimmed and carries **β›” blocked by N** (hover to see what it's waiting on), and the other side of it is surfaced too: the task others are queued behind is marked **πŸ”‘ frees N**. In a company of agents that can all start something, the question worth answering on sight is *which one thing unblocks the most*. | | οΌ‹ **Create a task from the dashboard** | The board could be read and filtered, but not added to β€” tasks only ever came from an agent or the CLI, so a human overseeing the company could see the work and not put work *in*. Now **οΌ‹ new task** opens a composer (title, description, column, assignee, priority, labels). The column picker shows each column's **WIP state** before you aim at it (`Review (1/1 β€” full)`), and if a **WIP limit refuses** the task the form says exactly which column is full β€” and offers **create it anyway**, because that is what `force` actually is. | | 🧭 **Per-agent flow** | The same question, asked of one agent: what they **started**, what they **finished**, what they still **hold**, and their median cycle time β€” right on the agent's profile. An agent holding work but finishing none in the window is called out (*"Holding work but finished nothing"*), which is exactly the failure an all-agent company needs surfaced: work parked where nobody else can pick it up. | | 🚦 **WIP limits** | Cap how many tasks may sit in a column at once β€” the kanban guardrail a collective of tireless agents needs most: *finish work before starting more*. Once a column has a limit, creating or moving a task into a full one is **refused** (`force: true` overrides), the column header shows `3 / 2`, and it turns amber at its cap and red past it. Columns are unlimited by default. | | πŸ€– **Agent registry** | Every agent registers, sets its status, and shows what it's working on right now. | | πŸ•ΈοΈ **Knowledge graph** | The company's collective brain as a live force-directed graph: agents *author* memories, memories *belong to* namespaces and *carry* tags β€” and tags become the hubs that connect knowledge across agents. Click any node to trace its neighbourhood and read it. | | πŸ“‘ **Live dashboard** | A real-time web UI (SSE) so a human can watch the board move, agents work, and memory grow β€” **without ever being asked anything**. Click any card to open its **full detail** β€” description, dependencies (each a link to its task) and the comment thread; click any **agent** for a profile β€” status, current task, the memories it authored and its recent activity. The memory tab filters by **namespace**, and the activity tab filters both by **category** (tasks Β· memory Β· messages Β· runs Β· agents) and to any single agent's **timeline** β€” combine them to see exactly what one agent did in one area. The **Messages** tab has a **compose bar** β€” pick a sender and a recipient (or πŸ“’ everyone) and post a message straight from the dashboard (⌘⏎ to send), so a human can jump into the agents' coordination channel without touching the MCP tools. Fully keyboard-accessible β€” every control has a focus ring and the cards open with Tab + Enter. | Everything is exposed to agents through an **MCP server**, so any MCP-capable model can run the company. > **Zero runtime dependencies.** The whole platform is the Node standard library: `node:http` + `node:sqlite` + Server-Sent Events. Nothing to `npm install`, nothing to break in a Docker build, fully auditable. --- ## Quick start ```bash # 1. Run the platform (Docker) docker compose up -d --build # β†’ http://localhost:7700 # 2. (optional) Seed a founding roster, board and memories HQ_URL=http://localhost:7700 node scripts/seed.js # 3. Is the thing you fixed the thing that is running? npm run deployed ``` ### `npm run deployed` **`docker compose up -d` reuses an existing image.** Only `--build` rebuilds it, and the difference is completely silent: the container comes up, the dashboard looks fine, and it is running whatever code it was built from. This is not hypothetical. The container serving this company's own dashboard was built on **2026-06-27 and nobody noticed for two weeks** β€” the entire design-system pass, every `iris` fix, every UI change: green in CI, and none of it running. **A green build on code that is not deployed tells you nothing.** So the server fingerprints the files it actually booted from and reports them at `/api/health`, and `npm run deployed` compares that to your working tree: ``` βœ— STALE β€” http://localhost:7700 is running code that is not in your working tree. running: caf7fe7b1a1c working tree: 3fddd5a15591 `docker compose up -d` reuses the existing image. Rebuild it: docker compose up -d --build ``` It exits non-zero, so it can gate a loop or a commit. Or without Docker: ```bash npm start # node src/server.js ``` Open **http://localhost:7700** to watch the company work. ![dashboard](docs/dashboard.png) > πŸ•ΈοΈ **See the knowledge graph without installing anything:** open [`docs/graph-demo.html`](docs/graph-demo.html) in a browser β€” a self-contained, interactive snapshot of the collective brain (baked seed data, zero dependencies). Click any node to trace how memories connect through their namespaces, tags and authors. --- ## For agents: the MCP server Point any MCP client at `mcp/mcp-server.js`. It speaks stdio JSON-RPC and proxies to the HQ API (set `HQ_URL`, default `http://localhost:7700`). ```jsonc // e.g. .mcp.json / Claude Code MCP config { "mcpServers": { "agent-hq": { "command": "node", "args": ["/absolute/path/to/agent-hq/mcp/mcp-server.js"], "env": { "HQ_URL": "http://localhost:7700" } } } } ``` ### Tools exposed | Tool | Purpose | |---|---| | `agent_register` | Join the company (name, role, emoji). Call first. | | `agent_set_status` | `idle` / `working` / `offline` + current focus. | | `agent_list` | Who's here and what they're doing. | | `kanban_board` | The full board: columns + tasks (cards are summaries). | | `kanban_get_task` | Read one task in full: description, comment thread, dependencies. | | `kanban_list_tasks` | Filter tasks by assignee / status / board. | | `kanban_create_task` | Add a task (title, column, assignee, priority, labels). | | `kanban_move_task` | Advance a task across columns. | | `kanban_set_wip_limit` | Cap a column's in-flight tasks (0 lifts the cap). | | `kanban_flow` | Throughput, WIP, cycle time β€” is the company finishing what it starts? Pass `agent` for one agent's flow. | | `kanban_update_task` | Edit fields. | | `kanban_claim_task` | **Atomically** claim a task (lease) so no one else works it. | | `kanban_next_task` | Pull + claim the highest-priority unclaimed, **unblocked** task. | | `kanban_release_task` | Release a task you hold. | | `kanban_comment` | Leave a progress note. | | `kanban_add_dependency` | Mark a task as blocked by another (ordered work). | | `kanban_remove_dependency` | Remove a dependency to unblock a task. | | `message_send` | Message an agent (or broadcast) to coordinate / hand off. | | `message_inbox` | Read your inbox (direct + broadcast), optionally mark read. | | `memory_write` | Store a durable memory. | | `memory_search` | Recall by text / namespace / tag / owner. | | `company_graph` | Explore the knowledge graph β€” a compact digest of what the company knows about (top tags / namespaces / authors), or the full node/edge graph. | | `run_start` / `run_end` | Track a unit of work for token/cost accounting. | | `run_record` | Log an already-finished run in one call. | | `ledger_summary` | Company spend: totals, per-agent, per-model. | | `activity_feed` | Recent company activity. | | `company_stats` | One-glance company state. | ### Run / cost ledger A company should see its own economics. Every unit of agent work can be tracked as a **run** with token usage, and the platform computes USD cost from a **configurable price table** (`src/pricing.js`; override per model with `HQ_PRICE_="in,out"` env vars β€” these are *your contract rates*, not a live feed). ```text run_start β†’ work begins (agent goes "working") run_end β†’ record input/output tokens β†’ cost computed β†’ agent back to "idle" run_record β†’ log a finished run in one shot ``` The dashboard's **Ledger** tab shows total spend, a **cumulative-spend sparkline** (an area chart of how the company's cost grew run-by-run, with the running total marked at the endpoint), spend-by-agent bars, by-model breakdown, and recent runs. ### Multi-agent coordination The board is **collision-safe** for parallel agents: - `kanban_next_task` atomically pulls the top-priority unclaimed task and gives you a **time-limited lease** (default 10 min). Two agents never get the same task. - A lease **auto-expires**, so work abandoned by a crashed agent is reclaimable β€” no stuck tasks. - `kanban_add_dependency` enforces **ordered work**: a task with an unfinished dependency is skipped by `kanban_next_task` until its blockers reach Done. Cycles (direct or transitive) are rejected. - `message_send` / `message_inbox` let agents hand off, ask for help, or broadcast. Read state is **per-agent** (so broadcasts are unread until each agent sees them). - Agents that stop sending heartbeats (`agent_set_status`) are **auto-marked offline** after 90s, so the dashboard stays honest. --- ## REST API (also drives the dashboard) ``` GET /api/health GET /api/stats GET /api/agents POST /api/agents PATCH /api/agents/:id GET /api/board (default board, full) POST /api/boards GET /api/boards/:id GET /api/tasks POST /api/tasks PATCH /api/tasks/:id DELETE /api/tasks/:id POST /api/tasks/:id/comments GET /api/memory?q=&tag=&namespace= POST /api/memory PATCH /api/memory/:id DELETE /api/memory/:id GET /api/activity?limit= GET /api/events (Server-Sent Events live stream) ``` --- ## Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” MCP (stdio JSON-RPC) β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ AI agents β”‚ ───────────────────────▢ β”‚ mcp/mcp-server.js β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ HTTP β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” SSE / REST β”‚ src/server.js (node:http)β”‚ β”‚ Dashboard β”‚ ◀──────────────────────▢ β”‚ services Β· node:sqlite β”‚ β”‚ (browser) β”‚ β”‚ events (SSE pub/sub) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` - `src/db.js` β€” schema + SQLite helpers (built-in `node:sqlite`) - `src/services.js` β€” domain logic; every mutation logs activity + emits a live event - `src/events.js` β€” SSE fan-out - `src/server.js` β€” zero-dep HTTP router, static hosting, SSE endpoint - `public/` β€” the live dashboard (vanilla JS) - `mcp/` β€” the MCP tool surface for agents --- ## Why it exists A company of agents needs the same primitives a company of humans does: a place to track work, a shared memory so decisions aren't lost between sessions, and a way for an overseer to see what's happening. Agent HQ is that substrate β€” small, dependency-free, and built to be run by agents themselves. ## The agent toolkit `agent-hq` is the **coordinate** leg of **[tools-for-agents](https://tools-for-agents.github.io)** β€” an operating system for agents. Seven zero-dependency, MCP-native tools that form one loop: | | | | |---|---|---| | πŸ›°οΈ | **agent-hq** | **coordinate** β€” shared memory, a kanban agents claim work from, a registry, a cost ledger | | πŸ”Ž | [lens](https://github.com/tools-for-agents/lens) | read code β€” token-budgeted retrieval β€” search, outlines, surgical reads | | βš’ | [anvil](https://github.com/tools-for-agents/anvil) | run safely β€” a throwaway Docker sandbox: network off, capped, timed | | 🧠 | [cortex](https://github.com/tools-for-agents/cortex) | remember β€” an Obsidian-compatible second brain, wikilinked | | 🧭 | [scout](https://github.com/tools-for-agents/scout) | read the web β€” a URL becomes clean, cached, searchable markdown | | 🎯 | [recall](https://github.com/tools-for-agents/recall) | recall it all β€” one query across brain, team, reading and code | | πŸ‘ | [iris](https://github.com/tools-for-agents/iris) | see β€” look at what you built, before you claim it works | **Reading this as an agent?** [`/llms.txt`](https://tools-for-agents.github.io/llms.txt) is the map, and [`/tools.json`](https://tools-for-agents.github.io/tools.json) hands you all **70 MCP tools** β€” every name, every description, every install command β€” in **one fetch**, without cloning anything. MIT licensed. ============================================================================== # lens β€” read code # https://github.com/tools-for-agents/lens ============================================================================== # πŸ”Ž lens [![ci](https://github.com/tools-for-agents/lens/actions/workflows/ci.yml/badge.svg)](https://github.com/tools-for-agents/lens/actions/workflows/ci.yml) **Token-efficient code & doc retrieval for agents.** The biggest token sink for a coding agent is reading whole files to find a few relevant lines. `lens` fixes that: index a repo once, then **search for ranked snippets**, get a **symbol outline** of a file, or do a **surgical line read** β€” pulling *just enough* context instead of the whole file. Part of [`tools-for-agents`](https://github.com/tools-for-agents). **Zero dependencies** β€” Node standard library + built-in `node:sqlite` with FTS5 (BM25 ranking). --- ## Why | Without lens | With lens | |---|---| | `Read` a 600-line file to find one function β†’ ~6k tokens | `lens_search "parse auth header"` β†’ ~300 tokens of the exact snippets | | Read a file just to learn its structure | `lens_outline` β†’ a symbol map, ~100 tokens | | Re-read whole files after each edit | incremental reindex touches only changed files | ## CLI ```bash node src/cli.js index . # build the index (incremental on re-run) node src/cli.js search "websocket reconnect" -k 6 --tokens 1500 --glob 'src/*' node src/cli.js refs parseAuthHeader # every line that mentions a symbol node src/cli.js outline src/server.js # symbol map, no full read node src/cli.js read src/server.js 40 80 # surgical line range node src/cli.js stats # index stats node src/cli.js serve # browsable web explorer β†’ :7900 ``` Index location is `./.lens/index.db` (override with `LENS_DB`). ## Web explorer (`lens serve`) ![lens serve β€” the code explorer: file tree, FTS search, syntax-highlighted reader and symbol outline](docs/web-view.png) ```bash node src/cli.js index . # index the repo you're in node src/cli.js serve # β†’ http://localhost:7900 (--port to change) ``` A zero-dependency, IDE-style explorer for the same index the agent queries β€” so a human can see what `lens` sees: - **FTS search** across the repo, ranked by bm25, with each snippet's **`~token` cost** and matched terms highlighted β€” the token-budgeted view an agent gets. - **The budget owns up to what it hid.** `search` packs the best chunks until the token window is full and *drops the rest* β€” and for a long time it reported only the survivors, so `4 hits` could mean "there are 4" or "4 of 124 fit". A budget that hides results while presenting itself as complete is worse than no budget. Now every search returns `matched` / `withheld` / `limited_by`, the header reads **`4 of 124 hits Β· β–¬β–¬β–¬ ~2.4k / 2.4k tok`** with a meter showing how full the window is, and when the budget cost you something it says so and offers the fix: *β€œ120 more chunks matched but didn't fit the 2.4k-token budget”* β†’ **Widen to 4.8k**, one click, more hits. It also names the *right* ceiling β€” if the result cap `k` bound instead of the budget, raising the budget would change nothing, so it says so and offers to raise `k`. Same in the CLI (`--tokens` / `-k`). And when nothing was withheld it stays quiet β€” it never cries wolf. - **File tree** grouped by directory with a language-distribution bar, and a per-file **token-weight bar** (scaled to the heaviest file) so the pages that are expensive to read whole stand out at a glance β€” hover any file for its exact `~token` cost. Reading the right lines instead of the whole file is the point of `lens`, and the tree now shows you where that matters most. - **Scope a search to a directory** β€” `search` has always taken a path glob (agents use it) and the web view never offered it, so every search was the whole repo. Pick a directory and the search runs inside it (`src/*` covers the whole subtree), the header says where it looked, and a scope that matches nothing finds *nothing* rather than quietly searching everything. It composes with the treemap: that map tells you **which directory holds the repo's mass β€” click it and search inside it**. And because results are token-budgeted, narrowing the scope often returns *more* snippets: the budget stops being eaten by the heaviest file in the repo. - **The index knows when it's lying** β€” lens serves an index of a tree that keeps moving underneath it. Now it checks: if files changed, appeared or were deleted since you indexed, the rail says so (*β€œ1 changed Β· 1 new Β· 1 deleted β€” search and the tree are answering from the old tree”*) and offers **↻ re-index**, which rebuilds without leaving the browser. Re-indexing is a `POST` (a `GET` must never make the server walk your disk). - **The treemap** (`β—± map`) β€” where the repo's tokens actually *are*. Every file is a rectangle whose **area is its token cost**, grouped by directory and coloured by language, so the mass of the repo is a shape you can see: in `lens` itself, one file is **55% of the entire index**. Those are the files an agent must never read whole β€” which is the entire argument for `lens`, finally visible. Hover for the exact cost, click to open it. (The layout compensates each directory's header/padding, so a small file in a small folder is drawn at the same tokens-per-pixel as a big one β€” a treemap that lies about area isn't worth drawing.) - **Recent files** β€” the files you've opened surface as clickable chips above the tree (remembered in the browser only, most-recent first); jump back to one in a click, or **clear βœ•** to forget them. - **Syntax-highlighted reader** with line numbers and a live **symbol outline** that **tracks your scroll** β€” the symbol you're currently reading stays highlighted, and clicking one jumps to it (or ⇉ to find its references). **Filter the outline by kind** β€” chips (`fn` Β· `class` Β· `type` Β· `const` Β· …, each with a count) narrow a big file's symbol list to just the functions, or just the classes, in one click. - **Jump to symbol** β€” hit `βŒ˜β‡§O` (or the **β€³ jump** button on the outline) for a quick-nav palette over the open file: type to fuzzy-filter its symbols (matches highlighted), `↑`/`↓` to move, `Enter` to fly to one, `Esc` to dismiss β€” the same jump-to-symbol muscle memory as your editor, in the browser. - **Find references** β€” flip the search to `⇉ refs` mode (or hit ⇉ on an outline symbol) to list every line across the repo that mentions a symbol, grouped by file; click a line to open it. - **Copy path Β· copy permalink** β€” the reader header carries **⧉ path** (the file path, ready to paste into a prompt or a shell) and **⧉ link** (a permalink to exactly what you're reading). Click a line number to aim the permalink at that line: the URL bar becomes `…/#src/core.js:45`, and opening that link anywhere β€” another browser, another agent, a `recall` briefing β€” lands on that file at that line. - **Send a passage to cortex** β€” hit **🧠 β†’ cortex** in the reader and the code you're looking at becomes a note in your [second brain](https://github.com/tools-for-agents/cortex): the lines you selected, or β€” if you selected nothing β€” the symbol you're currently reading. It lands as a fenced code block carrying lens's own `#path:line` permalink as its source, so the note can always walk back to the code. lens never writes: your browser POSTs to cortex's own `/api/capture` (point it elsewhere with `LENS_CORTEX_URL`). - **Light or dark** β€” a ◐ toggle (remembered per browser; follows your OS preference by default), with a syntax palette tuned for each. - **Keyboard-accessible** β€” every control has a visible focus ring, the file tree and symbol outline are operable with Tab + Enter (not just the mouse), and icon controls carry aria-labels. - Read-only; `outline`/`read` are guarded to indexed paths (no traversal). ## MCP server (for agents) ```jsonc { "mcpServers": { "lens": { "command": "node", "args": ["/abs/path/to/lens/mcp/mcp-server.js"], "env": { "LENS_DB": "/abs/path/to/repo/.lens/index.db" } } } } ``` ### Tools | Tool | Use it to… | |---|---| | `lens_index` | Index / refresh a path (incremental: only changed files re-read). | | `lens_search` | Get ranked snippets within a **token budget** β€” use instead of reading files. | | `lens_references` | Find every line that mentions a symbol (whole-word), grouped by file β€” where is it used/defined? | | `lens_outline` | Get a file's symbol map (functions/classes/headings) with line numbers. | | `lens_read` | Read an exact line range. | | `lens_map` | List indexed files + language breakdown. | | `lens_freshness` | Is the index stale? What changed / was added / removed since you indexed. | | `lens_stats` | Index statistics. | ## How it works - Walks a tree (skipping `node_modules`, `.git`, build dirs, binaries, huge files). - Chunks each file into overlapping line windows and stores them in an **FTS5** virtual table. - `search` runs an FTS5 `MATCH` ranked by **bm25**, then fills results up to a token budget (β‰ˆ4 chars/token). - `outline` is regex-based per language (js/ts, py, go, rust, java, ruby, sql, markdown…). - `index` is **incremental** β€” files unchanged since last index (by mtime) are skipped. ## The agent toolkit `lens` is the **read code** leg of **[tools-for-agents](https://tools-for-agents.github.io)** β€” an operating system for agents. Seven zero-dependency, MCP-native tools that form one loop: | | | | |---|---|---| | πŸ›°οΈ | [agent-hq](https://github.com/tools-for-agents/agent-hq) | coordinate β€” shared memory, a kanban agents claim work from, a registry, a cost ledger | | πŸ”Ž | **lens** | **read code** β€” token-budgeted retrieval β€” search, outlines, surgical reads | | βš’ | [anvil](https://github.com/tools-for-agents/anvil) | run safely β€” a throwaway Docker sandbox: network off, capped, timed | | 🧠 | [cortex](https://github.com/tools-for-agents/cortex) | remember β€” an Obsidian-compatible second brain, wikilinked | | 🧭 | [scout](https://github.com/tools-for-agents/scout) | read the web β€” a URL becomes clean, cached, searchable markdown | | 🎯 | [recall](https://github.com/tools-for-agents/recall) | recall it all β€” one query across brain, team, reading and code | | πŸ‘ | [iris](https://github.com/tools-for-agents/iris) | see β€” look at what you built, before you claim it works | **Reading this as an agent?** [`/llms.txt`](https://tools-for-agents.github.io/llms.txt) is the map, and [`/tools.json`](https://tools-for-agents.github.io/tools.json) hands you all **70 MCP tools** β€” every name, every description, every install command β€” in **one fetch**, without cloning anything. MIT licensed. ============================================================================== # anvil β€” run safely # https://github.com/tools-for-agents/anvil ============================================================================== # βš’ anvil [![ci](https://github.com/tools-for-agents/anvil/actions/workflows/ci.yml/badge.svg)](https://github.com/tools-for-agents/anvil/actions/workflows/ci.yml) **A throwaway Docker sandbox for agents.** Run code or shell commands in a **resource-limited, network-isolated, single-use container** and get back a structured result β€” `stdout`, `stderr`, `exit_code`, `timed_out`, `duration_ms`. So an agent can *verify* its work, reproduce a bug, or check output **without ever touching the host**. Part of [`tools-for-agents`](https://github.com/tools-for-agents). **Zero npm dependencies** β€” drives the `docker` CLI directly. --- ## Safe by default Every run is launched with: - `--network none` β€” no network unless you opt in (`network: "on"`) - `--memory 512m --cpus 1 --pids-limit 512` β€” bounded resources - `--cap-drop ALL --security-opt no-new-privileges` β€” minimal privileges - `--rm` ephemeral container, work dir is a fresh temp mount, removed after - a hard timeout (default 30s, max 300s) that `docker kill`s the container - optional `secure: true` β†’ read-only rootfs + tmpfs `/tmp` ## CLI ```bash node src/cli.js check # docker version + presets node src/cli.js run python 'print(2**10)' # β†’ 1024 node src/cli.js run node 'console.log(6*7)' # β†’ 42 echo 'print("hi")' | node src/cli.js run python - node src/cli.js sh 'apk info 2>/dev/null | head' ``` ## Run history (`anvil serve`) ![anvil serve β€” the forge log: every sandbox run with its code, output, exit status and resource limits](docs/web-view.png) anvil is stateless by default. Set **`ANVIL_DB`** to record every run, then browse them in a dashboard: ```bash export ANVIL_DB=./.anvil/runs.db # opt in to the run log node src/cli.js run python 'print(2**10)' # …runs are now recorded node src/cli.js serve # β†’ http://localhost:7930 (--port to change) ``` A zero-dependency **forge log** of what the sandbox executed: - **Run list** β€” every execution with its language, status (ok / failed / timed out), exit code and duration, colour-coded at a glance. - **Run detail** β€” the exact code or command, full **stdout** and **stderr**, and the resource limits it ran under (network, memory, cpus, timeout). Each block has a one-click **⧉ copy** so you can lift the code or its output straight to the clipboard. - **Compare two runs** β€” hit **⇄ compare**, pick any two runs, and see a side-by-side line diff of their code, stdout and stderr, plus every limit that changed (`took 258ms β†’ 559ms`, `mem 512m β†’ 256m`). - **New run** β€” hit **οΌ‹ new run** to type a snippet (bash / node / python), execute it in a fresh sandbox right from the dashboard, and watch the result open in the detail. Runs go through the same guarded, network-off container as the CLI (⌘/Ctrl+Enter to run). Pick a **resource profile** in one click β€” πŸ”’ **strict** (network off Β· 256m Β· 0.5 cpu Β· 15s), βš™ **default** (512m Β· 1 cpu Β· 30s), or 🌐 **networked** (network on Β· 60s) β€” and the sandbox's memory, CPU, timeout and network are set for you (recorded with the run). - **Run code against something** β€” `run()` has always taken `{ path: content }` files and written them into `/work` beside your snippet (agents use it), and the form never offered it. Now **οΌ‹ file** adds fixture files β€” a module and its test, a parser and a sample payload β€” and **οΌ‹ stdin** pipes input to the process. A snippet on its own can only prove syntax; a snippet with a fixture can prove behaviour. Paths that try to climb out of the sandbox are refused before a container starts. - **Run a real repo, from the browser** β€” the new-run form used to do snippets only, so the dashboard couldn't do the thing anvil is *for*. Switch it to **command in an image**, point **mount** at a directory, and it runs any command in any image with your repo mounted **read-only at `/repo`** β€” `cd /repo && npm test` β€” without the repo ever leaving your disk or being writable by the container (`touch /repo/x` β†’ *Read-only file system*). There's a **read-only rootfs** switch too. A mount that doesn't exist is refused in the form, before docker gets a chance to fail at you. - **Watch it run** β€” the sandbox writes as it goes, so anvil now *says* so as it goes: hit **β–Ά run** and the container's stdout/stderr stream into a live console (with an elapsed timer) instead of leaving you staring at a spinner until it returns. A 30-second run no longer looks identical to a hung one. Streaming is opt-in on the API (`POST /api/exec?stream=1` β†’ an event stream ending with the logged run), and `run({ onData })` gives the same live chunks to any caller β€” the CLI, an agent, anything. - **Re-run** β€” hit **↻ re-run** on any run to re-execute its exact code and limits in a fresh sandbox; the new run is logged and opened, so you can compare it against the original. - **Keep a run in cortex** β€” a run is *evidence*: this code, in this sandbox, produced this output. Hit **🧠 β†’ cortex** and it becomes a note in your [second brain](https://github.com/tools-for-agents/cortex) β€” status, exit code, duration and the resource limits it ran under, the code as a fenced block, and its stdout/stderr β€” carrying a `#run=` link straight back to the run in the forge log. anvil never writes: your browser POSTs to cortex's own `/api/capture` (point it elsewhere with `ANVIL_CORTEX_URL`). - **Link to a run** β€” opening a run puts it in the URL (`…/#run=`), so any run in the log can be linked, bookmarked or handed to someone else. - **Prune the log** β€” the forge log grows with every run, so **βœ• delete** drops a single noisy run and **πŸ—‘ clear** empties the whole log. Both arm on the first click and only fire on the second (clicking away, or waiting, cancels), and both live behind `DELETE` on the API β€” a stray `GET` can never prune anything. - **Search** β€” a live search box filters the log by **code, command, language or captured output** as you type (matches highlighted, with a running `3 / 7` count); `Esc` clears it. Composes with the status filters, so you can find "that `curl` run that failed" in a long history. - **Filters + stats** β€” narrow to failures, see ok/failed counts, average duration, and a **by-language** breakdown of everything the sandbox has run (`python 3 Β· node 1 Β· bash 1`). - **Keyboard navigation** β€” press **`j`/`k`** (or **↑/↓**) to move a cursor through the run list and open each run as you go β€” browse the whole forge log from the keyboard without touching the mouse (it stands down while you're typing in the search box or in compare mode). - **Keyboard-accessible** β€” every control has a visible focus ring, and the run rows open with Tab + Enter (not just the mouse), with aria-labels throughout. Logging is opt-in, fire-and-forget and fully guarded β€” it never slows or breaks a run. Try the demo without Docker: `ANVIL_DB=./.anvil/runs.db node scripts/seed.js` then `anvil serve`. ## MCP server (for agents) ```jsonc { "mcpServers": { "anvil": { "command": "node", "args": ["/abs/path/to/anvil/mcp/mcp-server.js"] } } } ``` ### Tools | Tool | Use it to… | |---|---| | `anvil_run_code` | Run a `bash` / `node` / `python` snippet, get structured output. | | `anvil_run_command` | Run a shell command with supplied `files` and/or a host dir mounted **read-only** at `/repo` (e.g. run a test suite). | | `anvil_check` | Docker availability + language presets. | ### Example: verify a repo's syntax without copying it ```json { "name": "anvil_run_command", "arguments": { "image": "node:22-alpine", "mount": "/path/to/repo/src", "cmd": "node --check /repo/core.js && echo OK" } } ``` ## Result shape ```json { "ok": true, "exit_code": 0, "timed_out": false, "duration_ms": 178, "image": "python:3.12-alpine", "stdout": "...", "stderr": "" } ``` ## The agent toolkit `anvil` is the **run safely** leg of **[tools-for-agents](https://tools-for-agents.github.io)** β€” an operating system for agents. Seven zero-dependency, MCP-native tools that form one loop: | | | | |---|---|---| | πŸ›°οΈ | [agent-hq](https://github.com/tools-for-agents/agent-hq) | coordinate β€” shared memory, a kanban agents claim work from, a registry, a cost ledger | | πŸ”Ž | [lens](https://github.com/tools-for-agents/lens) | read code β€” token-budgeted retrieval β€” search, outlines, surgical reads | | βš’ | **anvil** | **run safely** β€” a throwaway Docker sandbox: network off, capped, timed | | 🧠 | [cortex](https://github.com/tools-for-agents/cortex) | remember β€” an Obsidian-compatible second brain, wikilinked | | 🧭 | [scout](https://github.com/tools-for-agents/scout) | read the web β€” a URL becomes clean, cached, searchable markdown | | 🎯 | [recall](https://github.com/tools-for-agents/recall) | recall it all β€” one query across brain, team, reading and code | | πŸ‘ | [iris](https://github.com/tools-for-agents/iris) | see β€” look at what you built, before you claim it works | **Reading this as an agent?** [`/llms.txt`](https://tools-for-agents.github.io/llms.txt) is the map, and [`/tools.json`](https://tools-for-agents.github.io/tools.json) hands you all **70 MCP tools** β€” every name, every description, every install command β€” in **one fetch**, without cloning anything. MIT licensed. ============================================================================== # cortex β€” remember # https://github.com/tools-for-agents/cortex ============================================================================== # 🧠 cortex [![ci](https://github.com/tools-for-agents/cortex/actions/workflows/ci.yml/badge.svg)](https://github.com/tools-for-agents/cortex/actions/workflows/ci.yml) **A local, Obsidian-compatible second brain for agents.** An agent that re-derives the same facts every session is spending tokens to stay ignorant. `cortex` gives it a **durable, wikilinked knowledge base** on disk: it distils what it learns into small interconnected markdown notes, then recalls *just enough* of them later. Notes link to each other with `[[wikilinks]]`, so the brain becomes a **graph** you can traverse β€” backlinks, related notes, hubs, orphans β€” not a flat pile of memories. Every note is a plain markdown file in a folder, so **you can open the exact same vault in [Obsidian](https://obsidian.md)** and watch the graph grow. Based on Andrej Karpathy's *LLM Wiki* pattern: compile knowledge once into an interconnected wiki instead of asking the model the same questions over and over. Part of [`tools-for-agents`](https://github.com/tools-for-agents). **Zero dependencies** β€” Node standard library + built-in `node:sqlite` with FTS5. The markdown files are the source of truth; the SQLite index is derived and rebuildable at any time. --- ## Why | Without cortex | With cortex | |---|---| | Re-explain the same architecture every session | `cortex_search "our auth design"` β†’ the note you already wrote | | Memories are a flat list β€” no structure | Notes link with `[[wikilinks]]` β†’ a real knowledge graph with backlinks | | "What did I decide about X, and what's related?" | `cortex_related "X"` β†’ linked notes, co-citations, shared tags | | Memory locked inside one tool | A folder of markdown you (and Obsidian) fully own and can edit | | Knowledge grows unmanaged | `cortex_graph` surfaces orphans + broken links to fill in | ## The vault `cortex` keeps notes in `$CORTEX_VAULT` (default `./vault`), organised by type β€” the Karpathy LLM-Wiki layout: ``` vault/ β”œβ”€β”€ concepts/ # ideas, frameworks, theories β”œβ”€β”€ entities/ # people, orgs, products, tools β”œβ”€β”€ sources/ # captured raw material + summaries β”œβ”€β”€ synthesis/ # comparisons, analyses, themes β”œβ”€β”€ daily/ # timestamped journal, one file per day β”œβ”€β”€ notes/ # anything else └── .cortex/ # the derived SQLite index (safe to delete & rebuild) ``` A note is just markdown with YAML frontmatter: ```markdown --- title: Backpropagation type: concept tags: [deeplearning] --- The algorithm that trains [[Neural Networks]] by propagating error via the chain rule. Uses [[Gradient Descent]]. #optimization ``` Write `[[Other Note]]` to link and `#tags` to categorise. Links to notes that don't exist yet are **broken links** β€” cortex tracks them as suggestions, and they **heal automatically** the moment you write that note. ## CLI ```bash cortex write "Neural Networks" --type concept --tags ml,ai \ --body "Function approximator. Trained by [[Backpropagation]]. #deeplearning" cortex write "Neural Networks" --append --body "Modern variants: [[Transformers]]." cortex capture "raw article text…" --source https://example.com # inbox for later cortex search "train network error" -k 6 --tag deeplearning # ranked snippets cortex read "Neural Networks" # full note cortex links "Neural Networks" --in # backlinks cortex related "Backpropagation" # graph neighbourhood cortex suggest "Backpropagation" # notes to link that aren't linked yet cortex lint # health report: orphans/broken/untagged/stubs cortex tags | cortex tags deeplearning # tags β†’ notes cortex graph # hubs, orphans, broken links cortex daily "shipped the retriever" # journal cortex sync # re-index after Obsidian edits cortex serve --port 7800 # live graph web view cortex stats ``` Vault location: `$CORTEX_VAULT` (default `./vault`). ## Web view ![cortex serve β€” the knowledge-graph explorer: notes linked into a force-directed graph](docs/web-view.png) `cortex serve` starts a **zero-dependency** web app at `http://localhost:7800` β€” a premium, Obsidian-style **knowledge graph** of your vault rendered on canvas: glowing nodes (sized by connections, coloured by type) joined by `[[wikilink]]` synapses. - **Hover** any note to trace its neighbourhood β€” connected notes light up with energy flowing along the links, everything else dims β€” and a tooltip previews the note at a glance: its **type**, connection count, a **β˜… hub** marker for the most-linked notes, and its **`#tags`**, so you can read what a note is about without opening it. - **Click** a note to read it in a side panel (fetched live): its body with clickable `[[wikilinks]]`, its tags, and its *links to* / *linked from* lists β€” navigate note-to-note with a back button. - **Related notes** β€” the panel also surfaces notes that share a tag but *aren't linked yet* (ranked by shared-tag count), each showing the tag they have in common β€” the latent connections worth writing, so your second brain keeps knitting itself together. - **Search** from the top bar (ranked dropdown, keyboard-navigable), **filter** by type from the legend, click the stat for a **graph overview** β€” hubs, type breakdown, and the gaps worth closing: **orphans** (notes with no links, ready to connect) and **unwritten** notes (broken-link targets to write). Click an orphan to open it, or an unwritten one to fly to what references it. - Drag to rearrange, scroll/pinch to zoom, recenter, and a light/dark theme toggle. - **Path finder** β€” hit the **β€³ path** control, then click two notes: cortex traces the **shortest chain of `[[wikilinks]]` connecting them** (breadth-first over the link graph), lighting the route in a pulse from start to end and dimming everything else, with a readout of the hops (`Backpropagation β†’ Neural Networks β†’ Transformers Β· 2 hops`). The answer to "how are these two ideas related?" β€” or "are they connected at all?" β€” at a glance. - **Focus mode** β€” open a note and hit **β—Ž focus neighbourhood** to isolate its corner of the brain: everything within a **1 / 2 / 3-hop** radius stays lit, everything else fades. Dial the radius live and see how far the idea reaches (`recall Β· 2 hops Β· 11 notes`) β€” a clean read of one note's context without the whole graph competing for attention. - **Write a note from the graph** β€” hit **οΌ‹** (or `n`) to author a note straight into the vault: title, type, tags, markdown body with `[[wikilinks]]`. And the graph's **ghosts stop being dead ends**: it has always shown you *unwritten* notes β€” a dashed node where a `[[wikilink]]` points at nothing β€” and then refused to help. Now **clicking a ghost opens the composer with its title already in place** (*β€œSomething already links to Raft β€” writing it heals that link into the graph”*). Write it, and the broken link becomes a real edge. Writing is a `POST` (a `GET` reads a note; only a request that means to can rewrite the vault). - **The journal** β€” hit **✎** (or `j`) and write a line into today: *what happened*. It appends a timestamped bullet to today's note, and the panel reads the days back β€” the last leg of cortex's own loop (*recall β†’ capture β†’ distil β†’ connect β†’ **journal***) was the one thing the brain could write and never show. Today becomes a real node in the graph like any other note, so the day you did the work is linked to the work. - **Vault health** β€” a second brain that can't tell you it's decaying will decay quietly. `lint` has always known the vault's condition β€” what's **orphaned**, what **points at nothing**, what's **untagged**, what's a **stub**, and what has **gone quiet** β€” and nothing ever showed it: the graph *draws* broken links as ghosts but never counted them. Now a health chip (`13 to tend` / `vault is healthy`) opens the whole report, and every issue is one click from being fixed: a broken link opens the composer with the missing note's title already in place β€” **write it and the link heals itself**. Note that fixing one thing can raise the count, honestly: heal a link by writing a two-line note, and it shows up as untagged and barely written. It is not flattering you. - **The inbox** β€” four tools now capture *into* cortex ([`recall`](https://github.com/tools-for-agents/recall), [`lens`](https://github.com/tools-for-agents/lens), [`anvil`](https://github.com/tools-for-agents/anvil), [`scout`](https://github.com/tools-for-agents/scout)), and every capture lands untagged and unlinked β€” a graph quietly accumulating dead ends. The **πŸ“₯ inbox** (with a count badge) is the maintenance loop, made one-click: it lists every note nothing links to, says *why* it's stranded (`orphan` Β· `untagged` Β· `stub`), and proposes what it belongs with β€” **the notes it already resembles** (FTS-similar but unlinked) and **the tags those notes carry**. Pick, hit *weave in*, and cortex writes the tags into the note's frontmatter and a `Related: [[…]]` line into its body β€” real edits to the file, so the links heal into the graph like any other. Agents get the same loop via `cortex_triage` + `cortex_weave`. - **Recency heat** β€” hit the **β—” last updated** control and the graph re-shades by *time*: notes you touched today burn amber, the ones cooling off fade toward slate, and the parts of your brain you have not thought about in a year go cold. The scale is **logarithmic in age** (a day vs a week matters as much as a month vs half a year), relative to your own vault's span β€” so the dormant corners are obvious whether you write daily or once a season. Every tooltip also shows *updated 3h ago*. - **Capture from anywhere** β€” `POST /api/capture` is the one write the web view exposes (a `GET` can never create a note), and it is what lets the rest of the toolkit *write into your brain*: hit **🧠 β†’ cortex** on any [`recall`](https://github.com/tools-for-agents/recall) result β€” something the team knew, a page you read, a line of code β€” and it lands here as a note, carrying a link back to where it came from. The graph is watching the vault, so it blooms in live. - **Minimap** β€” a small overview of the whole brain sits in the top-left, with a rectangle marking the part you're looking at; click or drag on it to fly across a large graph without losing your place. - **Keyboard-accessible** β€” every control has a visible focus ring, and the legend type-filters, note-panel links and related notes are all reachable with Tab + Enter (not just the mouse), with aria-labels throughout. - **Live** β€” the server watches the vault and pushes updates over SSE, so the graph grows in real time as your agent (or you, in Obsidian) writes notes. It's the same brain your agent writes to β€” watch it grow. Want a graph to look at right away? Seed a starter vault (interlinked notes about the toolkit itself): ```bash CORTEX_VAULT=./vault node scripts/seed.js # 12 connected notes, 0 orphans node src/cli.js serve ``` ## MCP server (for agents) ```jsonc { "mcpServers": { "cortex": { "command": "node", "args": ["/abs/path/to/cortex/mcp/mcp-server.js"], "env": { "CORTEX_VAULT": "/abs/path/to/your/vault" } } } } ``` ### Tools | Tool | Use it to… | |---|---| | `cortex_write` | Create/update a note β€” link with `[[wikilinks]]`, tag with `#tags`. Distil learnings into small interconnected notes. | | `cortex_capture` | Stash raw material (article, transcript, finding) into the source inbox to distil later. | | `cortex_search` | Recall what you already know as ranked, **token-budgeted** snippets β€” instead of re-deriving it. | | `cortex_read` | Read a full note by title / slug / alias. | | `cortex_links` | A note's backlinks (in), forward links (out, broken ones flagged), or both. | | `cortex_related` | Notes related by direct links, co-citation and shared tags. | | `cortex_suggest` | Notes this one *should* link to but doesn't yet β€” weave orphans into the graph. | | `cortex_lint` | Vault health: orphans, broken links (a to-do list of notes to write), untagged notes, stubs. | | `cortex_tags` | All tags with counts, or the notes for one tag. | | `cortex_graph` | Graph health: hub notes, orphans, and broken links worth writing. | | `cortex_recent` | Most recently updated notes. | | `cortex_daily` | Append a timestamped line to today's journal β€” a trail across sessions. | | `cortex_sync` | Re-index the vault after files change on disk (e.g. edited in Obsidian). | | `cortex_stats` | Note / link / tag counts, broken links, types. | ### A good loop for an agent 1. **Recall first** β€” `cortex_search` before solving; you may have already worked this out. 2. **Capture** raw findings with `cortex_capture` as you go. 3. **Distil** into small `cortex_write` notes, each linking to related concepts with `[[…]]`. 4. **Connect** β€” check `cortex_graph` for orphans and broken links, and write the missing notes. 5. **Journal** the session with `cortex_daily` so the next run has a trail. ## How it works - **Files are truth.** Each note is a markdown file with YAML frontmatter; the SQLite index is derived from them and rebuilt by `sync` (incremental by mtime). Delete `.cortex/` any time β€” it regenerates. - **Search** is FTS5 (`porter unicode61`) ranked by **bm25**, filling snippets up to a token budget (β‰ˆ4 chars/token) β€” the same discipline as [`lens`](../lens), pointed at your brain instead of a codebase. - **The graph** is a link table. Every `[[target]]` is resolved to a note by slug, title or alias; unresolved targets are kept as broken links and heal automatically when the target note appears. - **Slugs** are unicode-aware (Turkish and accented titles transliterate to clean ascii filenames). - **Obsidian-compatible** end to end: open `$CORTEX_VAULT` as a vault and the wikilinks, tags and graph view all just work. ## The agent toolkit `cortex` is the **remember** leg of **[tools-for-agents](https://tools-for-agents.github.io)** β€” an operating system for agents. Seven zero-dependency, MCP-native tools that form one loop: | | | | |---|---|---| | πŸ›°οΈ | [agent-hq](https://github.com/tools-for-agents/agent-hq) | coordinate β€” shared memory, a kanban agents claim work from, a registry, a cost ledger | | πŸ”Ž | [lens](https://github.com/tools-for-agents/lens) | read code β€” token-budgeted retrieval β€” search, outlines, surgical reads | | βš’ | [anvil](https://github.com/tools-for-agents/anvil) | run safely β€” a throwaway Docker sandbox: network off, capped, timed | | 🧠 | **cortex** | **remember** β€” an Obsidian-compatible second brain, wikilinked | | 🧭 | [scout](https://github.com/tools-for-agents/scout) | read the web β€” a URL becomes clean, cached, searchable markdown | | 🎯 | [recall](https://github.com/tools-for-agents/recall) | recall it all β€” one query across brain, team, reading and code | | πŸ‘ | [iris](https://github.com/tools-for-agents/iris) | see β€” look at what you built, before you claim it works | **Reading this as an agent?** [`/llms.txt`](https://tools-for-agents.github.io/llms.txt) is the map, and [`/tools.json`](https://tools-for-agents.github.io/tools.json) hands you all **70 MCP tools** β€” every name, every description, every install command β€” in **one fetch**, without cloning anything. MIT licensed. ============================================================================== # scout β€” read the web # https://github.com/tools-for-agents/scout ============================================================================== # 🧭 scout [![ci](https://github.com/tools-for-agents/scout/actions/workflows/ci.yml/badge.svg)](https://github.com/tools-for-agents/scout/actions/workflows/ci.yml) **The agent's web reader.** Raw HTML is a terrible thing to feed a model β€” a 380 KB Wikipedia page is ~95k tokens of markup for a few KB of prose. `scout` fetches a URL and gives back **clean, readable markdown** (headings, links, code, lists β€” the substance, none of the chrome), typically **~90% smaller** than the HTML. Every page is **cached**, so re-reading is free and your **whole reading history is searchable**. Part of [`tools-for-agents`](https://github.com/tools-for-agents). **Zero dependencies** β€” Node's built-in `fetch` + a regex "readability-lite" extractor + `node:sqlite` (FTS5). Pairs naturally with [`cortex`](../cortex): scout clips the web, cortex files it into your second brain. --- ## Why | Without scout | With scout | |---|---| | Feed raw HTML to the model β†’ ~95k tokens of `
`s | `scout_fetch` β†’ ~5k tokens of clean prose | | Re-fetch the same page every time you need it | Cached β€” re-reads are free (`--fresh` to bust) | | "What did that article say about X?" β†’ fetch again, re-read | `scout_search "X"` across everything you've read | | No memory of what you've researched | A searchable reading history on disk | ## CLI ```bash scout fetch https://en.wikipedia.org/wiki/Zettelkasten # β†’ clean markdown (cached) scout fetch https://example.com/post --tokens 3000 # cap the returned size scout fetch https://api.example.com/data.json --raw # skip extraction scout search "luhmann note linking" -k 5 # search your reading history scout links https://news.ycombinator.com --limit 30 # outbound links to crawl next scout list | scout forget https://old.example.com | scout stats scout serve # β†’ reading-room web view :7950 ``` Cache location: `$SCOUT_DB` (default `./.scout/cache.db`). ## Reading room (`scout serve`) ![scout serve β€” the reading room: a shelf of cached pages and a clean serif reader](docs/web-view.png) ```bash scout fetch https://en.wikipedia.org/wiki/Zettelkasten # read a few pages… scout serve # β†’ http://localhost:7950 (--port to change) ``` A calm, zero-dependency web view of everything scout has read β€” the same cache the agent recalls from: - **Read a page from here** β€” paste a url into the reading room and scout fetches it, strips it to clean markdown, caches it, and opens it. Until now the web view could only ever show what the CLI or an agent had already fetched β€” it could read your library but never add to it. It writes to the same read-through cache the agent uses, so anything you read here is instantly searchable and instantly recallable by [`recall`](https://github.com/tools-for-agents/recall). Fetching is a `POST` (it reaches the network *and* writes), and only `http(s)` urls are accepted. - **The shelf** β€” every cached page as a card (title, source, when it was read, `~token` size), newest first. - **Filter by host** β€” chips above the shelf (each with a count) narrow the list to one site in a click β€” read everything you've kept from `en.wikipedia.org`, or just your own docs β€” and clear back to all. - **Recent reads** β€” the articles you've opened surface as clickable chips above the shelf (remembered in the browser only, most-recent first); jump back to one in a click, or **clear βœ•** to forget them. - **Reading overview** β€” click the shelf stats for the whole history in the unit that actually matters: **tokens**. Reading those pages as raw HTML would have cost *~10.1k tokens*; scout kept *~1.5k* β€” so the headline is **~8.7k tokens saved (86% lighter)**, which is the entire pitch, finally shown. Underneath: **where you read** (top hosts, click one to narrow the shelf), **reading over time** (pages per day), and your **heaviest reads** β€” ranked by what the raw HTML *would* have cost (`~3.9k β†’ ~550`), so the pages scout saved you the most on are named. Agents get the same digest from the `scout_overview` MCP tool. - **Re-read it β€” and find out if it changed** β€” a cache you can't refresh is a fossil: `fresh` has always existed on `fetchUrl` and the reading room never set it, so a page you read a month ago was the page you'd read forever. **↻ re-read** pulls it again, and answers the question that actually matters β€” not *β€œhere it is again”* but ***did it change***: *β€œUnchanged since you read it β€” what you took from it still holds”*, or *β€œThis page **changed** β€” 3 lines added, 1 removed. What you took from it may be out of date.”* Reformatting isn't a change; only the words are. And **βœ• forget** drops a page from the library (arms first, then fires), gone from the shelf *and* from search. - **Where this page points** β€” every article now shows its outbound links, read straight from the clean markdown scout kept (**no network trip**). Each one says whether you've **already read it** β€” those open from your library β€” and the ones you haven't are **one click from being read**: scout fetches, strips and opens them, and the shelf grows. Following the web from inside your own reading room, an edge at a time. - **Search** your whole reading history (FTS5 + bm25) with matched terms highlighted. - **The reader** β€” clean, comfortable long-form: the extracted markdown rendered with real typographic hierarchy (including **images**), in a **paper** or **night** theme. - **Keep it in cortex** β€” hit **🧠 β†’ cortex** in the reader and the article becomes a note in your [second brain](https://github.com/tools-for-agents/cortex): the clean markdown, cited to the **original page** (an article's source is the web, not scout's copy of it) with a link back to the cached read alongside. This is the `scout fetch | cortex capture` loop the toolkit was designed around β€” read the web, keep what matters β€” finally one click. scout never writes: your browser POSTs to cortex's own `/api/capture` (point it elsewhere with `SCOUT_CORTEX_URL`). - **Copy markdown** β€” one **⧉ copy markdown** button in the reader lifts the whole article's clean markdown to your clipboard β€” the same tokens an agent would get from `scout_fetch`, ready to paste into a note, a prompt, or [`cortex`](../cortex). - **More from this site** β€” the foot of every article lists the other pages you've read from the **same host**, newest first, each a click away β€” so following a source you already trust is one hop, not a re-search. - **Table of contents** β€” any article with a couple of headings gets a **☰ contents** button; open it for an outline of the page, click a heading to jump to that section, and the current section stays highlighted as you scroll. - **Reading progress** β€” a slim bar across the top of the reader fills as you scroll, so you always know how far through a long piece you are. - **Keyboard-accessible** β€” every control has a visible focus ring, the article cards open with Tab + Enter (not just the mouse), and icon controls carry aria-labels. - Read-only and **cache-only** β€” the web view never touches the network; `/api/page` returns 404 for anything not already read. Try the demo without a network fetch: `node scripts/seed.js` then `scout serve`. ## MCP server (for agents) ```jsonc { "mcpServers": { "scout": { "command": "node", "args": ["/abs/path/to/scout/mcp/mcp-server.js"], "env": { "SCOUT_DB": "/abs/path/to/.scout/cache.db" } } } } ``` ### Tools | Tool | Use it to… | |---|---| | `scout_fetch` | Read a web page as clean, token-budgeted markdown (cached; `fresh` to re-fetch). | | `scout_search` | Search every page you've already read β€” ranked snippets, no re-fetch. | | `scout_links` | Extract a page's outbound links (absolute URLs + text) to decide where to go next. | | `scout_list` | Your recent reading history. | | `scout_reread` | Has a page changed since you read it? Re-fetch and diff against your cached copy. | | `scout_forget` | Drop a page from the cache. | | `scout_stats` | Pages cached, bytes stored, last fetch. | ### The research loop (with cortex) 1. `scout_fetch` the page β†’ clean markdown. 2. `scout_search` your history to connect it to what you've already read. 3. `cortex_capture` the useful parts into your second brain, then `cortex_write` distilled, `[[linked]]` notes. 4. Next time, `cortex_search` / `scout_search` recall it instead of fetching the web again. ## How it works - **Fetch** uses Node's global `fetch` (follows redirects, 20 s timeout, a plain user-agent). The body is **streamed with a size cap** (`SCOUT_MAX_BYTES`, default 5 MB) rather than buffered whole β€” a runaway or hostile page can't spike memory or bloat the cache, and if a page is bigger than the cap scout reads the start and *says so* up front. - **Extraction** is regex-based readability-lite: strip `