AGENTS.md

Long-term Knowledge

Architecture

• @loreai/pi extension: three-package monorepo with shared SQLite DB: `packages/pi/` (`@loreai/pi`) is the third workspace package alongside `@loreai/core` and `opencode-lore`. Pi extension entry: `export default function(pi: ExtensionAPI)` wires 6 hooks — session_start (init DB, register recall tool), before_agent_start (LTM injection via `result.systemPrompt`), context (message rewriting for gradient), message_end (temporal capture), turn_end (trigger background distill/curate), session_before_compact (custom compaction). Both OpenCode and Pi share the same SQLite DB at `~/.local/share/opencode-lore/lore.db`, so switching hosts on the same project preserves knowledge/distillations/AGENTS.md.

• buildSection() renders AGENTS.md directly without formatKnowledge: AGENTS.md export/import: buildSection() iterates DB entries grouped by category, emitting `<!-- lore:UUID -->` markers via remark. splitFile() scans ALL_START_MARKERS (current + historical) — self-healing: N duplicate sections collapse to 1 on next export. Import dedup handled by curator LLM at startup when file changed. Missing marker = hand-written; duplicate UUID = first wins. ltm.create() has title-based dedup guard (case-insensitive, skipped for explicit IDs from cross-machine import).

• Host abstraction: LoreMessage/LorePart types + LLMClient interface decouple core from OpenCode SDK: `@loreai/core` is host-agnostic (zero dep on `@opencode-ai/sdk`). `packages/core/src/types.ts` defines `LoreMessage` (discriminated union on `.role`), `LorePart` (union of text/reasoning/tool/generic parts), `LoreToolState`, and `LLMClient` interface (single `prompt(system, user, opts)` method). `LoreGenericPart` has `type: string` catch-all which breaks discriminant narrowing — always use exported type guards `isTextPart`/`isReasoningPart`/`isToolPart` from `./types` for runtime checks (used throughout `gradient.ts` and `temporal.ts`). OpenCode adapter at `packages/opencode/src/llm-adapter.ts` implements `LLMClient`, wrapping `client.session.create()` + `client.session.prompt()`, owning agent-not-found retry and session rotation. Core modules (distillation/curator/search) accept `llm: LLMClient` param. Boundary casts use `as unknown as` since Lore and SDK part types are structurally compatible at runtime.

• Knowledge entry distribution across projects — worktree sessions create separate project IDs: Knowledge entries are scoped by project_id from ensureProject(projectPath). OpenCode worktree sessions (paths like ~/.local/share/opencode/worktree/<hash>/<slug>/) each get their own project_id. A single repo can have multiple project_ids: one for the real path, separate ones per worktree session. Project-specific entries (cross_project=0) are invisible across different project_ids. Cross-project entries (cross_project=1) are shared globally.

• Lore search pipeline: FTS5 with AND-then-OR fallback and RRF fusion: Lore search pipeline (`src/search.ts`): FTS5 with AND-then-OR fallback + RRF fusion. `ftsQuery()` builds AND queries (primary); `ftsQueryOr()` builds OR fallback when AND returns zero. Conservative stopword list keeps domain terms like 'handle', 'state', 'type'. `bm25()` column weights: title=6, content=2, category=3 (rank negative — more negative = better). Recall tool uses `reciprocalRankFusion<T>(lists, k=60)` across knowledge/temporal/distillation sources. `forSession()` uses OR-based BM25 since it ranks all candidates; safety net: top-5 project entries by confidence always included. `distillation_fts` added in migration v7.

• Lore temporal pruning runs after distillation and curation on session.idle: In src/index.ts, session.idle awaits backgroundDistill and backgroundCurate sequentially before running temporal.prune(). Ordering is critical: pruning must not delete unprocessed messages. Pruning defaults: 120-day retention, 1GB max storage (in .lore.json under pruning.retention and pruning.maxStorage). These generous defaults were chosen because the system was new — earlier proposals of 7d/200MB were based on insufficient data.

• Lore's 5 background LLM calls are all single-turn prompt-response: All Lore background LLM work (5 call types via `promptWorker()` + 1 auto-recovery) is single-turn: one user message in, one text response out, session rotated after each call. No multi-turn, no tool calling, no agentic loops. Call types: distillSegment (XML observations), metaDistill (XML consolidation), curator run (JSON ops), consolidate (JSON ops), expandQuery (JSON strings, 3s timeout). All parse text responses via regex/JSON.parse. This means decoupling from OpenCode SDK is straightforward — replace `session.prompt()` with direct provider API `fetch()` calls. Embedding calls (`src/embedding.ts`) already bypass the SDK with direct `fetch()` to Voyage/OpenAI APIs.

• LTM injection pipeline: system transform → forSession → formatKnowledge → gradient deduction: LTM injected via experimental.chat.system.transform hook. getLtmBudget() computes ceiling as (contextLimit - outputReserved - overhead) * ltmFraction (default 10%, configurable 2-30%). forSession() loads project-specific entries unconditionally + cross-project entries scored by term overlap, greedy-packs into budget. formatKnowledge() renders as markdown. setLtmTokens() records consumption so gradient deducts it. Key: LTM goes into output.system (system prompt) — invisible to tryFit(), counts against overhead budget.

• Monorepo structure: @loreai/core + opencode-lore packages with Bun workspaces: Bun workspace monorepo with three packages: `packages/core/` (`@loreai/core`, runtime-agnostic, esbuild build → `dist/node/` + `dist/bun/`), `packages/opencode/` (`opencode-lore`, OpenCode plugin, ships raw TS — build is no-op echo so `bun --filter '*' build` succeeds uniformly; OpenCode's loader runs TS directly under Bun), `packages/pi/` (`@loreai/pi`, Pi extension, ~132KB esbuild bundle). Root `package.json` is private with `workspaces: ["packages/*"]` but MUST have `main` and `exports` pointing to `./packages/opencode/src/index.ts` — trampoline required because OpenCode's `file:///` plugin loader resolves from repo root; without it plugin loading silently fails. Declarations via `tsc -p tsconfig.build.json` into `dist/types/`, full tree copied to both target dirs for barrel re-exports. Tests run via `bun test` from root with preload at `packages/core/test/setup.ts`.

• OpenCode plugin SDK has no embedding API — vector search blocked: The OpenCode plugin SDK (`@opencode-ai/plugin`, `@opencode-ai/sdk`) exposes only session/chat/tool operations. There is no `client.embed()`, embeddings endpoint, or raw model inference API. The only LLM access is `client.session.prompt()` which creates full chat roundtrips through the agentic loop. This means Lore cannot do vector/embedding search without either: (1) OpenCode adding an embedding API, or (2) direct `fetch()` to provider APIs bypassing the SDK (fragile — requires key extraction from `client.config.providers()`). The FTS5 + RRF search infrastructure is designed to be additive — vector search would layer on top as another RRF input list, not replace BM25.

• SQLite #db/driver subpath import for Bun/Node dual-runtime: Core package uses Node subpath imports (`#db/driver` in `package.json`) to resolve `bun:sqlite` or `node:sqlite` at runtime. `driver.bun.ts` re-exports `Database` from `bun:sqlite` + `sha256` via `node:crypto`. `driver.node.ts` extends `DatabaseSync` from `node:sqlite` adding a `.query(sql)` shim with WeakMap-based statement caching — API parity with `bun:sqlite`'s `.query()`. All 99+ `.query()` call sites work unchanged. Tests run under Bun; esbuild bundles use `conditions: ["node"]` or `["bun"]` to select the driver. API differences: `.query()` vs `.prepare()`, `{ create: true }` is Bun-only. FTS5, transactions, pragmas work identically. `node:sqlite` is stable without flags in Node 22.5+. No native addons needed; Drizzle adoption orthogonal — FTS5/BM25 queries don't benefit from ORM.

• Worker session prompt helper with agent-not-found retry: `packages/core/src/worker.ts` owns only host-agnostic tracking: `workerSessionIDs` Set, `isWorkerSession()`, `parseSourceIds()`. The `promptWorker()` helper with OpenCode SDK session lifecycle (session.create/prompt, agent-not-found retry, session rotation) moved to `packages/opencode/src/llm-adapter.ts` as the OpenCode implementation of `LLMClient`. Core modules (distillation, curator, search) call `llm.prompt(system, user, { model, workerID })` — no direct SDK access. Each adapter `prompt()` creates a fresh worker session internally (rotation).

Decision

• Curator prompt scoped to code-relevant knowledge only: CURATOR_SYSTEM in src/prompt.ts now explicitly excludes: general ecosystem knowledge available online, business strategy and marketing positioning, product pricing models, third-party tool details not needed for development, and personal contact information. This was added after the curator extracted entries about OpenWork integration strategy (including an email address), Lore Cloud pricing tiers, and AGENTS.md ecosystem facts — none of which help an agent write code. The curatorUser() function also appends guidance to prefer updating existing entries over creating new ones for the same concept, reducing duplicate creation.

• Lore standalone ACP server using Pi as agentic engine: Lore is planned to become a standalone ACP (Agent Client Protocol) server, independent of OpenCode. Architecture: Lore speaks ACP to editors (Zed, JetBrains), uses Pi (`@mariozechner/pi-coding-agent`) internally as the agentic loop engine, and layers its memory system via Pi extensions. ACP proxy approach was rejected because proxies cannot modify the downstream agent's internal message array or system prompt — losing gradient context management and LTM injection, Lore's most valuable features. As a full ACP agent, Lore owns the LLM interaction with full control. Pi was chosen for its extension hooks (message injection, history filtering, custom compaction, custom tools) that map to Lore's existing OpenCode hooks. Requires a research spike first to verify Pi's extension API compatibility.

Gotcha

• Calibration used DB message count instead of transformed window count — caused layer 0 false passthrough: Fixed bugs to watch: (1) Calibration must use transformed window count via `getLastTransformedCount()`, not DB message count — delta≈1 → layer 0 passthrough → overflow. (2) `actualInput` must include `cache.write` — cold-cache ~3 tokens otherwise falls to layer 0. (3) Trailing pure-text assistant messages cause Anthropic prefill errors; drop loop must run at ALL layers (layer 0 shares ref with output). Never drop messages with tool parts (`hasToolParts`) — causes infinite loops. (4) Unregistered projects get zero context management → stuck compaction loops; recovery deletes messages after last good assistant message.

• Lore auto-recovery can infinite-loop without re-entrancy guard: Three v0.5.2 bugs causing excessive background LLM requests: (1) session.error handler injected recovery prompt → could overflow again → loop. Fix: `recoveringSessions` Set as re-entrancy guard. (2) Curator ran every idle because `onIdle || afterTurns` short-circuited (onIdle=true). Fix: `||` → `&&`. Lesson: boolean flag gating a numeric threshold needs AND. (3) `shouldSkip()` fell back to `session.list()` on unknown sessions. Fix: remove list fallback, cache in `activeSessions`.

• Published package main/types must point to dist, not src: Workspace `package.json` `main`/`types` must point to built output (e.g. `./dist/node/index.js`, `./dist/node/index.d.ts`) — published consumers get the tarball, not the workspace, and Node can't execute `.ts`. Workspace-internal consumers resolve via tsconfig `paths` mapping `@loreai/core` → `../core/src/index.ts`, bypassing package.json entry points. The `exports` field with `bun` condition routes Bun users to source. Dual strategy: `main`/`types` for published users (dist), tsconfig paths for workspace dev (src).

• Test DB isolation via LORE_DB_PATH and Bun test preload: Lore test suite uses isolated temp DB via `packages/core/test/setup.ts` preload (`bunfig.toml` at repo root: `preload = ["./packages/core/test/setup.ts"]`). Preload sets `LORE_DB_PATH` to `mkdtempSync` path before any imports of `src/db.ts`; `afterAll` cleans up. `src/db.ts` checks `LORE_DB_PATH` first. `agents-file.test.ts` needs `beforeEach` cleanup for intra-file isolation and `TEST_UUIDS` cleanup in `afterAll` (shared with `ltm.test.ts`). Tests covering OpenCode-specific code (plugin init, recovery functions) live in `packages/opencode/test/`. Driver-level tests in `packages/core/test/db-driver.test.ts`.

• TypeScript can't resolve @loreai/core via Bun conditions — use tsconfig paths: TypeScript @loreai/core resolution — two tsconfigs needed. (1) Dev/typecheck tsconfig must map `@loreai/core` → `../core/src/index.ts` via `compilerOptions.paths` because tsc doesn't understand the `bun` export condition and falls to `default` → `dist/node/index.js` which doesn't exist pre-build. (2) Build tsconfig (`tsconfig.build.json`) MUST NOT include this mapping — it pulls core's src into the consumer's rootDir, causing TS6059 errors; relies on node_modules resolution against pre-built `@loreai/core/dist/`, requiring core be built first. Also: core build must copy the entire `dist/types/` tree (not just `index.d.ts`) into `dist/node/` and `dist/bun/`, because barrel `export * as foo from './foo'` needs peer `.d.ts` files. Clean stray `.d.ts` leaks with `find packages/*/src -name '*.d.ts' -delete`.

Pattern

• Dual-publish same package under two names via tarball mirror: To publish identical package content under two npm names (e.g. canonical `@scope/name` + legacy alias), pack the canonical once, then extract/rename/repack the mirror. Steps in CI: (1) `bun pm pack` canonical → `scope-name-X.Y.Z.tgz` (bun rewrites `workspace:*` deps to concrete versions). (2) Extract to tempdir, modify `package.json` `name` field via `jq`, (3) `tar czf` repack as legacy-name tarball. Both tarballs have byte-identical source content, differing only in `name`. In `.craft.yml`, add an explicit non-workspace npm target for the legacy name with `includeNames: /^legacy-name-\d.*\.tgz$/` — craft's per-target regex prevents cross-contamination with the workspace-discovered canonical target. Verify no regex overlap: canonical's `/^scope-name-\d.*\.tgz$/` won't match `legacy-name-*` and vice versa.

• Lore logging: LORE_DEBUG gating for info/warn, always-on for errors: src/log.ts provides three levels: log.info() and log.warn() are suppressed unless LORE_DEBUG=1 or LORE_DEBUG=true; log.error() always emits. All write to stderr with [lore] prefix. This exists because OpenCode TUI renders all stderr as red error text — routine status messages (distillation counts, pruning stats, consolidation) were alarming users. Rule: use log.info() for successful operations and status, log.warn() for non-actionable oddities (e.g. dropping trailing messages), log.error() only in catch blocks for real failures. Never use console.error directly in plugin source files.

• Lore release process: craft + issue-label publish: Craft publishes 4 tarballs from 3 workspace packages: `@loreai/core`, `@loreai/opencode`, `@loreai/pi` (workspace-discovered) + `opencode-lore` (legacy mirror, explicit non-workspace target). `.craft.yml` has two npm targets: one with `workspaces: true` + `includeWorkspaces: /^@loreai\//`, another with `id: opencode-lore` + `includeNames: /^opencode-lore-\d.*\.tgz$/`. Per-target `includeNames` regex in craft's `BaseTarget.getArtifactsForRevision` routes each tarball to exactly one target (no cross-contamination). CI pack step: `bun pm pack` each workspace, then extract `loreai-opencode-*.tgz`, swap `name` field to `opencode-lore` via jq, repack as `opencode-lore-*.tgz` — `workspace:*` deps already rewritten to concrete versions by bun. CRITICAL: use `bun pm pack` not `npm pack`. Bump via `npm version --workspaces --include-workspace-root`. Flow: trigger release.yml → craft issue → 'accepted' label → publish.yml runs craft with npm OIDC.

• PR workflow for opencode-lore: branch → PR → auto-merge: All changes (including minor fixes and test-only changes) must go through a branch + PR + auto-merge, never pushed directly to main. Workflow: (1) git checkout -b <type>/<slug>, (2) commit, (3) git push -u origin HEAD, (4) gh pr create --title "..." --body "..." --base main, (5) gh pr merge --auto --squash <PR#>. Branch name conventions follow merged PR history: fix/<slug>, feat/<slug>, chore/<slug>. Auto-merge with squash is required (merge commits disallowed). Never push directly to main even for trivial changes.

• Recall logic extracted to core, thin tool wrappers per host: Recall search+format logic lives in `packages/core/src/recall.ts` as host-agnostic `runRecall({projectPath, sessionID, query, scope, llm, knowledgeEnabled, searchConfig})` returning a formatted markdown string. Host adapters wrap it in their tool framework: `packages/opencode/src/reflect.ts` uses `tool({args, execute})` with OpenCode's Zod-ish schema; `packages/pi/src/reflect.ts` uses `pi.registerTool({parameters: Type.Object(...), execute})` with Typebox. Both adapters are ~75 lines — all BM25/FTS5/RRF fusion, vector search, cross-project discovery, lat.md section search stays in core. Pattern applies to any future host (ACP, CLI): keep logic in `@loreai/core`, write a thin tool-framework wrapper per host.

Preference

• Code style: No backwards-compat shims — fix callers directly. Prefer explicit error handling over silent failures. Derive thresholds from existing constants rather than hardcoding magic numbers. In CI, define shared env vars at workflow level, not per-job. Dry-run before bulk destructive operations (SELECT before DELETE). Prefer `jq`/`sed`/`awk` over `node -e` for JSON manipulation in CI scripts.