Run a research agent on your codebase

Decide what 'the codebase' means: one repo, a monorepo with N packages, or a multi-repo system. Be explicit, the agent's job description depends on it. Then write the read-only rule in plain text: the agent reads, summarizes, cites, and writes only to the workspace docs. It does not edit code, does not run scripts that modify state, does not commit, does not push. Lock this in before you give the agent a runtime.

The biggest mistake is to ask the agent specific questions before it has a high-level map. The agent will answer with whatever files it grep'd, often missing the canonical implementation. Instead, ask the agent to produce a top-down architecture map first: services, modules, key abstractions, data flow. Save that to the Brief, refer back to it in every subsequent question.

Read the codebase top-down.

Output an architecture map as a Brief section titled "Architecture map v1":
1. System overview (1 paragraph: what this codebase is, who uses it)
2. Modules / services (5-15 entries, each with: name, purpose, entry-point files)
3. Data flow (how a typical request / event moves through the system)
4. Load-bearing files (5-10 files that are hit constantly + a 1-line note on why)
5. Open questions (things you couldn't determine from reading alone)

Constraints:
- Cite file paths for every claim. If you can't cite a path, mark it "(inferred)".
- Don't editorialize. Describe what's there, not what should be.
- Confidence level per module: high / medium / low. Low-confidence modules go in "Open questions" too.

Every research question the agent answers should land in a structured log: question, answer, file paths cited, confidence (high / medium / low), open follow-ups. The log is the durable artifact, the chat history disappears, the log persists. Make the agent write to the log on every answer.

Not every question is a research question. The agent should answer: 'where is this implemented?', 'why is this designed this way?', 'what's the data flow for X?', 'what are the test patterns?'. The agent should refuse: 'fix this bug', 'add a feature', 'refactor X', 'update the schema'. Refusal isn't lazy, it's the boundary that keeps the read-only scope intact.

Pick 10 real questions you have about the codebase. Run them through the agent. Spot-check every answer against the source files. Compute accuracy: how many high-confidence answers were actually right? How many flagged confidence levels matched the actual evidence quality? This calibration loop is what separates a useful research agent from a confident-bullshit machine.

Run the calibration pass on the 10 pilot questions.

For each question + answer pair in the Question log:
1. Re-read the cited file paths.
2. Verify the answer matches the source code.
3. Update the row with: verified (correct / partial / hallucinated), notes on what was wrong.

Then output a calibration report as a Brief section titled "Pilot calibration v1":
- Accuracy by confidence bucket (high / med / low)
- Common failure modes (wrong file, missing context, over-confident)
- 3 specific system-prompt tweaks to apply, ordered by expected lift

Constraint: be brutal in the assessment. A confidently wrong answer is worse than a 'I don't know.'

An agent nobody asks questions of is dead. Wire it into the workflow: a Slack channel, a recurring #ask-the-codebase ritual, a 'before-you-ask-a-senior-engineer' default. Give it a name, give it a place. The teams that get the most out of research agents treat them as a junior team member with infinite patience for 'newbie' questions.

Every Friday, the agent produces a written explainer covering the architectural patterns that emerged from the week's questions. Over time, these explainers become the de facto codebase documentation: ground truth, structured, version-controlled. Better than the README that hasn't been updated since 2022.

Read the last 7 days of the Question log surface.

Identify the most-asked-about architectural theme. ("What were the top 3 file paths cited this week? What domain did the questions cluster around?")

Write a weekly explainer as a Brief section titled "Weekly explainer <week-of>":
1. Theme (1 sentence)
2. Why it came up this week (1 paragraph)
3. The architectural pattern (2-3 paragraphs, with file path citations)
4. Open questions / known gaps
5. Suggested next read-through if the team wants to deepen on this

Constraints:
- Cite real file paths for every claim. If you can't cite, drop the claim.
- Write for a senior engineer joining the team, not for marketing.
- Length: 300-600 words, no longer.

The agent's explainers are the first draft, not the final word. Every Monday, spend 30 min reading last week's explainer with the source code in the other window. Correct hallucinations, sharpen claims, add the institutional context the agent doesn't have. Then check the edited version into the team's docs repo. Over time, this is how you build the codebase documentation that always seemed too expensive to maintain.

Once stable, expand: wire in static analysis tools the agent can run (cloc, semgrep, dependency graph), expand from one repo to a monorepo or a multi-repo cluster, integrate with the team's existing docs site. Eventually the agent's job will shrink, the docs the team builds from explainers replace many of the questions. That's success, not failure.