How My AI Agents Learn From Code Review

The question I got most after writing about Ghulam was the practical one: does it get better?

It does. Not because the model improves, but because the system writes things down.

The short version: I comment on a pull request, the same agent that wrote the code amends it on the same branch, and when that PR merges the lesson is recorded somewhere every future task on that project will read it.

One comment, one change

When I leave a review comment, a GitHub webhook hits Ghulam's /github route. The signature gets verified, the comment is matched back to the PR that produced it, and the original coding agent — frontend or backend, whichever wrote the code — is re-dispatched in amendment mode.

Amendment mode is deliberately narrow:

• clone the existing PR branch;
• apply the requested change;
• commit back to the same branch;
• post a completion comment on the PR.

No new branch. No second PR. No re-planning from scratch. Review stays in one thread, and the diff I already looked at stays the diff I'm looking at.

The dispatch carries more than the comment text. It keeps the repo, the comment body, who commented, the original task description, and the PR branch. That last pair matters most — without the original task, an agent amends code whose purpose it no longer knows, and "fix the padding" quietly turns into a redesign.

From one fix to a lasting change

Amending a branch solves today. The interesting part happens on merge.

When the PR lands, the feedback loop records what the agent got wrong: a dated bullet appended to the project brief's ## Recent corrections section. That write is automatic.

There's a second, slower store beside it. Each project repo can carry a corrections.md file holding the standing rules for that codebase. Ghulam reads that file on every task but doesn't write to it — I promote lessons into it by hand, once they've proven they keep coming back.

So the automatic loop captures everything, and the repo file keeps what survived. Plain markdown, in git, reviewable in a diff.

A good correction is specific enough to act on: "don't put network calls in the widget build method" rather than "be more careful."

Where the memory gets used

On the next task for that project, the project resolver fetches each repo's corrections.md and concatenates it into graph state. Each agent then assembles its cached prompt prefix from four sources: the project brief, retrieved project memory from a repo-scoped ChromaDB collection, a bounded set of corrections, and the task spec with its acceptance criteria.

The corrections arrive as an explicit ## Corrections block, and the system prompt is direct about what to do with it:

Your task context includes a ## Corrections block — mistakes from previous tasks on this project that must not be repeated. Read it before starting work.

Retrieval is capped rather than dumping the whole history, so an old project doesn't slowly eat its own context window.

Feedback flows both ways

The part I didn't design and now depend on: corrections don't only travel from human to agent.

During discovery, a coding agent often finds the PM agent assumed something false about the codebase. When that happens, it's required to report spec corrections in the PR description — each assumption it had to overrule, listed explicitly.

So the planning layer gets corrected by the layer that actually touched the code, in writing, on the record. It's the cheapest quality upgrade in the system, because the PM agent is the component most likely to be confidently wrong.

On the input side the PM agent has its own version of this: before planning anything, it returns a structured decision — proceed, or ask one focused clarifying question. Feedback that reframes the goal triggers a revision instead of a guess.

Why a markdown file is the right substrate

At my scale I have dozens of accepted tasks, not millions of trajectories. Nothing about the model changes in this design; the learning lives entirely in the context an agent reads. That's a feature, not a shortcut.

The last two rows are the real argument. When a correction turns out to be wrong, I delete a line. When someone disagrees with the agent's learned behavior, they can read the exact sentence that caused it and change it in a PR.

Agent behavior in Ghulam lives in three readable places, none of them the weights: prompts, loaded at runtime from Langfuse with a short cache so they can be edited without a redeploy; briefs and corrections in markdown; and project memory in ChromaDB, isolated per project.

What I'm building next

The loop works. These are the edges I'm actively working on.

✂️ Pruning corrections

Corrections accumulate; nothing retires them. Bounded retrieval caps the cost today, but corrections about deleted code should expire on their own.

⚖️ Detecting conflicts

"Always use the shared HTTP client" and a later "use a raw request here" can coexist happily in one file. Conflicts should be surfaced at write time, not resolved silently by whichever agent reads them.

🔁 Closing the write loop

Merge-time corrections land in the project brief automatically. Promoting the recurring ones into a repo's `corrections.md` is still my judgment call — which is fine for now, and a candidate for automation once I trust the signal.

🔒 Promoting corrections into code

A correction that keeps recurring shouldn't stay a sentence in a prompt. It should become a lint rule, a test, or a CI check — something that can't be reasoned around.

✍️ Better review comments

The loop is exactly as good as what I write in the PR. Specific comments teach the next three tasks; vague ones teach nothing. Worth measuring.

The pattern underneath

The agents are stateless between tasks. Everything that looks like learning is a written record, retrieved and injected before the work starts.

That's a small claim, and it's the reason it works: a review comment stops being a one-time fix and becomes part of the project's institutional memory — in a file I can read, edit, and roll back.

The next upgrade isn't a smarter model. It's moving the corrections that keep coming back out of markdown and into code that can't negotiate.