Stolperfalle.

AI coding agents forget. Every session starts from nothing, so the trap one session fell into is waiting, unmarked, for the next: the silent misconfiguration, the API that behaves differently from its docs, the tool that fails without an error. Run agents across many repositories and the same lesson gets learned again and again.

Stolperfalle is German for trip hazard. The tool marks the spot so the next agent steps around it.

Until July 2026 it was called Stolperstein. Stolpersteine are Gunter Demnig’s brass Holocaust memorial stones, set into German pavements. A dev tool has no business borrowing that name, so I renamed it.

One store, three paths through it. Six MCP tools (the protocol coding agents use to call outside tools) over one SQLite file: query, propose, reflect, confirm, flag and status. Search runs on full text and local embeddings (search by meaning) together.

The loop, in motion23 Oct → 30 Oct 2026

open the diagram to zoom →

• Writepropose · reflectAn agent proposes what it learned. Or reflect drafts candidates from a session summary, and the agent reviews them before proposing.
• ReadClaude Code hooksWhen a prompt or a failed shell command carries a real error signal (a traceback, a non-zero exit code, an HTTP status), the hook calls query and injects the best-matching unit. Plain conversation doesn’t trigger it.
• Qualityconfirm · flagAgents confirm advice that worked and flag advice that was wrong, stale or duplicated.

The first question to ask about a tool that reads error output. The answer, operation by operation:

Around the data.

• IsolationEvery unit has an owner org, and query only returns units from trusted orgs. Today that is a read filter that trusts all by default; write-side permissions are planned. The clean boundary today is one install per team or client.
• ProvenanceEach install signs its units with its own Ed25519 signing key (a did:key, a public-key identity), stored outside the database. A database leak does not let anyone forge units.
• Running itOne container, about 1 GB with the embedding model, and one SQLite volume. Migrations snapshot the database before any breaking change. reflect costs about one US cent a call: under €30 a month even at 100 calls a day, zero without a key.

The store audit, October 2026.

I had four Claude reviewer agents audit 599 of the store’s roughly 640 non-archived knowledge units, each taking a slice with an index of the whole store. The reviews, the fact-checks and the rewrites were model work. A human decided what to apply.

88% kept is a model’s judgment, not ground truth. The six left in place: four rewrites that couldn’t be applied, because propose rejected each fix as a duplicate of the unit it was fixing, one suspected error that turned out to be right, and one duplicate pair kept on purpose because the two complement each other. The audit also found twelve near-identical units about zsh versus bash, proposed weeks apart. Paraphrases slip past the duplicate check.

Choosing the model behind reflect.

reflect ran on gpt-4o-mini, which OpenAI shuts down on 14 January 2027. Before replacing it I built 12 test cases from real units in the store, each a session summary with the learnings a careful human would extract and the noise that must not be extracted. A separate Claude agent graded six anonymous outputs per case (the seventh configuration failed every call), labels shuffled per case. Each row is a model and its reasoning setting.

A model-name swap would have failed silently.

OpenAI’s named replacement for gpt-4o-mini is a reasoning model and rejects temperature with an HTTP 400. reflect caught every LLM error, logged a warning and fell back to its heuristic, so production would have kept “working” with much worse candidates and nothing watching that warning. Now a 400, 401, 403 or 404 from the provider raises. Only timeouts, server errors (5xx), refusals and unparseable output fall back, they log at error level, and status counts them by reason.

Production now runs claude-sonnet-5-5 at low effort. The cheapest model was the worst value: terse, generic candidates that dropped product names and versions. Sonnet and Haiku found the same 13 learnings; Sonnet had fewer bad candidates, and the cost difference is small in absolute terms, so I picked Sonnet. Opus did not beat Sonnet, at twice the price. The whole comparison cost about $0.36.

Limits, stated in the eval itself: 12 cases, one run per configuration, one grader. The grader was a Claude model, so the gap between the Claude and OpenAI rows may carry some self-preference. A second, non-Anthropic grader was planned and did not run. Read neighboring rows as indicative, not precise. The full write-up is in the repo.

As of October 2026, and listed in the repo’s own overview, because a memory tool that hides its holes is not one you should trust with your errors.

• No redaction on write.Agents can and did write client names and internal hostnames into knowledge units. The audit found them and they were rewritten by hand. Until redaction exists, keep one install per client.
• No hard delete.Flagging a unit archives it but keeps the row. Erasure, say a GDPR Art. 17 request, needs an operator to delete it from SQLite.
• Confidence is mostly unrated.94% of units sit at the default confidence with zero confirmations, because agents rarely call confirm. Treat confidence as weak until it is derived from usage.
• No edit path.A correction is a new unit that supersedes the old one, and a near-identical fix can be rejected as a duplicate of the thing it fixes.
• No measured productivity effect yet.The pilot below would measure it.
• One LLM provider.Extraction in reflect runs on the Anthropic SDK. Switching providers is a code change today.

The pilot that would measure it.

Four weeks, one team, one install. Before and after, on the errors the hooks detect:

• How often the same error signature recurs across sessions.
• How often an injected unit is followed by a successful next command.
• How many units get confirmed or flagged, which shows whether the quality signal starts working.

Stolperfalle is not a fork. Its knowledge units follow Mozilla AI’s open cq schema, a shared format for what agents learn, and stay strictly valid on the wire. The fields I need beyond it ride in an extensions slot, under stolperfalle:* keys.

That slot didn’t exist. I proposed my extensions upstream in discussion #286, scoped it in issue #406 as one generic slot instead of a looser schema, and it merged as PR #453 on 23 June 2026. Any implementation can use it.

Upstream declined to promote severity, kind and contributing orgs into the core schema: importance should emerge from usage, not self-declaration. The October audit points the same way: declared severity skews high, confidence never moves off its default, and duplicates are paraphrases. I posted those findings back to the discussion.

Leaving is cheap. MIT license. One SQLite file. Units serialize to the upstream schema, so they can move to any other cq implementation. The LLM is optional: remove the key and everything except reflect’s draft quality keeps working.