An Agent's Ceiling Isn't the Model — It's Your Team's Knowledge: A Working Knowledge-Flywheel Playbook
Adapted from a practice write-up by Tencent's technical engineering team, "An Agent's Ceiling May Not Be the Model, But the Team's Knowledge" (author: danteyang). The methodology skeleton is preserved; content unrelated to the technical substance has been removed.
Once a team starts using agents for both engineering and operations, one question surfaces fast: the quality of an agent's output is a direct function of the quality of knowledge you can feed it. Making knowledge "producible, findable, injectable, and retirable" is an engineering problem every agent-enabled team has to solve.
I. Why the AI Era Needs a Rebuilt Knowledge Base
Knowledge used to be scattered across requirement docs, review notes, code review comments, and individual experience. A new hire trying to understand a module's historical decisions would dig through the wiki, ask around, then comb commit messages — easily a day or two of work. Worse, different people kept hitting the same pitfalls, because the lessons stayed locked in the head of whoever filed that one MR.
The old model of knowledge management was simply "a person writes it → a person finds it → a person reads it."
Now that agents are on the front line, the problem has shifted. It's not that agents can't read long documents — a few thousand characters is nothing for a modern context window. What actually blocks things is two distinct issues:
First, retrieval precision. With thousands of documents, how does an agent pull the two or three genuinely relevant ones out of the pile for a specific question? Long-form natural-language writing has fuzzy semantic boundaries, so recall drifts easily.
Second, injection efficiency. Even when the right document is found, maybe only one paragraph of a long retrospective is relevant to the current situation — you can't stuff the whole document in every time. Tokens cost money, and excess context is noise.
Structured knowledge — cards with explicit applicability conditions, core conclusions, and do's/don'ts — solves both problems. But there's an easy way to go wrong: don't build two separate systems, one "for humans" and one "for agents." Good structured knowledge is just as readable and maintainable when a human opens it — the same way a well-written skill doc or prompt reads clearly to a person too. Conversely, if a piece of knowledge looks baffling to a human reader, its maintenance quality won't hold up either, and it will rot over time. The guiding principle is human-and-agent co-reading: one knowledge base, with both humans and agents as consumers.
| ❌ Traditional model | ✅ AI-era model | Why it has to change |
|---|---|---|
| Written by humans, read by humans | Produced automatically by process, consumed by agents, corrected by human judgment | Relying on people to write it always fails; only process guarantees continuous output |
| Long documents, written only for human readability | Human-and-agent co-reading — comfortable for people, precisely retrievable for agents | Structure makes retrieval precise and injection efficient, while staying human-legible keeps it maintainable and slows decay |
| Static storage, only additions, no removals | Self-reinforcing flywheel — freshness windows, automatic retirement and revival | At scale, stale content actively interferes with an agent's decisions |
In one line: a traditional knowledge base is a "warehouse" optimizing for volume. An AI-era knowledge foundation is a "supply system" optimizing for precise matching, fast injection, and automatic retirement of what's gone stale.
II. Define It First: Who It Serves, What It Holds, What Success Looks Like
Most failed knowledge-foundation projects aren't technical failures — they're failures to think the goal through. Jumping straight to picking tools and building a platform usually ends with something nobody uses. Answer four questions before writing a line of code:
| Question | What it means |
|---|---|
| What scenario does it serve? | Not "I want to build a knowledge base," but "I want to fix low efficiency at YY in scenario XX" — anchor the goal to an actual business pain point |
| Who consumes it, and how? | Both humans and agents are usually consumers. The structure has to satisfy both: understandable and maintainable by people, precisely retrievable and injectable for agents |
| What goes in it? | Inventory across four layers: L1 common foundation → L2 business domain → L3 scenario strategy → L4 incident increments. Each layer needs clear content, an owner, and a refresh cadence |
| What does success look like? | Observable metrics: injection hit rate, profile coverage, blind-search decline rate, rework decline rate, conversation adoption rate, etc. |
Once the goal is clear, the rest of the design falls into place. For instance, the judgment that "relying on humans to capture knowledge always fails" directly implies binding knowledge capture to the key checkpoints of the engineering workflow — once the code ships, the knowledge is left behind automatically.
One hard rule follows from this: knowledge that hasn't shipped, or isn't linked to a repo, stays quarantined and never enters the agent's reference pool — it's only promoted automatically once it ships. Without this gate, the knowledge base quickly drowns in half-finished, invalid documents.
The same idea applies whether you're doing engineering or operations: find the "mandatory checkpoints" in your workflow (tests passed, shipped to production, incident closed...) and attach knowledge capture to them. Knowledge becomes a byproduct of the process instead of extra work. The dual-gate admission logic (only objectively-verified signals get in) is universal.
III. Six Steps to Build an AI Knowledge Foundation
Don't start by picking tools. Six steps, in order, each with a concrete deliverable, keep the project from stalling out halfway.
Step 1: Inventory — what do you already have
Take stock first, sorted into four layers:
| Layer | Content | Examples |
|---|---|---|
| L1 Common foundation | Org-wide dev standards, coding conventions | Code samples for logging/storage/RPC/auth, etc. |
| L2 Business domain | Domain knowledge organized by business line | Platform docs, integration guides, business architecture |
| L3 Scenario strategy | Decision rules and strategies for specific scenarios | Handling standards, prompt templates for controls |
| L4 Incident increments | New rules/lessons distilled from day-to-day incidents | New standards captured after an incident; pitfall cards auto-extracted after a feature ships |
The inventory reveals the gaps and the overlaps, and from there you can shape your own layered structure — one MCP capability foundation, some number of knowledge bases, and however many agent consumers you need. You don't need many bases, but the layered-inventory method transfers directly.
Step 2: Choose the shape — what kind of knowledge base you actually need
Knowledge bases come in three shapes, each good at different things. The key lesson: don't build just one — combine them.
| Type | Good for | Example |
|---|---|---|
| Document-style | Long-form background, tutorials, architecture explanations | Wiki docs + onboarding area + long-form retrospectives |
| Structured storage | Clear rules needing exact matches, while still staying human-legible | Git/database-backed storage, markdown/YAML/JSON-rendered knowledge cards |
| RAG vector store | Large volume, mostly semantic matching | Vector-retrieval storage |
Step 3: Pick the tooling — platform first, adapter layer on top
The logic is simple: use an internal platform if one exists; build only if it doesn't. The agent's tool layer needs an adapter that packages underlying data capabilities into standardized tools — MCP (Model Context Protocol) is currently the dominant approach.
Split underlying data capabilities (database queries, log search, model calls, etc.) into multiple independent MCP servers by business domain — this is the key design decision. Don't pile every tool into one server: models have a cap on how many tools a single server can expose, too many tools burn extra tokens, and different business lines need isolation anyway. Once split, each agent connects only to the tool group it actually needs, without interfering with the others.
MCP (Model Context Protocol): a standard protocol letting agents call external tools — think of it as the USB port of the agent world. RAG (Retrieval-Augmented Generation): search the knowledge base first, then let the model answer using what it found.
Step 4: Produce knowledge — four modes running in parallel
One hard rule: don't count on anyone writing it voluntarily. Four production modes can run side by side:
| Mode | Description |
|---|---|
| A: Auto-captured by workflow | Knowledge production is bound to key engineering/ops checkpoints, zero manual effort |
| B: Driven by a retrospective agent | An agent analyzes conversation logs weekly; blind spots that missed a hit turn into an automatic MR — every user conversation implicitly tests knowledge quality |
| C: Agent auto-generation | Ops feeds in incident context, the agent turns it into structured knowledge plus a precision/recall check; time-to-live drops from days to minutes |
| D: Instant capture during ops handling | Rules get captured while an incident is being handled, so the very next similar incident can use them immediately |
Step 5: Govern the knowledge — freshness requires both entry and exit
A knowledge base with only additions and no removals turns into a junkyard within months. Governance needs four things:
- Dual-gate admission: only knowledge that has shipped and is linked to a repo formally enters the agent's reference pool
- Three-tier quality grading: directly usable / needs confirmation / needs verification, auto-assigned by confidence
- Freshness windows: re-validation required after a default period; anything past its window with zero references is auto-archived
- Auto-retirement with a safety net: expired-and-unreferenced entries get archived, but anything still referenced or endorsed gets automatically revived — preventing valuable knowledge from being killed by mistake
A governance dashboard lets knowledge owners see what's pending review, about to expire, or incomplete, so manual effort concentrates on high-value review only — the system metabolizes itself without anyone needing to watch it constantly.
Step 6: Distribute and consume — get knowledge to whoever needs it
The traditional approach relies on the agent searching for itself — if it can't find it, that's that. A better approach is the platform actively injecting knowledge: before an agent starts a task, the system has already packaged the relevant knowledge for that scenario and handed it over.
| Traditional model | Active-injection model |
|---|---|
| Agent searches on its own (uncertain whether it'll even call it) | Platform actively injects (100% guaranteed delivery) |
| Long documents, hard for an agent to consume | Structured cards / YAML / prompt templates |
| No usage record | Injection is logged, so every piece of knowledge's consumption is traceable |
Each agent has its own knowledge subscription — a requirements-review agent only sees requirement templates and historical pitfalls, an ops agent only sees judgment standards and bad-case rules. Knowledge packages act as the distribution container, organized as either repo-specific or shared packages, never broadcast indiscriminately.
What can be reused directly, and what needs customization?
Directly reusable: the four-question goal-framing exercise, the idea of binding capture to workflow checkpoints, dual-gate admission, three-tier quality grading, freshness-plus-retirement-and-revival, knowledge packages as distribution containers, and logging injections.
Needs adaptation: the actual trigger checkpoints should map to your own business's key workflow steps; the field structure of knowledge cards should be designed around how your agents actually consume them; how you split MCP tools depends on your own business domains; and the freshness cadence should be tuned to how fast your knowledge actually changes.
IV. The Flywheel That Keeps Knowledge Moving
Building the knowledge base is step one. Getting it to spin on its own is what actually makes it work: more delivery → richer knowledge → stronger agents → faster delivery. Once this loop starts, a team's capability scales with usage itself.
The flywheel has six stages: produce → distill → inject → consume → feed back → refresh — a self-reinforcing loop.
A few design choices keep the flywheel spinning:
Accumulation through use. A retrospective agent scans conversation logs weekly, and any question it couldn't answer automatically becomes an MR to fill the gap — every user conversation is implicitly helping patch the knowledge base's blind spots.
Injection is logged automatically. Who used which piece of knowledge, and how many times, is recorded by the system, not reported by hand — which entries are popular and which have never been touched is visible on a dashboard at a glance.
It's fine to decide not to capture something. Plenty of finished work genuinely has nothing worth capturing, and the system is allowed to skip it — density matters more than volume for a knowledge base.
There's also the operational question of how to get people to actually want to participate. The most effective lever isn't a management mandate — it's building tooling so that knowledge capture itself doesn't feel like extra work. For example, during the back-and-forth of an end-to-end feature build, knowledge gets captured passively as a side effect of the human-AI conversation, and gets attributed automatically to the right person. A contribution leaderboard and self-review citations are nice bonuses, but the core mechanism is still workflow automation doing the actual work.
Here's the flywheel dashboard the team actually uses day to day — knowledge volume, quality, freshness, and agent-usage effectiveness are all visible on one screen, all of it generated automatically by the "injection is logged" mechanism:
Anti-patterns to avoid:
| Anti-pattern | Consequence → fix |
|---|---|
| Forcing contribution via KPIs | Knowledge written under pressure is low quality and maintained even less willingly → automation should carry the load, incentives are secondary |
| Unlimited growth with no exit path | Knowledge piles up, signal-to-noise keeps dropping → freshness windows plus auto-retirement/revival |
| Tacit knowledge never made explicit | An expert leaves and takes all the know-how with them → dedicated retrospectives, handover checklists, and expert knowledge extraction |
| Contributors and organizational recognition out of balance | People write a lot but get no recognition, and motivation naturally fades → quantified leaderboards, citable self-reviews, and management performance signals |
V. What Actually Changes Once It's Running
The short version of what changes once a knowledge system is in place: people stop doing repetitive searches and repeating the same mistakes, and get to spend that time on judgment and creative work instead. Concretely, there are four layers of value:
| Value | Description |
|---|---|
| Organizational memory | Staff turnover no longer takes critical knowledge with it; incident retrospectives become an "error log" and a benchmark that lets new people stand on their predecessors' shoulders |
| Faster ramp-up | A three-stage onboarding system (orientation → growth → output) meaningfully compresses the time it takes a new hire to become productive |
| Knowledge equity | Regardless of team or tenure, the knowledge an agent can access is the same — reducing the "you only get the answer if you ask the right person" information asymmetry |
| Self-reinforcing flywheel | More use → richer knowledge → stronger agents → higher output → more people willing to use it — a continuously accelerating positive loop |
More important than any single number is keeping the system actually running — this can't be a one-time "knowledge campaign" that goes still afterward. Knowledge volume needs to keep growing, freshness needs to be maintained by automation rather than someone watching it, and agent call frequency needs to actually reflect that the knowledge is being consumed.
The human role changes too: not a mover of knowledge anymore, but the quality gatekeeper. Agents handle the repetitive bulk of the work; humans handle judgment and correction. The cognitive load is low — do the work normally, ship normally, and knowledge capture just happens alongside it, with no extra "go write documentation" step required.
If your team is building out a knowledge system, start from one concrete scenario's closed loop rather than trying to cover everything at once. Get one working, then expand horizontally. Getting the flywheel spinning matters more than spinning it fast.