Partner status
- Claude Partner Network member
- Claude Code partner badge — verify it on Credly
- Delivered from India and Germany across global time zones
- Founder-led engagements — senior advisors and the delivery engineers in one team
RAG over a 10MB PDF is a demo. RAG over fifty terabytes of scattered, multi-modal enterprise data is an architecture problem — and a different one than it was two years ago, because the context window stopped being the binding constraint.
1. What a million-token context window actually changed
A great deal of received RAG wisdom was written when the context window was the scarce resource. Chunk small, retrieve few, compress hard — all of it downstream of a budget that no longer bites the same way. Claude models now carry a one-million-token context window, which is roughly a mid-sized codebase or a few thousand pages.
This does not make retrieval obsolete. It moves the problem. You are no longer optimizing to fit; you are optimizing for precision and cost, because everything you put in the window is something the model has to weigh and something you pay for. Filling a million tokens with loosely relevant material produces worse answers than five hundred well-chosen ones, and costs considerably more.
The practical consequence: aggressive chunking that shreds a document's structure is now a net loss. It was a workaround for a constraint that has largely lifted, and it takes coherence with it.
2. Routing before retrieving
The scaling failure everyone hits is the precision wall. Past a few million chunks in a single index, similarity search starts returning things that are statistically close and semantically useless, and answer quality falls even though nothing in the pipeline errored.
The fix is to stop treating retrieval as one undifferentiated lookup. Shard by domain and put a routing step in front, so a question about a contract never searches the telemetry index at all. Each source is exposed to the agent as its own MCP server — the same boundary used for tools elsewhere, which means access control, credentials and audit are per-source rather than global.
graph TD
Q[User question] --> R[Routing step: which sources apply]
subgraph SRC["Sources - one MCP server each"]
direction LR
S1[Documents: vector]
S2[Logs: time-series]
S3[Records: read-only SQL]
S4[Graph: entity edges]
end
R --> SRC
SRC --> RK[Re-rank and assemble context]
RK --> C[Claude synthesizes with citations]
C --> A[Answer plus source attribution]
Routing is also the main cost lever. A question answerable by an indexed SQL lookup should never become a distributed vector search, and a router that recognises the difference is usually the single largest saving in the system.
3. The retrieval problems that did not go away
3.1 Vectors are bad at relationships
Vectors find things that mean similar things. They cannot answer "who approved the merge request that caused the outage", because that is a traversal, not a similarity. Where those questions matter, a knowledge graph alongside the vector store handles the deterministic edges while vectors handle the fuzzy matching — exposed as two MCP servers, so the agent chooses per question rather than a pipeline choosing for it.
3.2 Parent–child chunking
Search the small chunk, return the large one. Embedding at paragraph granularity keeps retrieval sharp; returning the parent document keeps the model's context coherent. With a million-token window there is rarely a reason to hand back the fragment alone.
// Parent-child retrieval record
{
"parent_doc_id": "pol_2026_sec_framework",
"child_chunk_id": "pol_chk_004",
"matched_on": "All encryption keys must be
rotated every 90 days...",
"return": "full parent document, with
section path preserved"
}
3.3 Re-ranking still earns its place
Vector search is fast and blunt. Retrieve broadly, then re-score the candidates against the actual question before assembling context. This is the cheapest quality improvement available in most pipelines, and it matters more rather than less as the window grows — a large window makes it easy to include weak candidates you would previously have been forced to drop.
3.4 Citations are an architecture requirement
For enterprise use, an answer without attribution is unverifiable and therefore unusable in any regulated process. Claude's citation support ties assertions back to the passages they came from, which turns spot-checking from a re-read of the whole source into a glance. It also gives you a measurable quality signal: an answer whose citations do not support it is a specific, catchable failure rather than a vague sense that the system is drifting.
4. How we evaluate it
Retrieval quality is not a feeling. Build a set of questions with known-good answers and known-good sources, and measure two things separately: did retrieval surface the right material, and did the model use it correctly. Conflating them hides which half is broken — and they fail for entirely different reasons and get fixed in entirely different places.
5. What this delivers
- Precision over volume: routing and re-ranking mean the model reasons over relevant context, not merely abundant context.
- Cost that scales sublinearly: cheap sources answer cheap questions; distributed vector search is reserved for questions that need it.
- Per-source governance: MCP boundaries make access control, credentials and audit a property of each source rather than of the whole system.
- Verifiable answers: citations make correctness checkable, which is what makes the system usable where being wrong has consequences.