This post is not for beginners. It assumes you have shipped at least one RAG system and ran into its limits. If you are earlier in that journey and want a production RAG guide built from the ground up, DM me on LinkedIn - I will write one for you.
The Problem With One RAG Strategy
Most RAG pipelines follow one path: upload a document, split it into chunks, embed those chunks, and retrieve the nearest matches when someone asks a question. For simple documents this works. For everything else, it doesn't.
A legal PDF structured around cross-referenced clauses, a financial report where the answer lives inside a table, an enterprise architecture document describing chains of system dependencies - none of these should go through the same pipeline. Each has a different structure, and that structure changes what correct retrieval looks like.
That was the core problem I set out to solve while building Zyven Studio - an adaptive RAG engine where the system decides how to ingest each document and how to search it, based on what the content actually looks like.
Try Zyven StudioExplore the adaptive RAG engine - upload a document and see how it routes itselfA RAG system should not just retrieve text. It should understand the shape of knowledge before deciding how to search.Field note · 2026
Why Naive RAG Breaks on Complex Documents
Naive vector search is genuinely good at what it does. Embed text, find similar chunks, return context to the LLM. For FAQs, wiki pages, support articles, and plain meeting notes, this is fast, cheap, and accurate enough.
The failures appear when documents have structure that plain chunking destroys. Split a legal document at 500 tokens and you lose the referential logic that makes a clause meaningful. OCR a PDF report and the table that holds the answer becomes a broken string of characters. Ask a multi-hop question - which team owns the system that depends on this service? - and nearest-neighbor search has no way to follow the chain.
- FAQs, wiki pages, and support articles - naive RAG is accurate and fast
- Legal documents with cross-references - chunking destroys referential context
- PDFs with tables and charts - OCR breaks the structure that carries the answer
- Multi-hop questions - vector similarity cannot follow chains of relationships
- Global questions across many documents - requires summarization, not chunk retrieval
The Architecture: Three Adaptive Layers
Instead of a single pipeline, the system has three adaptive layers. Each has a specific job: the Ingestion Router decides how to process an incoming document, the Query Router decides how to search when a question arrives, and the Fusion and Reranking layer cleans and ranks the combined results before they reach the LLM.
Each layer operates independently. A document routed through Graph RAG can still be queried via the Naive Route for simple factual questions.
1. Ingestion Router - routes documents to the right processing pipeline
2. Query Router - routes questions to the right retrieval strategy
3. Fusion + Reranking - normalizes, deduplicates, and ranks mixed resultsLayer 1: The Adaptive Ingestion Router
The ingestion router is the first decision point. Instead of sending every file through the same pipeline, it inspects the document first - checking file type, document structure, table density, image density, and metadata.
The critical design constraint is avoiding an LLM call for every routing decision. Running inference on every upload is expensive and slow. Instead the router uses heuristics first - file type, table count, image density - and falls back to a lightweight classifier only when heuristics are insufficient. The LLM is a last resort for genuinely ambiguous documents.
The heuristics-first approach keeps ingestion practical at production volume.
Heuristics first (file type, table count, image density)
→ Lightweight classifier if heuristics are insufficient
→ LLM fallback for ambiguous documents onlyThree Ingestion Paths
Path A - Naive RAG. For simple text: notes, FAQs, support articles, wiki pages, and transcripts. The document is split into chunks, embedded, and stored in a vector database. Fast, cheap, and correct for the documents it is designed for.
Path B - Graph RAG. For documents where relationships matter: legal documents, codebases, policies, org charts, and enterprise architecture descriptions. The document is parsed for entities and relationships, stored as a knowledge graph. This enables questions like 'Who owns the service that depends on this system?' - questions that require following connections, not matching similarity.
Path C - Visual Document Retrieval. For documents where the layout carries meaning: financial reports, slide decks, pricing tables, forms, and diagrams. The system processes the page image together with extracted text, tables, and layout metadata - preserving the structure that plain OCR destroys.
Layer 2: The Adaptive Query Router
When a user asks a question, the query router decides which retrieval strategy to use. Not every question needs graph traversal. Not every question needs visual retrieval. Using the wrong strategy wastes compute and returns worse results.
Rather than running a full LLM call to classify every query, the router uses semantic routing: embed the user's question, compare it against predefined intent patterns, and select the best retrieval path. This is significantly faster for real-time systems and avoids the cost and latency of inference on every query.
Five Query Routes
Naive Route: Simple factual questions - 'What is corporate law?' Standard vector search, nearest chunks returned.
Visual Route: Questions about tables, charts, or layouts - 'Extract the pricing table from this PDF.' Visual document retrieval locates the page and extracts the structured data.
Local Graph Route: Relationship-based questions - 'Which team owns the system connected to this service?' The system follows graph edges, not embedding similarity.
Global Graph Route: Broad questions spanning the full knowledge base - 'What are the common risks across all documents?' Graph community summaries, not individual chunks.
Hybrid Route: Complex questions needing multiple evidence types. Vector search, graph traversal, and visual retrieval may run in parallel - results are merged and passed to the fusion layer.
Layer 3: Fusion and Reranking
When multiple retrievers run in parallel, the combined output is noisy - text chunks, graph nodes, table data, and visual summaries mixed together with no clear relevance signal.
The fusion layer normalizes results across formats, removes duplicates, reranks by relevance, and discards low-signal context. Only cleaned, ranked evidence reaches the LLM.
Without this step, the model arbitrates between contradictory context instead of answering. Reranking moves that burden out of generation and into retrieval, where it belongs.
The fusion layer is the interface between retrieval and generation.
Raw results from parallel retrievers
text chunks (vector search)
graph nodes+edges (graph traversal)
table data (visual retrieval)
↓
Fusion Layer
normalize formats
remove duplicates
rerank by relevance
drop low-signal context
↓
Clean, ranked evidence → LLM generationSeeing It in Practice: Zyven Studio
Zyven Studio is the production implementation of this architecture. You can upload a document, paste text directly, or let the system determine the ingestion mode - Instant for simple text, Deep for relationship-heavy documents, or Auto for the router to decide.
One feature that distinguishes it is the Knowledge Atlas - a live visualization of the knowledge graph built from your documents. Most RAG systems hide everything inside a black-box vector store. Making the graph visible lets users see what the system knows, how it connects things, and why an answer is grounded in specific relationships. That transparency is what builds trust in the system rather than just utility from it.
Try Zyven StudioUpload a document and explore the Knowledge AtlasWhat Production RAG Actually Requires
The most common mistake in production RAG is treating document ingestion as a solved step. It isn't. The ingestion strategy determines what the system can retrieve. Flatten a relationship-heavy document into text chunks and no retrieval strategy will recover the connections you destroyed.
The same applies to query routing. Sending every question through the same retrieval path is fast to build and slow to trust. Some questions are factual lookups. Some require graph traversal. Some require reading a page as a visual layout. Matching the question to the right strategy is the difference between a system that is correct and one that is merely plausible.
The lesson from Zyven Studio is not that complex RAG is always better. It is that the right RAG for each document type and each question type outperforms any single strategy applied uniformly - and that choosing between them is a decision the system should make, not the user.
- Simple documents need chunked vector search - fast, cheap, and accurate enough
- Relationship-heavy documents need graph extraction before they can be searched effectively
- Layout-heavy PDFs need visual processing, not OCR text extraction
- Complex multi-part questions need hybrid retrieval followed by reranking
- Every retrieval result needs to be ranked for relevance before reaching the LLM
What's Next on This Topic
A few follow-up topics I'm planning to write up. If any of them would be useful for your work, send me a message and I'll prioritize it.
DM me on LinkedInTell me which topic you want next- How to keep a Knowledge Graph up-to-date as documents change
- Multimedia RAG - handling audio and video alongside text
- Security, governance, and evaluation loops for production RAG