RAG is more than embeddings
RAG quality depends as much on data preparation as on the model. Chunking, metadata, retrieval strategy and filtering directly affect relevance.
A robust architecture must also provide source traceability and the ability to refuse an answer when available context is insufficient.
A good RAG system is therefore a complete document pipeline, not just a vector similarity query.
Choosing a chunking strategy
Chunks that are too small lose business context. Chunks that are too large reduce retrieval precision and increase the number of tokens sent to the model.
Chunking should reflect the actual document structure: headings, paragraphs, tables, contractual clauses or functional blocks.
It is useful to measure several strategies instead of choosing one arbitrary size permanently.
Combine vector retrieval with business filters
Semantic search must not bypass access rights or business context. Before similarity is even calculated, the system should narrow the authorized corpus.
Tenant, document type, period, status, language or confidentiality level are filters that should participate in retrieval.
In many cases, a hybrid strategy combining lexical and vector search performs better than a purely semantic approach.
Designing for observability
Tracking retrieved documents, similarity scores, latency and errors makes it easier to identify whether a problem comes from retrieval or generation.
This observability is essential for progressively improving prompts, retrieval strategies and models.
Traces should make an answer reproducible without unnecessarily exposing sensitive data.
Evaluate quality before production
A RAG system cannot be validated with a few demos. Teams should build a representative question set and measure retrieval precision as well as answer quality.
Automated tests can detect regressions after changing the model, prompt or chunking strategy.
The goal is not merely a plausible answer, but a useful, traceable answer grounded in the right sources.
