Optimizing RAG at Scale: Chunking, Retrieval, and Bayesian Search

Building a Retrieval-Augmented Generation (RAG) system is easy; making it production-ready is exceptionally difficult. Most developers start with a 'naive RAG' setup: split documents into 512-token chunks, embed them with a standard model, and perform a simple top-k vector search. While this works for a weekend demo, it fails the moment it encounters complex legal contracts, technical API documentation, or nuanced customer support tickets.

When scaling these systems at n1n.ai, we found that the difference between a 70% recall and a 95% recall lies in the architecture of the retrieval pipeline itself. This guide explores how to move from 'semantic search + hope' to a measured, tunable retrieval engine using advanced chunking, hybrid retrieval, and Bayesian optimization.

The Chunking Crisis: Why Fixed Windows Fail

In production, a fixed 512-token window is often the enemy of relevance. If a legal clause is split in the middle, the embedding loses the context of the obligation. If an API reference is too large, the 'signal' of a specific function gets drowned in the 'noise' of the surrounding boilerplate.

To solve this, we implement a tiered chunking strategy. By using n1n.ai to access high-reasoning models like Claude 3.5 Sonnet or OpenAI o3, you can even implement 'Agentic Chunking' where the LLM determines the semantic boundaries of a document.

from abc import ABC, abstractmethod
from dataclasses import dataclass

@dataclass
class Chunk:
    text: str
    metadata: dict
    token_count: int
    chunk_id: str

class ChunkingStrategy(ABC):
    @abstractmethod
    def chunk(self, document: str, metadata: dict) -> list[Chunk]: ...

class RecursiveChunker(ChunkingStrategy):
    """Respects structure: markdown headers, code blocks, paragraphs."""
    def __init__(self, separators=["\n## ", "\n### ", "\n\n", "\n", " "], chunk_size=512):
        self.separators = separators
        self.chunk_size = chunk_size

class SemanticChunker(ChunkingStrategy):
    """Uses embedding similarity to find natural boundaries."""
    def __init__(self, model="text-embedding-3-small", threshold=0.7):
        self.model = model
        self.threshold = threshold

Pro Tip: For technical documentation, use a RecursiveChunker that is aware of code blocks. For conversational data, ensure your overlap is at least 15-20% to maintain the flow of dialogue across chunks.

Hybrid Retrieval: The Best of Both Worlds

Pure vector search (dense retrieval) is great for synonyms but terrible for exact matches like SKU numbers, error codes, or specific function names. Conversely, BM25 (sparse retrieval) excels at keyword matching but misses the conceptual 'vibe' of a query.

We utilize a Hybrid Retriever that combines both, followed by a Cross-Encoder reranker. This 'funnel' approach ensures that we don't just find documents that are mathematically similar, but documents that are actually relevant.

class HybridRetriever:
    def __init__(self, vector_store, bm25_index, reranker, weights=(0.4, 0.3, 0.3)):
        self.vector = vector_store
        self.bm25 = bm25_index
        self.reranker = reranker
        self.weights = weights

    async def retrieve(self, query: str, k=20, final_k=5):
        # Stage 1: Parallel retrieval
        vector_results = await self.vector.search(query, k=k)
        bm25_results = await self.bm25.search(query, k=k)

        # Stage 2: Reciprocal Rank Fusion (RRF)
        fused = self._rrf(vector_results, bm25_results, k=60)

        # Stage 3: Cross-encoder rerank (top 50 → top 5)
        # Using n1n.ai's high-speed endpoints for reranking models
        reranked = await self.reranker.rerank(query, fused[:50])

        return reranked[:final_k]

By moving from a single-modality search to this hybrid approach, we typically see a 10-15% jump in Recall@10. The Cross-Encoder is the 'secret sauce' here; while a Bi-Encoder (standard embedding) only has a ~0.75 correlation with true relevance, a Cross-Encoder hits ~0.92.

Query Transformation: Helping Users Help Themselves

Users are notoriously bad at writing search queries. They use ambiguous terms or ask multi-part questions that no single chunk can answer. Query expansion and decomposition are essential.

By leveraging n1n.ai, you can route these transformations to smaller, faster models like DeepSeek-V3 or GPT-4o-mini to keep latency low while significantly improving search coverage.

Bayesian Optimization: Tuning the Black Box

Why choose chunk_size=512? Why is the vector weight 0.4? Most teams guess. We treat these as hyperparameters and optimize them using Bayesian Search via Optuna. This allows us to find the 'Pareto Frontier'—the perfect balance between recall and latency.

def objective(trial: optuna.Trial) -> tuple[float, float]:
    config = RetrievalConfig(
        chunk_size=trial.suggest_categorical("chunk_size", [256, 512, 1024]),
        vector_weight=trial.suggest_float("vector_weight", 0.1, 0.9),
        # ... other params
    )
    recall, latency = evaluate_config(config, golden_set)
    return recall, latency / 1000

In our tests on legal datasets, this automated tuning cut p95 latency by 62% while increasing recall from 78% to 95%.

Conclusion

Retrieval is not a 'set and forget' component; it is core infrastructure. To build a world-class RAG system, you must:

1. Chunk based on document structure, not arbitrary limits.
2. Implement hybrid search with a reranking step.
3. Use Bayesian optimization to stop guessing your parameters.
4. Monitor everything—from embedding latency to recall sampling.

By building on top of the stable, high-speed LLM infrastructure provided by n1n.ai, you can focus on these high-level architectural improvements rather than worrying about API uptime or model rate limits.

Get a free API key at n1n.ai