Recovering PDF Outlines via Typography Loop Engineering for RAG

Retrieval-Augmented Generation (RAG) is only as effective as the data it consumes. For enterprise developers, the most common data source is the PDF—a format notorious for being a 'visual layout' rather than a 'structured data' medium. When you extract text from a PDF, you often lose the hierarchical structure (H1, H2, H3) that provides essential context for semantic search. This article explores a sophisticated method called 'Loop Engineering' to recover document outlines by analyzing span-level typography and validating candidates through high-performance LLMs available via n1n.ai.

The Problem: The Flat Text Trap

Most standard PDF parsers treat documents as a stream of characters. While this works for simple keyword searches, it fails for RAG. If a vector database stores a paragraph without knowing it belongs under the section 'Quarterly Risk Assessment,' the embedding loses its relational significance. To solve this, we must reconstruct the Table of Contents (ToC) directly from the body typography, even when the metadata is missing.

The Six Deterministic Signals of Typography

Before involving an LLM, we must extract deterministic features from the document's PDF spans. A 'span' is a sequence of characters with uniform formatting. We focus on six primary signals:

1. Font Size: Headings are statistically larger than body text. We calculate the mode font size of the document and flag anything where size > mode_size.
2. Font Weight: Boolean flags for 'Bold' or 'Black' weights are high-precision indicators for headings.
3. Vertical Spacing (Leading): Headings usually have more whitespace above them than standard paragraphs.
4. Case Sensitivity: ALL CAPS spans are frequent candidates for top-level headers (H1).
5. Font Face: A switch from a Serif body font to a Sans-Serif heading font is a strong structural signal.
6. Indentation and Alignment: Centered text or specific left-margins often denote specific levels of a hierarchy.

By leveraging n1n.ai to process these features, developers can build a robust preprocessing layer that filters out noise before the heavy lifting begins.

Implementing the Loop Engineering Workflow

Loop Engineering refers to a bounded iterative process where a set of rules proposes structure, and an LLM validates it. This ensures we don't hallucinate headers while maintaining high recall.

Step 1: Span Extraction with Python

Using libraries like PyMuPDF (fitz), we extract every span with its metadata. We then create a pandas DataFrame (toc_df) to track potential candidates.

import fitz

def extract_spans(pdf_path):
    doc = fitz.open(pdf_path)
    spans = []
    for page in doc:
        blocks = page.get_text("dict")["blocks"]
        for b in blocks:
            if "lines" in b:
                for l in b["lines"]:
                    for s in l["spans"]:
                        spans.append({
                            "text": s["text"],
                            "size": round(s["size"], 2),
                            "flags": s["flags"],
                            "font": s["font"]
                        })
    return spans

Step 2: Candidate Generation

We apply a heuristic filter. For instance, if a span is bold and its size is at least 20% larger than the body text, it is marked as a Candidate.

Step 3: The LLM Validation Loop

This is where the 'Loop' happens. We send clusters of candidates to an LLM (such as Claude 3.5 Sonnet or DeepSeek-V3 via n1n.ai) to verify if they form a logical hierarchy. The LLM identifies if a candidate is a false positive (e.g., a bolded figure caption) or a true structural heading.

Pro Tip: Use a 'Bounded Loop.' Do not send the whole document. Send the candidate, the text immediately following it, and the current inferred hierarchy level. This reduces token costs and latency.

Integrating with the RAG Pipeline

Once the toc_df is validated, we drop it back into the RAG pipeline. Instead of chunking by character count, we chunk by Section.

Why n1n.ai is Essential for This Workflow

Building document intelligence requires switching between different models based on complexity. A simple H1/H2 validation might only need a fast model like GPT-4o-mini, while complex technical manuals might require the reasoning capabilities of OpenAI o3 or DeepSeek-R1.

n1n.ai provides a unified API to access these models, ensuring that your Loop Engineering pipeline is both cost-effective and highly accurate. By using a single integration, you can benchmark which model identifies your specific document structures best without rewriting your entire backend.

Conclusion

Recovering a PDF's outline is no longer a guessing game. By combining deterministic typographic signals with an LLM validation loop, you can transform messy PDFs into structured knowledge graphs. This not only improves RAG performance but also enables complex downstream tasks like automated summarization and legal compliance checking.

Get a free API key at n1n.ai