Building a Production RAG Pipeline on AWS: Bedrock Knowledge Bases, OpenSearch Serverless, and Terraform
- Authors
- Name
- Nino
- Occupation
- Senior Tech Editor
Every engineering org has the same problem. New engineer joins, asks "how does the deployment pipeline handle database migrations?" The answer exists — buried in a Confluence page from 2023, a Slack thread from last April, and a post-mortem that three people remember reading. Nobody can find it. So the question gets asked again. And again.
I built a production-ready RAG pipeline on AWS to solve this. Not a notebook demo. Not a "just call the API" tutorial. A real pipeline — document ingestion from S3, vector embeddings via Amazon Titan, retrieval from OpenSearch Serverless, generation via Claude 3.5 Sonnet through Bedrock, a Lambda query API behind API Gateway, and CloudWatch observability across the whole stack. All deployed with Terraform. Zero console clicks.
The goal: any engineer can hit an API endpoint (or a Slack bot wired to it) with a natural language question and get an accurate answer with citations back to the source documents.
+------------------------------------------------------------------+
| PRODUCTION RAG PIPELINE |
+----------------------------------+-------------------------------+
| Metric | Value |
+----------------------------------+-------------------------------+
| Documents indexed | 2,400+ (Confluence, runbooks, |
| | post-mortems, ADRs) |
| Avg query latency (p50) | 1.8 seconds |
| Avg query latency (p99) | 4.2 seconds |
| Monthly AWS cost | ~$380 (at ~500 queries/day) |
| Embedding model | Amazon Titan Text Embed v2 |
| Generation model | Claude 3.5 Sonnet via Bedrock |
| Infrastructure | 100% Terraform, zero console |
+----------------------------------+-------------------------------+
Why RAG Over Fine-Tuning
Before writing any Terraform, I had to answer the first question: why RAG instead of fine-tuning a model on our internal docs?
| Factor | RAG | Fine-Tuning |
|---|---|---|
| Data freshness | Real-time (re-ingest) | Stale (retrain cycle) |
| Citation support | Native (source docs) | None |
| Cost to update | ~$0 (S3 sync) | 1000s/retrain |
| Hallucination control | Grounded in sources | No guarantees |
| Setup complexity | Medium | High |
| Latency | Higher (~2s) | Lower (~500ms) |
| Domain-specific language | Depends on retrieval | Strong |
| Data privacy | Data stays in your VPC | Sent to training API |
RAG wins when documents change frequently. Internal runbooks, ADRs, and post-mortems update weekly. Fine-tuning means retraining every time a doc changes. RAG means dropping a new file in S3.
You also need citations. "According to the post-mortem from March 2026..." is the difference between a useful answer and a hallucination you can't verify. For teams seeking ultimate stability and flexibility, combining these architectures with robust LLM API gateways like n1n.ai ensures your infrastructure remains resilient and cost-effective.
Architecture Overview
+-----------------------------------------------------------------------------+
| PRODUCTION RAG PIPELINE |
| |
| +----------+ +-----------------+ +--------------+ |
| | | | Bedrock Data | | Amazon | |
| | S3 |--->| Source |--->| Titan | |
| | (docs) | | (chunking) | | Embed v2 | |
| +----------+ +-----------------+ +------+-------+ |
| | |
| v |
| +--------------------+ |
| | OpenSearch | |
| | Serverless | |
| | (vector store) | |
| +------+-------------+ |
| | |
| +----------+ +-----------------+ | |
| | API | | Lambda | | |
| | Gateway |--->| (query API) |---> Bedrock KB ---> Claude 3.5 |
| +----------+ +-----------------+ (retrieve + Sonnet |
| generate) (answer) |
| |
| +-------------------------------------------------------------+ |
| | CloudWatch: latency, tokens, errors, cost anomaly alarms | |
| +-------------------------------------------------------------+ |
+-----------------------------------------------------------------------------+
Why OpenSearch Serverless over the alternatives?
- Pinecone: Great product, but adds a third-party dependency and egress costs. Bedrock Knowledge Bases have native AOSS integration — no custom retrieval code needed.
- pgvector (Aurora/RDS): Solid for small datasets. Falls behind at scale — no built-in ANN indexing tuning, and you're paying for a full database instance.
- OpenSearch Serverless: Native Bedrock integration, managed scaling, no cluster to babysit. The minimum cost stings ($350/month for 2 OCUs), but you're trading ops burden for dollars.
Component Cost Breakdown
- OpenSearch Serverless (2 OCUs): ~$350
- Bedrock - Titan Embed v2: ~$4 (ingestion)
- Bedrock - Claude 3.5 Sonnet: ~$18 (500 queries/day)
- Lambda: ~$2
- API Gateway: ~$2
- S3 (document storage): <$1
- CloudWatch: ~$3
- Total: ~$380/month
Yes, OpenSearch Serverless is 92% of the bill. When integrating external or backup models through providers like n1n.ai, you can dramatically optimize your multi-model routing costs.
OpenSearch Serverless Collection
OpenSearch Serverless for vector search requires three things: an encryption policy, a network policy, and a data access policy. Without all three, the collection won't create.
resource "aws_opensearchserverless_security_policy" "encryption" {
name = "rag-kb-encryption"
type = "encryption"
policy = jsonencode({
Rules = [
{
Resource = ["collection/rag-knowledge-base"]
ResourceType = "collection"
}
]
AWSOwnedKey = true
})
}
resource "aws_opensearchserverless_security_policy" "network" {
name = "rag-kb-network"
type = "network"
policy = jsonencode([
{
Rules = [
{
Resource = ["collection/rag-knowledge-base"]
ResourceType = "collection"
},
{
Resource = ["collection/rag-knowledge-base"]
ResourceType = "dashboard"
}
]
AllowFromPublic = true
}
])
}
resource "aws_opensearchserverless_collection" "kb" {
name = "rag-knowledge-base"
description = "Vector store for RAG knowledge base"
type = "VECTORSEARCH"
depends_on = [
aws_opensearchserverless_security_policy.encryption,
aws_opensearchserverless_security_policy.network,
]
}
Bedrock Knowledge Base
The knowledge base ties everything together: it connects S3 (documents) to the embedding model (Titan) to the vector store (OpenSearch Serverless).
resource "aws_bedrockagent_knowledge_base" "main" {
name = "internal-knowledge-base"
role_arn = aws_iam_role.bedrock_kb.arn
knowledge_base_configuration {
type = "VECTOR"
vector_knowledge_base_configuration {
embedding_model_arn = "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
}
}
storage_configuration {
type = "OPENSEARCH_SERVERLESS"
opensearch_serverless_configuration {
collection_arn = aws_opensearchserverless_collection.kb.arn
vector_index_name = "bedrock-knowledge-base-default-index"
field_mapping {
vector_field = "bedrock-knowledge-base-default-vector"
text_field = "AMAZON_BEDROCK_TEXT_CHUNK"
metadata_field = "AMAZON_BEDROCK_METADATA"
}
}
}
}
Query API with Lambda
The query layer is a Lambda function behind API Gateway. It takes a natural language question, calls Bedrock's RetrieveAndGenerate API, and returns a structured response with citations.
import json
import os
import logging
import boto3
logger = logging.getLogger()
logger.setLevel(logging.INFO)
bedrock_agent = boto3.client("bedrock-agent-runtime")
KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
def handler(event, context):
body = json.loads(event.get("body", "{}"))
question = body.get("question", "").strip()
result = bedrock_agent.retrieve_and_generate(
input={"text": question},
retrieveAndGenerateConfiguration={
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": KNOWLEDGE_BASE_ID,
"modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
}
}
}
return {
"statusCode": 200,
"body": json.dumps({"answer": result["output"]["text"]})
}
Observability and Cost Control
Instrumenting CloudWatch dashboards and metric alarms ensures you spot token spikes and latency regressions immediately. When scaling enterprise workloads or routing fallback calls through aggregators like n1n.ai, always maintain active cost anomaly alarms.
Lessons Learned
- OpenSearch Serverless minimum cost is the biggest surprise. $350/month for 2 OCUs before indexing a single document.
- Chunking strategy matters more than model choice. Bad chunks lead to poor retrieval regardless of the summarizer.
- Bedrock Knowledge Bases handle 80% of the plumbing. Don't rebuild standard ingestion pipelines from scratch.
Action Items Checklist
- Create S3 bucket for documents
- Deploy OpenSearch Serverless collection
- Deploy Bedrock Knowledge Base with Titan Embed v2
- Configure Lambda query handler and API Gateway
- Set up CloudWatch dashboards and alarms
Get a free API key at n1n.ai