Architectural Trade-offs in Migrating Agentic RAG Systems to AWS Serverless

Migrating a complex Retrieval-Augmented Generation (RAG) system from a monolithic virtual machine to a fully serverless cloud infrastructure sounds straightforward on paper: replace persistent services with managed equivalents, wire up event triggers, and watch your idle infrastructure costs drop to zero. However, real-world cloud migrations rarely adhere to ideal blueprints.

When building or scaling production LLM applications—whether directly using managed cloud providers or aggregating model providers via platforms like n1n.ai—architects repeatedly face edge cases where provider-level constraints force deviations from textbook designs.

This article explores five specific architectural trade-offs encountered while migrating an agentic RAG application off a single EC2 instance (t4g.small running PostgreSQL, Streamlit, local PyTorch embeddings, and Grafana) into an AWS serverless ecosystem comprising AWS Lambda, Amazon Bedrock, CloudFront, S3 Vectors, and DynamoDB.

Architecture Overview: Before vs. After

Decision 1: Replacing Aurora Serverless v2 with S3-Hosted SQLite & DynamoDB

The Trap

The initial plan called for replacing PostgreSQL with Amazon Aurora Serverless v2, using the RDS Data API over HTTPS to allow AWS Lambda functions to query the database outside a VPC without requiring an expensive NAT Gateway.

However, Aurora Serverless v2 has a strict minimum scaling floor of 0.5 ACU (Aurora Capacity Units), translating to roughly $43/month even while sitting entirely idle. Setting the auto-pause floor to 0 ACU avoids the base fee, but introduces an auto-pause delay of 300 seconds and a cold-resume penalty of 15 to 30 seconds. For a portfolio application or low-traffic tool visited sporadically, a 30-second initial response latency severely degrades user experience.

The Solution: A Read-Only SQLite Artifact

Analysis of the workload revealed that the database held only ~88 KB of static reference data (278 project records and associated library metadata), rebuilt exclusively during ingestion updates and read-only at query time.

Instead of running a relational server:

1. The offline AWS Step Functions ingestion pipeline compiles the database into a projects.sqlite file containing raw tables and an FTS5 full-text search index.
2. The SQLite artifact is uploaded to Amazon S3.
3. AWS Lambda downloads projects.sqlite to its ephemeral /tmp directory upon cold start.
4. State operations (user conversations, feedback logs, spend tracking) are split off into Amazon DynamoDB.

Total database infrastructure cost dropped from ~$43/month to under **$1/month**.

+-------------------+      1. Build DB      +-------------------+
|  Step Functions   | --------------------> |   S3 Bucket       |
| Ingestion Pipeline|                       | (projects.sqlite) |
+-------------------+                       +-------------------+
                                                      |
                                                      | 2. Fetch on Cold Start
                                                      v
+-------------------+   3. Local Query      +-------------------+
|  AWS Lambda API   | --------------------> | Lambda /tmp Storage|
|  (Read Path)      |                       | (SQLite + FTS5)   |
+-------------------+                       +-------------------+

Unexpected Advantage: Search Quality

PostgreSQL full-text search (ts_rank_cd) measures term frequency and position without global Inverse Document Frequency (IDF) weighting. SQLite’s native bm25() function weights rare search terms across the corpus appropriately. Moving to SQLite FTS5 resulted in better keyword search relevance matching than standard PostgreSQL full-text search.

Query Adaptation Code

PostgreSQL-specific syntax required adaptation for SQLite compatibility:

-- Original PostgreSQL Query (Uses DISTINCT ON)
SELECT DISTINCT ON (author) repo, author, github_url, score, author_total_score
FROM projects 
WHERE author_total_score IS NOT NULL
ORDER BY author, author_total_score DESC;

-- Adapted SQLite Equivalent (Uses GROUP BY with MAX aggregation)
SELECT repo, author, github_url, score, MAX(author_total_score) AS author_total_score
FROM projects 
WHERE author_total_score IS NOT NULL AND author IS NOT NULL
GROUP BY author 
ORDER BY author_total_score DESC 
LIMIT ?;

Decision 2: Forgoing Lambda Response Streaming for Buffered JSON

The Dilemma

AWS Lambda supports response streaming, allowing LLM output tokens to stream directly to web clients. However, native Lambda response streaming is restricted to Node.js managed runtimes. Running Python requires either a custom runtime layer or the AWS Lambda Web Adapter.

The Trade-Off Analysis

In an Agentic RAG system, the LLM backend executes up to six turns of tool calls, query rewrites, and reranking stages before generating the final user answer.

[User Query] -> [Query Rewrite] -> [Vector Search] -> [Rerank] -> [Tool Call Loop] -> [Final Answer Synthesis]
|<------------------------------ Non-streamed Overhead ------------------------------>| |<- Streaming Available ->|

Response streaming only accelerates the output phase of the final response generation. The client must still wait for all preliminary agent execution turns to finish.

The Security & Billing Risk

Streaming responses over serverless infrastructure exposes a financial vulnerability: if a client disconnects mid-stream (e.g., closing the browser tab), the underlying Lambda execution and Bedrock token consumption continue running until completion. On public-facing endpoints with fixed budget caps, malicious or abandoned queries can exhaust token allowances.

Conclusion: Opting for standard buffered JSON payloads simplified the runtime architecture, guaranteed complete cost control on interrupted connections, and added negligible perceived latency given the multi-step nature of the agent pipeline.

Decision 3: Resolving CloudFront OAC POST Restrictions via Proof-of-Origin Headers

The Problem

The standard security architecture for static sites hosted on CloudFront with a Lambda API backend relies on Origin Access Control (OAC) paired with AuthType: AWS_IAM on the Lambda Function URL.

However, CloudFront OAC does not calculate payload hashes for HTTP POST requests. As a result, signed POST requests sent to AWS Lambda fail with an InvalidSignatureException. Because API endpoints like /api/ask and /api/feedback require POST payloads, OAC cannot natively protect these endpoints.

Switching to Amazon API Gateway resolves signature authorization but imposes a hard 29-second integration timeout, which multi-step LLM agent loops frequently exceed.

The Workaround: Custom Proof-of-Origin Header

The resolution uses a Lambda Function URL configured with AuthType: NONE, protected by a custom HTTP header passed from CloudFront:

[Browser Client] 
       |
       v HTTP POST
[Amazon CloudFront]
       |
       | Adds Header: X-Origin-Secret: <Token>
       v
[AWS Lambda Function URL (AuthType: NONE)]
       |
       | Compares secret using secrets.compare_digest()
       v
[Execution / Bedrock Call]

Python Header Verification Implementation

To prevent timing attacks, header comparison must use constant-time string comparison:

import os
import secrets
import json

ORIGIN_SECRET = os.environ.get("ORIGIN_SECRET_VAL")

def lambda_handler(event, context):
    headers = event.get("headers