Building a Resilient RAG System with .NET 10 and AWS Bedrock

Large Language Models (LLMs) like Claude 3.5 Sonnet and GPT-4o are remarkable reasoning engines, but they face two critical operational bottlenecks in an enterprise environment: a fixed knowledge cutoff date and zero visibility into private internal systems. While fine-tuning was once the go-to solution, it is often too expensive and slow for dynamic documentation.

The industry-standard solution is Retrieval-Augmented Generation (RAG). This pattern treats the LLM as an on-demand reasoning engine rather than a static database. In this guide, we will build a production-grade RAG service using the latest .NET 10 features, the Microsoft.Extensions.AI abstraction layer, and AWS Bedrock. For developers looking for high-performance access to these models without the infrastructure overhead, aggregators like n1n.ai provide a streamlined alternative for multi-model integration.

The Architecture Overview

Our system follows a two-stage pipeline:

1. Ingestion (Offline): Documents are chunked, vectorized using Amazon Titan Text Embeddings, and stored in an OpenSearch k-NN index.
2. Query (Online): User queries are vectorized, a similarity search is performed against OpenSearch, and the retrieved context is fed into Claude 3.5 Sonnet for a grounded response.

Why .NET 10 and Microsoft.Extensions.AI?

With .NET 10, Microsoft has solidified the Microsoft.Extensions.AI library. This provides a unified set of interfaces (IChatClient, IEmbeddingGenerator) that allow you to swap LLM providers (like AWS Bedrock, Azure OpenAI, or n1n.ai) with minimal code changes. This decoupling is essential for future-proofing enterprise AI applications.

Prerequisites and AWS Setup

Before writing code, ensure you have enabled model access in the AWS Console (Region: us-east-1):

• Amazon Titan Text Embeddings G1 (amazon.titan-embed-text-v1)
• Anthropic Claude 3.5 Sonnet

You will also need a vector store. For local development, use Docker to run OpenSearch:

docker run -d -p 9200:9200 -p 9600:9600 \
  -e "discovery.type=single-node" \
  -e "plugins.security.disabled=true" \
  --name opensearch-rag \
  opensearchproject/opensearch:2.11.0

Step 1: Project Initialization

Create a new Web API project and install the necessary dependencies:

dotnet new webapi -n DotnetRagAws
cd DotnetRagAws
dotnet add package Microsoft.Extensions.AI
dotnet add package AWSSDK.BedrockRuntime
dotnet add package OpenSearch.Client

Step 2: Implementing the Embedding Generator

We need to adapt AWS Bedrock's Titan model to the IEmbeddingGenerator interface. This allows our ingestion logic to remain provider-agnostic.

public class BedrockEmbeddingGenerator : IEmbeddingGenerator<string, Embedding<float>>
{
    private readonly IAmazonBedrockRuntime _bedrockClient;
    private readonly string _modelId = "amazon.titan-embed-text-v1";

    public BedrockEmbeddingGenerator(IAmazonBedrockRuntime bedrockClient) 
        => _bedrockClient = bedrockClient;

    public async Task<GeneratedEmbeddings<Embedding<float>>> GenerateAsync(
        IEnumerable<string> values, 
        EmbeddingGenerationOptions? options = null, 
        CancellationToken ct = default)
    {
        var embeddings = new List<Embedding<float>>();
        foreach (var text in values)
        {
            var payload = JsonSerializer.Serialize(new { inputText = text });
            var request = new InvokeModelRequest
            {
                ModelId = _modelId,
                ContentType = "application/json",
                Body = new MemoryStream(Encoding.UTF8.GetBytes(payload))
            };
            var response = await _bedrockClient.InvokeModelAsync(request, ct);
            using var reader = new StreamReader(response.Body);
            var result = JsonSerializer.Deserialize<TitanResponse>(await reader.ReadToEndAsync());
            embeddings.Add(new Embedding<float>(result!.Embedding));
        }
        return new GeneratedEmbeddings<Embedding<float>>(embeddings);
    }

    private record TitanResponse([property: JsonPropertyName("embedding")] float[] Embedding);
    public void Dispose() { }
    public object? GetService(Type t, object? k) => null;
    public EmbeddingGeneratorMetadata Metadata => new("BedrockTitan");
}

Step 3: Setting Up the OpenSearch Vector Store

For RAG to work, we need a k-NN (k-Nearest Neighbors) index. Titan embeddings produce 1,536 dimensions. We use Cosine Similarity to measure the distance between the query and the documents.

public async Task CreateIndexAsync(IOpenSearchClient client)
{
    await client.Indices.CreateAsync("rag-index", c => c
        .Settings(s => s.Setting("index.knn", true))
        .Map<DocumentChunk>(m => m
            .Properties(p => p
                .KnnVector(v => v.Name(n => n.Embedding).Dimension(1536).Method(mi => mi
                    .Name("hnsw").Engine("nmslib").SpaceType("cosinesimil")))
            )
        )
    );
}

Step 4: The Chat Client (Claude 3.5 Sonnet)

Integrating Claude 3.5 Sonnet via IChatClient allows us to use standard ChatMessage objects. This is where we inject the retrieved context.

public class BedrockChatClient : IChatClient
{
    private readonly IAmazonBedrockRuntime _client;
    public BedrockChatClient(IAmazonBedrockRuntime client) => _client = client;

    public async Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages, 
        ChatOptions? options = null, 
        CancellationToken ct = default)
    {
        // Implementation details for Bedrock InvokeModelRequest...
        // Map standard ChatMessage to Anthropic's JSON format
        return new ChatResponse(new ChatMessage(ChatRole.Assistant, "Grounded response text"));
    }
    // ... other interface members
}

Step 5: Orchestrating the RAG Flow

The Orchestrator is the heart of the system. It follows these steps:

1. Vectorize the user's question.
2. Search OpenSearch for the top 3 (k=3) most relevant chunks.
3. Construct a system prompt that forces the model to use only the provided context.
4. Invoke the LLM.

public async Task<string> AskAsync(string question)
{
    var queryVector = await _generator.GenerateAsync(new[] { question });
    var docs = await _vectorStore.SearchAsync(queryVector.First().Vector.ToArray());
    
    var context = string.Join("\n", docs.Select(d => d.Content));
    var prompt = $"Answer based ONLY on this context: {context}\nQuestion: {question}";
    
    var response = await _chatClient.GetResponseAsync(new[] { new ChatMessage(ChatRole.User, prompt) });
    return response.Text;
}

Pro Tips for Production

1. Hybrid Search: Don't rely solely on vector search. Use OpenSearch's ability to combine BM25 (keyword) and k-NN (semantic) search for better accuracy on technical terms.
2. Rate Limiting: AWS Bedrock has TPS limits. If you scale, consider using n1n.ai to balance loads across different model providers or regions.
3. Token-Aware Chunking: Instead of 500 characters, use a tokenizer to split text into 256 or 512 tokens to ensure you don't break semantic meaning.

Conclusion

By combining .NET 10's modern abstractions with the power of AWS Bedrock, you can build highly resilient AI systems that respect your private data boundaries. Whether you choose to manage your own AWS infrastructure or use a unified API like n1n.ai, the RAG pattern remains the most effective way to deploy LLMs in the enterprise.

Get a free API key at n1n.ai