Skip to Content
Cloud Engineering Academy
16. Generative AI on AWS for Banking

Knowledge Bases and RAG

Build a retrieval-augmented generation design on Amazon Bedrock Knowledge Bases: ingest approved documents, choose a vector store and chunking, query with citations, and enforce who may see which source.

intermediate⏱ 2 h 🔴 Paid — optionalAIP-C01AIF-C01

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.BedrockRuntime 4.1.0.2, AWSSDK.BedrockAgentRuntime 4.1.0.2, AWSSDK.BedrockAgent 4.1.0.2) · pgvector/pgvector:pg17 · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the RAG pipeline (parse, chunk, embed, store, retrieve, generate) and where each step can fail.
  • Choose between a Bedrock Managed Knowledge Base and a customer-managed one, and pick a vector store (S3 Vectors, OpenSearch, Aurora PostgreSQL with pgvector) for a given need.
  • Create a knowledge base, a data source and an ingestion job with the AWS CLI, and query it from C# with citations.
  • Design source-level access control so an adviser only retrieves documents they are entitled to see.
  • Run a free local alternative with pgvector in Docker and an in-memory vector store in C#.
  • Estimate the cost of a knowledge base and decide when the free alternative is enough.

Prerequisites

Why this matters in banking

A bank's knowledge is scattered across wikis, PDFs and shared drives, and much of it is restricted: fraud-investigation procedures, staff disciplinary rules, unreleased product terms. A RAG assistant that indexes "everything" will happily retrieve an investigation procedure for a call-centre agent who should never see it. The model did nothing wrong; the retrieval layer ignored the access rules. Getting RAG right in a bank means treating the index as a new copy of every document with its own access-control problem.

The idea in plain language

Retrieval-augmented generation (RAG) means: before asking the model, search your own approved documents and paste the best passages into the prompt, so the answer comes from your text and can cite it.

The pipeline has two phases.

Ingestion (offline, repeated when documents change):

  1. Parse each file into text (PDF, Word, HTML).
  2. Chunk the text into pieces small enough to be specific and large enough to make sense.
  3. Embed each chunk with an embedding model, producing a vector.
  4. Store the vectors with the text and metadata in a vector store.

Query (online, for every question):

  1. Embed the question with the same model.
  2. Retrieve the most similar chunks (optionally filtered by metadata).
  3. Generate: send the chunks and the question to a foundation model that writes an answer with citations.

Analogy: a good reference librarian. Ingestion is cataloguing the library once and re-cataloguing when a book is replaced. Query time is the librarian pulling three relevant pages, handing them to an expert to summarise, and writing down which book each fact came from. The access-control question is whether the librarian checks your staff pass before fetching from the restricted shelf.

What problem the AWS service solves

Building that pipeline yourself means writing connectors, a parser for each file type, a chunker, an embedding batch job, a vector database and a sync process that notices updated documents. Amazon Bedrock Knowledge Bases runs the pipeline for you. You point it at a data source (for example an S3 bucket), start an ingestion job, and then call Retrieve (just the passages) or RetrieveAndGenerate (passages plus a generated answer with citations).

AWS now describes two kinds of knowledge base, and recommends the managed one for most new work.

Bedrock Managed Knowledge BaseCustomer-managed Knowledge Base
Who runs storage and indexingBedrockYou choose, provision and scale the vector store
EmbeddingsBuilt-in managed model at no extra cost, or your own (Titan Text Embeddings V2, Cohere Embed and others; 1,024 dimensions, float32)Any supported Bedrock embedding model
RerankingBuilt-in managed reranker at no extra cost (not available if you choose a custom embedding model)Optional reranker model
ConnectorsS3, SharePoint, Confluence, Google Drive, OneDrive, Salesforce, ServiceNow, Zendesk, Box, Web Crawler and customS3 and custom
Document-level access controlACL awareness at retrieval for connectors that support it (not the Web Crawler)You build it with metadata filters
Agentic retrievalSupported (multi-step, decomposes complex questions)Not supported
ChunkingDefault or fixed-size, or noneDefault or fixed-size, with more options such as hierarchical ⚠ VERIFY
Created inConsole → Amazon Bedrock AgentCore → Built-in tools → Knowledge BaseConsole → Amazon Bedrock → Knowledge Bases
Best forEnd-to-end managed RAGCustom vector store needs, graph or structured data

Honest limits:

  • Managed Knowledge Base regions are limited. On 2026-11-18 the list included us-east-1, us-west-2, eu-west-1, eu-west-2 (London), eu-central-1, three Asia Pacific Regions and GovCloud (US-West). Check before you plan a Region.
  • The embedding type cannot be changed after creation. Switching between managed and custom embeddings means creating a new knowledge base.
  • Your SDK may lag the API. In AWSSDK.BedrockAgentRuntime 4.1.0.2 there was no ManagedSearchConfiguration type for managed knowledge base retrieval when this lesson was written, so use the CLI or a newer SDK version for managed retrieval ⚠ VERIFY.
  • A knowledge base built on an Amazon Kendra GenAI index is not an option for new work. Kendra closed to new customers on 2026-07-30. Use a Bedrock knowledge base instead.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure AI Search with an indexer and skillsetBedrock Knowledge Base ingestion jobOne service ingests, chunks, embeds and indexes
Azure OpenAI "On Your Data"RetrieveAndGenerateOne call that retrieves and answers with citations
Azure AI Search vector indexS3 Vectors, OpenSearch, Aurora pgvector, or the managed storeYou choose the store for a customer-managed KB
Security trimming with search.in(group_ids, ...)Managed KB ACL awareness, or metadata filtersFilters are per query; the server must build them from verified identity
Azure AI Search semantic rankerManaged reranker or a Bedrock rerank modelRerankers reorder candidates after retrieval
SharePoint connectorManaged KB SharePoint connectorACLs can follow the source when enabled

Architecture

Security-boundary view. There are three places where access is decided, and a bank needs all three:

  1. Who may call the knowledge base. IAM on bedrock:Retrieve for the application role, scoped to one knowledge base ARN.
  2. Which chunks that caller may see. Metadata filters or ACL awareness, driven by the user's verified identity, not by anything the browser sends.
  3. What leaves the boundary. Only the chunks the user may see are placed into a prompt, after redaction.

Choosing a vector store (customer-managed)

StoreGood forNotes from the AWS documentation
Amazon S3 VectorsCost-effective storage for infrequent queries; simple to runFloat embeddings only; cosine or Euclidean; up to 1 KB of custom metadata and 35 metadata keys per vector, and ingestion fails if you exceed them; non-filterable keys AMAZON_BEDROCK_TEXT and AMAZON_BEDROCK_METADATA
OpenSearch ServerlessRich search, higher query rates, binary vectorsfaiss engine; Euclidean recommended for float vectors; has an always-on minimum capacity cost ⚠ VERIFY current minimum
OpenSearch managed clusterExisting OpenSearch estateThe knowledge base needs public network access to the domain; domains behind a VPC are not supported
Aurora PostgreSQL with pgvectorYou already run Aurora and want SQL joins and filtersNeeds a table with id, vector, text, metadata and optional custom-metadata columns, an HNSW index with cosine ops, a Secrets Manager secret; the cluster must be in the same account
Neptune Analytics (GraphRAG)Questions that follow relationships between factsCreate the graph with the vector dimension up front
Pinecone, Redis Enterprise Cloud, MongoDB AtlasAn existing third-party storeThird-party credentials in Secrets Manager; third-party terms apply

Match the embedding dimension to the model: Titan Text Embeddings V2 supports 1,024, 512 and 256; the dimension must match the store.

Chunking in one paragraph

Chunk size is a trade-off. Too small and a chunk loses context (a rule without its exception). Too large and the chunk is vague and wastes prompt space. Default or fixed-size chunking (a few hundred tokens with overlap) is a good start for policy documents. Split by document structure when you can (one policy clause per chunk), and keep the document ID and section title in metadata so a citation points somewhere useful. Measure retrieval quality on real questions before you tune anything else.

Step-by-step implementation

Cost note: a managed knowledge base is paid while it exists. Use the free local alternative below if you only want to learn the mechanics. If you do build in AWS, do it in one session and delete everything the same day.

Create the source bucket

An S3 bucket with encryption on, Block Public Access on and versioning, holding fictional policy documents only (use the three files in labs/m16-policy-assistant/data/policies/).

Create a managed knowledge base

Console: Amazon Bedrock AgentCore → Built-in tools → Knowledge Base → Create Managed Knowledge Base. Or use the CLI shown below.

Add the S3 data source and start ingestion

Connect the bucket, then start an ingestion job. Wait for status COMPLETE.

Retrieve passages

Call Retrieve and read the text, score and source location of each result. Check that the dispute question returns the dispute policy.

Generate with citations, through your own pipeline

Feed the retrieved passages into the Converse call from the previous lesson with the fenced prompt and guardrail from the lab. This keeps redaction, the guardrail and audit in your code.

Tear everything down

Delete the data source, the knowledge base and the bucket contents, then confirm nothing tagged Project=cea-lab remains.

AWS Console instructions

Create (managed): Console → Amazon Bedrock AgentCore → Built-in tools → Knowledge Base → Create Managed Knowledge Base. Optionally under Additional configurations choose the embedding type (Managed is the default), the IAM role (create a new service role is recommended) and a KMS key for the managed vector store. Pick Amazon S3 as the data source type, enter the bucket, leave parsing as the managed parser and chunking as Default chunking, then Create Knowledge Base. AWS documents creation as taking about 2 to 5 minutes (longer with a customer-managed key). Navigation labels change; search the console for "Knowledge Base" if a path differs ⚠ VERIFY.

Test: open the knowledge base and use the Test panel to enter a question and inspect the retrieved chunks and their sources.

Sync: open the data source and choose Sync after you change documents.

AWS CLI

# 1. Create the source bucket (fictional documents only). Block Public Access is on for new buckets by default;
#    this command states it explicitly.
BUCKET=hl-m16-kb-$(aws sts get-caller-identity --query Account --output text --profile learn-sandbox)
aws s3api create-bucket --bucket "$BUCKET" --region us-east-1 --profile learn-sandbox
aws s3api put-public-access-block --bucket "$BUCKET" --profile learn-sandbox \
  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3 cp labs/m16-policy-assistant/data/policies/ "s3://$BUCKET/policies/" --recursive --profile learn-sandbox

# 2. Create a managed knowledge base (service-managed embeddings). kb-config.json is below.
aws bedrock-agent create-knowledge-base --name hl-m16-policy-kb \
  --role-arn arn:aws:iam::<account-id>:role/hl-m16-kb-role \
  --knowledge-base-configuration file://kb-config.json \
  --tags Project=cea-lab,Lab=m16-l4,Environment=learn,Owner=your-name \
  --profile learn-sandbox --region us-east-1

# 3. Connect the S3 data source (S3 connector config is below).
aws bedrock-agent create-data-source --name s3-policies \
  --knowledge-base-id <kb-id> \
  --data-source-configuration file://bedrock-s3-managed-connector-configuration.json \
  --data-deletion-policy DELETE \
  --profile learn-sandbox --region us-east-1

# 4. Start ingestion, then poll until status is COMPLETE.
aws bedrock-agent start-ingestion-job --knowledge-base-id <kb-id> --data-source-id <ds-id> \
  --profile learn-sandbox --region us-east-1
aws bedrock-agent list-ingestion-jobs --knowledge-base-id <kb-id> --data-source-id <ds-id> \
  --query "ingestionJobSummaries[0].status" --profile learn-sandbox --region us-east-1

# 5. Query a managed knowledge base: retrievalConfiguration uses managedSearchConfiguration. ⚠ VERIFY field names.
aws bedrock-agent-runtime retrieve --knowledge-base-id <kb-id> \
  --retrieval-query text="How many days to raise a card dispute?" \
  --retrieval-configuration '{"managedSearchConfiguration":{"numberOfResults":5}}' \
  --profile learn-sandbox --region us-east-1

kb-config.json for a managed knowledge base with the default managed embedding model (from the AWS documentation):

{
  "type": "MANAGED",
  "managedKnowledgeBaseConfiguration": {
    "embeddingModelType": "MANAGED"
  }
}

bedrock-s3-managed-connector-configuration.json for the S3 data source:

{
  "type": "MANAGED_KNOWLEDGE_BASE_CONNECTOR",
  "managedKnowledgeBaseConnectorConfiguration": {
    "connectorParameters": {
      "type": "S3",
      "version": "1",
      "connectionConfiguration": {
        "bucketName": "<your-bucket>",
        "bucketOwnerAccountId": "<account-id>"
      },
      "deletionProtectionConfiguration": { "enableDeletionProtection": false }
    }
  }
}

The key point for a bank sits in the CLI: you query with an IAM-authorised call, and the filter is part of that call. Whoever builds the call decides what is visible.

Infrastructure as Code

Create the source bucket with encryption, Block Public Access, TLS-only and versioning, then create the knowledge base with the CLI above (or an L1 construct or provider resource once your tooling version supports the managed type ⚠ VERIFY).

PolicySourceBucket.cs
using Amazon.CDK;
using Amazon.CDK.AWS.KMS;
using Amazon.CDK.AWS.S3;
using Constructs;

namespace Harbourline.GenAi.Infra;

public sealed class PolicySourceStack : Stack
{
    public PolicySourceStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var key = new Key(this, "SourceKey", new KeyProps { EnableKeyRotation = true });

        _ = new Bucket(this, "PolicySources", new BucketProps
        {
            Encryption = BucketEncryption.KMS,
            EncryptionKey = key,
            BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
            EnforceSSL = true,
            Versioned = true,
            RemovalPolicy = RemovalPolicy.DESTROY, // lab only
            AutoDeleteObjects = true,              // lab only
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m16-l4");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
        Amazon.CDK.Tags.Of(this).Add("Owner", "your-name");
    }
}

Complete code

All three programs below compiled against AWSSDK.BedrockRuntime 4.1.0.2 and AWSSDK.BedrockAgentRuntime 4.1.0.2 when this lesson was written.

Embeddings and an in-memory store (no vector database needed)

This is "RAG from scratch": embed documents with Titan Text Embeddings V2, store them in a list, search by cosine similarity. It is the best way to understand what a knowledge base does for you.

InMemoryRag.cs
using System.Text;
using System.Text.Json;
using Amazon.BedrockRuntime;
using Amazon.BedrockRuntime.Model;

namespace Harbourline.GenAi.Rag;

public sealed class TitanEmbedder(IAmazonBedrockRuntime bedrock, int dimensions = 256)
{
    public async Task<float[]> EmbedAsync(string text, CancellationToken ct = default)
    {
        // Titan embeddings use InvokeModel with a model-specific JSON body, not Converse.
        var body = JsonSerializer.Serialize(new { inputText = text, dimensions, normalize = true });
        var response = await bedrock.InvokeModelAsync(new InvokeModelRequest
        {
            ModelId = "amazon.titan-embed-text-v2:0",
            ContentType = "application/json",
            Accept = "application/json",
            Body = new MemoryStream(Encoding.UTF8.GetBytes(body)),
        }, ct);

        using var json = await JsonDocument.ParseAsync(response.Body, cancellationToken: ct);
        return json.RootElement.GetProperty("embedding").EnumerateArray().Select(e => e.GetSingle()).ToArray();
    }
}

public sealed record Chunk(string DocumentId, string Text, float[] Vector);

public sealed class InMemoryVectorStore
{
    private readonly List<Chunk> _chunks = [];

    public void Add(Chunk chunk) => _chunks.Add(chunk);

    public IReadOnlyList<(Chunk Chunk, float Score)> Search(float[] query, int top = 3) =>
        _chunks.Select(c => (Chunk: c, Score: Cosine(query, c.Vector)))
            .OrderByDescending(x => x.Score)
            .Take(top)
            .ToList();

    // Titan vectors are normalised, so the dot product equals cosine similarity; dividing by the lengths keeps it
    // correct for any embedding model.
    public static float Cosine(float[] a, float[] b)
    {
        float dot = 0, na = 0, nb = 0;
        for (var i = 0; i < a.Length; i++)
        {
            dot += a[i] * b[i];
            na += a[i] * a[i];
            nb += b[i] * b[i];
        }

        return na == 0 || nb == 0 ? 0 : dot / (MathF.Sqrt(na) * MathF.Sqrt(nb));
    }
}

Use it: embed each policy paragraph once and Add it, embed the question, call Search, then place the top chunks into the fenced prompt from the lab. Titan Text Embeddings V2 calls cost fractions of a cent for a few paragraphs ⚠ VERIFY the price.

Query a knowledge base from C#

KnowledgeBaseSearch.cs
using Amazon.BedrockAgentRuntime;
using Amazon.BedrockAgentRuntime.Model;

namespace Harbourline.GenAi.Rag;

public static class KnowledgeBaseSearch
{
    public static async Task<IReadOnlyList<(string Uri, double Score, string Text)>> SearchAsync(
        IAmazonBedrockAgentRuntime runtime, string knowledgeBaseId, string question, CancellationToken ct = default)
    {
        var response = await runtime.RetrieveAsync(new RetrieveRequest
        {
            KnowledgeBaseId = knowledgeBaseId,
            RetrievalQuery = new KnowledgeBaseQuery { Text = question },
            RetrievalConfiguration = new KnowledgeBaseRetrievalConfiguration
            {
                // Customer-managed knowledge bases use vector search settings.
                // Managed knowledge bases use managedSearchConfiguration (newer SDK or CLI, ⚠ VERIFY).
                VectorSearchConfiguration = new KnowledgeBaseVectorSearchConfiguration { NumberOfResults = 5 },
            },
        }, ct);

        return response.RetrievalResults
            .Select(r => (r.Location?.S3Location?.Uri ?? "unknown", (double)(r.Score ?? 0), r.Content.Text))
            .ToList();
    }
}

Source-level access control with a server-built filter

FilteredSearch.cs
using Amazon.BedrockAgentRuntime;
using Amazon.BedrockAgentRuntime.Model;

namespace Harbourline.GenAi.Rag;

public static class FilteredSearch
{
    public static async Task<IReadOnlyList<string>> SearchAsync(
        IAmazonBedrockAgentRuntime runtime, string kbId, string question,
        string department, IReadOnlyList<string> allowedClassifications, CancellationToken ct = default)
    {
        // The filter comes from the caller's VERIFIED entitlements (for example claims from your identity provider),
        // never from anything the browser sends. Documents must carry matching metadata when they are ingested.
        var filter = new RetrievalFilter
        {
            AndAll =
            [
                new RetrievalFilter { Equals = new FilterAttribute { Key = "department", Value = department } },
                new RetrievalFilter
                {
                    In = new FilterAttribute
                    {
                        Key = "classification",
                        Value = new Amazon.Runtime.Documents.Document(
                            allowedClassifications.Select(c => new Amazon.Runtime.Documents.Document(c)).ToList()),
                    },
                },
            ],
        };

        var response = await runtime.RetrieveAsync(new RetrieveRequest
        {
            KnowledgeBaseId = kbId,
            RetrievalQuery = new KnowledgeBaseQuery { Text = question },
            RetrievalConfiguration = new KnowledgeBaseRetrievalConfiguration
            {
                VectorSearchConfiguration = new KnowledgeBaseVectorSearchConfiguration
                {
                    NumberOfResults = 5,
                    Filter = filter,
                },
            },
        }, ct);

        return response.RetrievalResults.Select(r => r.Content.Text).ToList();
    }
}

Code explained line by line

  1. TitanEmbedder.EmbedAsync serialises inputText, dimensions and normalize into the Titan request body. normalize: true returns unit-length vectors, so cosine similarity and dot product agree.
  2. InvokeModelAsync returns a stream in response.Body; the program parses the JSON and reads the embedding array into floats.
  3. InMemoryVectorStore.Search scores every stored chunk against the query vector and returns the top few. A real store does the same with an index so it scales to millions of chunks.
  4. KnowledgeBaseSearch calls Retrieve: it returns passages, scores and S3 locations, not an answer. Doing generation yourself keeps redaction, guardrail and audit under your control.
  5. In FilteredSearch, RetrievalFilter combines conditions with AndAll. The Equals filter pins the department; the In filter restricts classification to the values the caller is allowed. The filter is part of the request, so retrieval itself enforces the rule, rather than your code trimming results afterwards.

Controlling access to sources

Pick the lightest design that meets your rules, and remember that the knowledge base does not know your users: it knows IAM principals and whatever filters you pass.

PatternHowGood forTrade-off
One knowledge base per audienceSeparate KBs (and buckets) for "all staff", "cards team", "financial crime"; IAM decides which KB a role can callFew, clear audiencesMore indexes to run and pay for
Metadata filtersTag each document with department and classification; the API builds a filter from the user's claimsMany overlapping audiencesA forgotten tag or a missing filter leaks documents
Source ACL awarenessIn a managed knowledge base with supported connectors (not the Web Crawler), enable ACL awareness so retrieval follows document permissions from the sourceContent already protected in SharePoint or ConfluenceNot available in GovCloud; verify behaviour with your own tests
Do not index itKeep the most sensitive material out of RAG entirelyInvestigation procedures, personal dataLower coverage, simplest risk

Defence in depth: apply a filter and a guardrail on the response and an audit log of which document IDs were retrieved for which user. Test the filter with a user who is not entitled, and make that test part of your pipeline.

Free alternative: pgvector in Docker (no AWS account)

This design-and-practice exercise uses a PostgreSQL image with the pgvector extension. It is the same idea as Aurora PostgreSQL with pgvector, which is a supported knowledge-base store. Vectors here have four numbers so you can read them; a real embedding has hundreds.

# Start PostgreSQL with pgvector (the image tag pgvector/pgvector:pg17 existed on Docker Hub when this was checked)
export PGPASSWORD=choose-a-throwaway-value   # local lab only; never reuse a real password
docker run -d --name hl-pgvector -e POSTGRES_PASSWORD="$PGPASSWORD" -p 127.0.0.1:5432:5432 pgvector/pgvector:pg17

# Create the schema, load three toy chunks and run a filtered similarity query
docker exec -i hl-pgvector psql -U postgres <<'SQL'
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
  id bigserial PRIMARY KEY,
  document_id text NOT NULL,
  classification text NOT NULL,
  chunk text NOT NULL,
  embedding vector(4) NOT NULL
);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
INSERT INTO chunks (document_id, classification, chunk, embedding) VALUES
 ('POL-001','internal','Disputes within 120 days.','[0.9,0.1,0.0,0.1]'),
 ('POL-002','internal','Close account at zero balance.','[0.1,0.9,0.1,0.0]'),
 ('POL-009','restricted','Staff investigation procedure.','[0.8,0.2,0.1,0.1]');
SELECT document_id, chunk,
       round((1 - (embedding <=> '[0.85,0.15,0.0,0.1]'))::numeric, 3) AS score
FROM chunks
WHERE classification IN ('public','internal')
ORDER BY embedding <=> '[0.85,0.15,0.0,0.1]'
LIMIT 3;
SQL

# Clean up
docker rm -f hl-pgvector

Expected result (run on 2026-11-18): POL-001 with a score near 0.998 and POL-002 near 0.278. The restricted POL-009 is excluded by the WHERE clause even though its vector is close to the query. That single line is the source-access-control idea in miniature: similarity decides relevance, the filter decides permission.

<=> is pgvector's cosine distance operator, so 1 - distance is the similarity. hnsw (embedding vector_cosine_ops) builds an approximate nearest-neighbour index; the Aurora knowledge-base documentation uses the same index type.

Security considerations

⚠ Learning shortcut — not for production

The lab uses a few fictional documents, one bucket and a broad service role, and the free exercise has no filtering by real identity. Production uses least-privilege roles for the knowledge base service role, KMS customer-managed keys on the bucket and index, VPC endpoints, per-audience knowledge bases or enforced filters, ingestion pipelines that only accept approved documents, and an audit trail of document IDs retrieved.

  • The index is a copy of your documents. Classify it, encrypt it with a key you control, and delete from it when you delete the source (set the data deletion policy deliberately).
  • Only approved content gets ingested. Poisoned documents are a prompt-injection route: an attacker who can edit a wiki page can plant instructions your assistant will read. Control who can write to the sources.
  • Fence retrieved text. Retrieved passages are data, never instructions. The lab's PromptBuilder escapes angle brackets so a document cannot close its own fence.
  • Return citations. Citations let a person verify an answer and let an auditor trace it.
  • Do not put customer personal data in the knowledge base unless a data-protection assessment says you may. For UK GDPR, an erasure request must be able to reach the index as well as the source ⚠ ask legal and the DPO.
  • Quality is a risk control. Build a small set of real questions with expected source documents and run it after every change to chunking, embeddings or documents.

Banking use case

Harbourline Bank's contact-centre advisers ask policy questions during calls. The team creates a managed knowledge base over a single S3 bucket of approved procedures, with a department and classification tag per document. The adviser tool authenticates the adviser, looks up their team, builds the filter on the server and calls Retrieve. The retrieved passages are redacted, fenced and sent through the guardrailed Converse call from the lab. The screen shows the answer with source links and a "check the source" banner. Nothing is written to a customer record. Every question logs user ID, document IDs retrieved and outcome, never the question text.

Common mistakes

  • Indexing everything and filtering "later". The index must carry the access rules from day one.
  • Letting the browser send the filter. A tampered request widens access. Build it on the server.
  • Skipping evaluation. Without a question set you cannot tell whether changing chunk size helped.
  • One giant chunk size for all documents. A table, a legal clause and a FAQ need different treatment.
  • Forgetting to re-sync. The index reflects the last ingestion. Policy changes are invisible until you sync, so automate it and record the policy version in metadata.
  • Choosing a custom embedding model, then expecting the managed reranker. The managed reranker is only available with the default managed embeddings.
  • Treating the managed store as free. Index storage and retrieval calls are billed.

Troubleshooting

SymptomLikely causeFix
Ingestion job failsSource permissions, unsupported file, S3 Vectors metadata over 1 KB or 35 keysRead the job's failure reasons; shrink metadata or fix permissions
Retrieval returns nothingIngestion not complete, wrong knowledge base ID, filter too strictCheck job status, then retry without a filter to isolate
Answers ignore a new policyDocuments not re-syncedStart a new ingestion job
Retrieves a document a user must not seeMissing or wrong metadata, filter not appliedFix tags; add a test that uses an unentitled user
AccessDeniedException on RetrieveRole lacks bedrock:Retrieve on the knowledge base ARNAdd the action scoped to the KB
ValidationException on the filterValue type or key mismatchCheck metadata types; for OpenSearch use keyword fields for custom metadata
Managed retrieval option missing in the SDKSDK predates the managed typeUse the CLI or upgrade the SDK ⚠ VERIFY
Poor answers though retrieval looks rightPrompt does not require grounding or citationsStrengthen the system prompt; add a grounding guardrail

Cost and cleanup

Cost label: paid for the AWS build; free for the pgvector and in-memory exercises. The prices below come from the Bedrock pricing page on 2026-11-18 and may change ⚠ VERIFY.

ItemPrice basisExample
Managed KB index storage5.00 USD per GB of raw data per month100 MB of policies is about 0.50 USD per month, pro rata for a day
Managed KB embeddings and reranking0 USD (service-managed models)Included
Managed KB Retrieve call1.00 USD per 1,000 calls2,000 questions a day is 2.00 USD a day
Agentic retrieval4.00 USD per 1,000 agentic calls plus the underlying Retrieve callsUse only where multi-step retrieval pays for itself
Titan Embeddings V2 (custom or from-scratch)Per input token, tinyPennies for a lab ⚠ VERIFY
OpenSearch ServerlessAlways-on minimum capacityOften the largest cost of a lab: avoid it for learning ⚠ VERIFY

A one-hour lab with the managed knowledge base, three small documents and a few dozen queries should cost well under one dollar. The design is only paid if you leave it running.

Teardown (run all of it, in this order):

aws bedrock-agent delete-data-source --knowledge-base-id <kb-id> --data-source-id <ds-id> --profile learn-sandbox --region us-east-1
aws bedrock-agent delete-knowledge-base --knowledge-base-id <kb-id> --profile learn-sandbox --region us-east-1
aws s3 rm "s3://$BUCKET" --recursive --profile learn-sandbox
aws s3api delete-bucket --bucket "$BUCKET" --profile learn-sandbox --region us-east-1

# Versioned buckets keep old versions: if delete-bucket fails, empty versions and delete markers first.
# Confirm nothing tagged Project=cea-lab remains.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --profile learn-sandbox --region us-east-1

Free alternative for everything above: the in-memory store and the pgvector container. Remove the container with docker rm -f hl-pgvector.

Exercises

  • 🟢 Guided: run the pgvector container and the SQL. Change the query vector to be closer to POL-002 and confirm the order flips. Then change the WHERE clause to include restricted and watch POL-009 appear.
  • 🟡 Challenge: write a C# test for InMemoryVectorStore using hand-made four-number vectors that proves the nearest chunk ranks first and an unrelated chunk ranks last. Add a classification field to Chunk and make Search accept an allowed list.
  • 🔴 Live problem: the head of the cards team asks, "Can we just index the whole shared drive? People will find things faster." List the five questions you would ask first, and propose a phased rollout that starts with one audience and a classification rule.

Assignment

Write a RAG design note (one page) for an internal "policy Q and A" assistant: sources, owners, classification, chunking choice and why, vector store choice and why, who can retrieve what (the access-control pattern), how documents are re-synced, the evaluation question set (ten questions with expected source), and the cost estimate. Include the retrieval and generation diagram. This note becomes part of the module assignment.

Solutions

Knowledge check

Interview questions

  • Why use RAG instead of fine-tuning for policy answers? Policies change; RAG updates by re-indexing and can cite the source. Fine-tuning is slower to change, harder to audit and bakes data into the model.
  • How do you stop a RAG assistant revealing documents a user should not see? Enforce access at retrieval with per-audience indexes, metadata filters built server-side from verified identity, or source ACL awareness, and test with an unentitled user.
  • How do you know your RAG system is good? Maintain a question set with expected source documents, measure retrieval hit rate and answer grounding, and rerun after every change.
  • What are the risks of ingesting a wiki? Poisoned or stale pages, personal data, and prompt-injection text. Control write access, review, and fence retrieved text.

Certification practice

Completion checklist

  • I can draw the ingestion and query phases of RAG and name where each can fail
  • I can compare managed and customer-managed knowledge bases and pick a store
  • I ran the pgvector exercise (or built the knowledge base) and saw a restricted document excluded by the filter
  • I can explain at least two ways to enforce source-level access control
  • I ran the cleanup steps and confirmed nothing tagged Project=cea-lab is still billing

Key takeaways

  • RAG puts your approved text in the prompt at question time; the index is a new, sensitive copy of your documents.
  • AWS recommends the Bedrock Managed Knowledge Base for most new work; choose customer-managed when you need a particular store.
  • Relevance is similarity; permission is a filter. Enforce permission at retrieval, built on the server from verified identity.
  • Measure with a question set, re-sync on change, and cite sources.
  • The knowledge base is paid while it exists: use the local pgvector and in-memory options to learn, and tear down the AWS version the same day.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page