GenAI in plain language
Explain foundation models, tokens, context windows, embeddings, RAG and agents in plain words, and say why a model's answer can be confidently wrong.
Assumes: .NET 10 LTS ยท C# only, no AWS calls in this lesson ยท model facts checked against the Amazon Bedrock model cards on 2026-11-18. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain what a foundation model, a token, a context window and a temperature setting are, without jargon.
- Describe what an embedding is and compare two pieces of text by cosine similarity in C#.
- Draw the difference between a plain prompt, retrieval-augmented generation (RAG) and an agent.
- List at least five reasons a model's answer can be wrong, and say which ones better prompting cannot fix.
- Sort a handful of banking use cases into good fit, controlled fit and not without strong controls.
Prerequisites
- Module 16 overview: why a bank must answer hard questions before it adds AI.
- Comfort reading C#. This lesson makes no AWS calls, so you need no AWS account, SDK or credentials.
- Tools: .NET 10 SDK (or Docker with the
mcr.microsoft.com/dotnet/sdk:10.0image).
Why this matters in banking
A product manager at Harbourline Bank asks you to "put a chatbot on the policy wiki". Two weeks later a support agent pastes a customer's full IBAN and email address into it, asks whether a disputed payment can be refunded, and acts on the confident answer. The answer was fluent, polite and wrong: the model had never seen the current dispute policy and filled the gap with something plausible.
None of that is an AWS problem. It is a misunderstanding of what a generative model is. If you know what the machine is actually doing, you can predict where it will fail, and you can design the controls (redaction, retrieval, guardrails, human review) before an incident, not after. That is the whole point of this module.
The idea in plain language
A large language model (LLM) is a program that has read a very large amount of text and learned to predict what text is likely to come next. You give it some text (the prompt); it writes a continuation, one small piece at a time. When the prompt looks like a question and the model has seen many question-and-answer pairs, the continuation looks like an answer.
Three consequences follow, and everything else in this module hangs off them:
- It is fluent, not factual. It optimises for text that sounds right. Whether the text is true depends on whether the truth was common in its training data and present in your prompt.
- It has no memory of your bank. It does not know your policies, your customers or yesterday's incident unless you put that text in the prompt.
- It does not run the same way twice. By default there is randomness in how it picks each next piece.
Analogy: a foundation model is a very fast, very well-read new graduate. Brilliant at drafting and summarising, and happy to bluff when it does not know. You would never let a new graduate approve a loan or move money unsupervised, and you would give them the policy manual before asking a policy question. Retrieval is the manual; human review is the supervisor.
The vocabulary you need
| Term | Plain meaning | Why you care |
|---|---|---|
| Foundation model (FM) | A large model trained on broad data that can be reused for many tasks | You rent one through a service such as Amazon Bedrock rather than train your own |
| Token | A chunk of text, roughly three-quarters of an English word. Roughly four characters on average for English | Models read and write tokens, and you are billed per token |
| Prompt | Everything you send: instructions, retrieved documents, the user's question | It is the only way to give the model facts and rules |
| System prompt | Standing instructions ("you are a policy assistant, answer only from the documents") | Your first, weak, line of defence |
| Context window | The most tokens the model can consider at once (prompt plus answer) | The two Claude 5.5 models on Amazon Bedrock list a 1 million token window on their model cards, and many models are far smaller |
| Temperature | A knob for randomness. Low means more repeatable, high means more varied | Use low for policy answers and classification |
| Embedding | A list of numbers that captures the meaning of a piece of text | It lets a computer find text with similar meaning |
| Hallucination | A confident statement that is not supported by facts | The main reason outputs need checking |
| Inference | Running the trained model to get an answer | What Bedrock charges you for |
Tokens, in numbers you can feel
You do not need an exact tokenizer to plan. A useful rule of thumb for English is about four characters per token, so a 12-word question is around 13 to 16 tokens and a one-page policy is around 500 to 700 tokens. Numbers, code, IBANs and non-English text use more tokens per character. The rule is only for estimating; the real count comes back in the API response (you will use it in Calling Bedrock from C#).
The context window matters for two reasons. First, cost and latency grow with prompt size: sending a 200-page document with every question is slow and expensive. Second, a model can lose track of details in the middle of a very long prompt, so "just put everything in" is not a retrieval strategy.
Embeddings and similarity
An embedding model turns text into a list of numbers (a vector), typically a few hundred to a thousand numbers. The training makes texts with similar meaning land close together. To find the most relevant policy paragraph for a question, you embed the question, embed every paragraph once, and pick the paragraphs whose vectors point in the most similar direction. The usual measure is cosine similarity: 1 means the same direction, 0 means unrelated.
Here is the idea in runnable C#. It uses a toy "embedding" (word counts) so you can see the mechanics without any service. Real embeddings come from a neural network and understand synonyms; the comparison maths is identical.
using System.Text.RegularExpressions;
string[] policies =
[
"A customer may raise a card dispute within 120 days of the booking date.",
"A current account can be closed once the balance is zero and no payments are pending.",
"Branches open from 09:00 to 17:00 on weekdays.",
];
const string question = "How many days do I have to dispute a card payment?";
// 1. A rough token estimate: English text averages about four characters per token.
static int EstimateTokens(string text) => (text.Length + 3) / 4;
Console.WriteLine($"The question is roughly {EstimateTokens(question)} tokens.");
// 2. A toy "embedding": how often each word appears. Real embeddings are 256 to 1,024 numbers produced by a
// neural network, but comparing them uses exactly the same maths as below.
static Dictionary<string, double> Embed(string text) =>
Regex.Matches(text.ToLowerInvariant(), "[a-z0-9]+")
.Select(m => m.Value)
.Where(word => word.Length > 3)
.GroupBy(word => word)
.ToDictionary(g => g.Key, g => (double)g.Count());
static double Cosine(Dictionary<string, double> a, Dictionary<string, double> b)
{
var dot = a.Where(kv => b.ContainsKey(kv.Key)).Sum(kv => kv.Value * b[kv.Key]);
var lengthA = Math.Sqrt(a.Values.Sum(v => v * v));
var lengthB = Math.Sqrt(b.Values.Sum(v => v * v));
return lengthA == 0 || lengthB == 0 ? 0 : dot / (lengthA * lengthB);
}
var query = Embed(question);
foreach (var (text, score) in policies
.Select(p => (p, Cosine(query, Embed(p))))
.OrderByDescending(x => x.Item2))
{
Console.WriteLine($"{score:0.00} {text}");
}Expected output:
The question is roughly 13 tokens.
0.43 A customer may raise a card dispute within 120 days of the booking date.
0.00 A current account can be closed once the balance is zero and no payments are pending.
0.00 Branches open from 09:00 to 17:00 on weekdays.The toy finds the right policy because the question and the policy share the words "dispute", "days", "card" and "payment". Ask it "How long do I have to raise a chargeback?" and it scores 0.00 for everything, because "chargeback" shares no word with the policy. A real embedding model would still rank the dispute policy first. That gap between matching words and matching meaning is why retrieval uses embeddings.
What problem the AWS service solves
This lesson is about concepts, but you should know where each concept lands on AWS so the rest of the module has a map. Amazon Bedrock is the managed service that gives you foundation models behind one API, without you running GPUs. The pieces:
- Plain prompt: you call a model with text and get text back (the Converse API).
- Retrieval-augmented generation (RAG): before calling the model, you search your own approved documents and paste the best passages into the prompt. The model answers from them and can cite them.
- Agent: a loop where the model decides which tool to call (look up an account, search a wiki), your code runs the tool, and the result goes back to the model until the task is done.
The more autonomy you move to the right, the more you gain and the more can go wrong. A plain prompt can only say something wrong. A RAG assistant can say something wrong about a real document. An agent can do something wrong. Each step needs stronger controls, which is why agents get tight limits in a bank.
Prompting, RAG or fine-tuning?
People often ask whether to "train the model on our data". For most bank assistants the answer is no.
| Approach | What it changes | Use when | Banking caution |
|---|---|---|---|
| Better prompting | The instructions and examples in the prompt | Format, tone, simple rules | Cheap and reversible: try first |
| RAG | Adds fresh, approved facts to the prompt at question time | Answers must reflect current policy, with citations | You must control who may see which source document |
| Fine-tuning | Adjusts the model's weights with your examples | A stable style or narrow task, lots of good examples | Bakes data into the model; hard to delete under UK GDPR erasure |
| Training from scratch | Everything | Almost never | Out of scope for a bank's first projects |
Why answers go wrong
| Cause | What it looks like | Does a better prompt fix it? |
|---|---|---|
| Hallucination | A made-up policy number or regulation, stated confidently | Partly: "answer only from the documents" helps, retrieval helps more |
| Stale knowledge | Describes a rule that changed after the training cutoff (the two Claude 5.5 models list a knowledge cutoff of June 2026) | No: add current documents through RAG |
| Missing context | The answer needs a fact you never supplied | No: retrieve it or ask the user |
| Retrieval failure | The right paragraph was not found, so the model improvises | No: fix chunking, search and evaluation |
| Ambiguity | "Refund" means three things in three products | Partly |
| Non-determinism | Two runs give different answers | Lower temperature narrows it; it never reaches zero |
| Prompt injection | Text in a document or email says "ignore your instructions" and the model complies | Never fully; needs layered defences |
| Bias | Different quality or tone for different groups of customers | No: it needs testing and governance |
| Over-trust by humans | The reader believes it because it sounds sure | No: it is a process and training problem |
Notice how few of these are cured by writing a cleverer prompt. Most need a control around the model, which is the engineering this module teaches.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Azure OpenAI Service / Azure AI Foundry | Amazon Bedrock | Many providers behind one set of APIs; you choose the model per call |
Azure.AI.OpenAI chat completions | Bedrock Converse API | One request shape across providers, so swapping a model is a string change |
| Azure AI Search with vector fields | Bedrock Knowledge Bases | Managed ingestion, chunking and retrieval (next lessons) |
| Azure AI Content Safety | Bedrock Guardrails | Filters on both the prompt and the response, plus PII and grounding checks |
Microsoft.Extensions.AI (IChatClient) | Same package, with a Bedrock adapter | Keep your own code provider-neutral |
| GitHub Copilot | Kiro and the Amazon Q Developer console features | Covered in a later lesson |
Architecture
Where does a generative feature sit in a bank's estate? Always behind your own API, never called directly from a browser, and always with a person or a rule between the model and anything that affects a customer.
Security-boundary view: the user, your API and the retrieval store sit inside your trust boundary. The model call crosses into a managed service, so what you send is a data-sharing decision. You will practise every control in that diagram in the lessons and the lab.
Step-by-step implementation
This lesson has one hands-on exercise: run the cosine-similarity program and extend it.
Create the console app
dotnet new console -n TinyEmbeddings -f net10.0
cd TinyEmbeddingsReplace Program.cs with the code above.
Run it
dotnet runYou should see the output shown earlier. If you use Docker instead of a local SDK, mount the folder and run dotnet run inside mcr.microsoft.com/dotnet/sdk:10.0.
Break it on purpose
Change the question to use a synonym (for example "chargeback") and watch every score drop to 0.00. Write down in one sentence why a real embedding model would not have that problem.
AWS Console instructions
None needed. You will meet the Bedrock console in Amazon Bedrock basics.
AWS CLI
None in this lesson. A first command you can try later, once you have a profile, lists the foundation models your Region offers:
# Lists models by provider; needs the learn-sandbox SSO profile and bedrock:ListFoundationModels
aws bedrock list-foundation-models --by-provider anthropic --profile learn-sandbox --region us-east-1Infrastructure as Code
Not applicable: this lesson creates no AWS resources.
Complete code
The full program is the listing above; it needs only the .NET 10 SDK. The lab for this module, Lab 16.1, is the complete, tested codebase.
Code explained line by line
static int EstimateTokens(string text) => (text.Length + 3) / 4;rounds characters divided by four up. It is an estimate for planning only.Embedlower-cases the text, splits it into words with a regular expression, drops words of three letters or fewer, and counts the rest. The result is a sparse vector: a dictionary of word to count.Cosinecomputes the dot product over the words the two vectors share, divided by the product of their lengths. A score of 1 means identical direction.- The final loop scores every policy against the question and prints them best first. This is the retrieval step of RAG in miniature: embed, compare, sort.
Security considerations
โ Learning shortcut โ not for production
The toy embedding in this lesson matches words, not meaning, and it runs on hard-coded sample text. A production system uses a real embedding model, retrieves only from approved documents the user is allowed to see, and removes personal data before anything is sent to a model.
Three habits to start now:
- Treat every prompt as an outbound data transfer. Anything you paste into it leaves your process. Decide what may leave before you write the code, not after.
- Treat model output as untrusted input. Never execute it, never put it in a SQL string, never render it as HTML without encoding.
- Never put secrets, full card numbers or customer identifiers in a prompt unless a data-protection assessment says you may. Your compliance and legal teams own that decision; this course does not claim any approach is compliant.
Banking use case
Sort these fictional Harbourline Bank requests. The categories are the same ones you will use in the last lesson of the module.
| Request | Fit | Why |
|---|---|---|
| Search and summarise the approved card-dispute policy for a call-centre agent | Good fit | Internal, read-only, cites sources, a person decides |
| Draft a first version of an internal release note from a change log | Good fit | Low impact, easy to review |
| Summarise a customer complaint email for a case handler | Controlled fit | Contains personal data: redact first, log, review |
| Suggest a reply to a customer for an agent to edit and send | Controlled fit | A human approves every message |
| Decide whether to approve an overdraft | Not without strong controls | A decision about a customer; needs governance, explainability and a qualified human |
| Let the assistant move money when a customer asks in chat | Not without strong controls | An autonomous action with financial impact |
Common mistakes
- "The model knows our policies." It does not. Why it happens: the answer sounds authoritative. Avoid it by retrieving the document and citing it.
- "Temperature zero means deterministic." It narrows variation, but you can still see differences. Test for stability rather than assume it.
- "A longer system prompt is a security control." A system prompt is an instruction, not an enforcement point. Treat it as one layer.
- "We will fine-tune on customer data to make it smarter." That bakes personal data into model weights. Start with RAG over approved documents.
- Counting characters as tokens. Cost and limits are in tokens; estimate with the four-characters rule and read the real counts from responses.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
dotnet: command not found | .NET SDK not installed locally | Use the Docker image mcr.microsoft.com/dotnet/sdk:10.0 |
Every score is 0.00 | The question shares no words longer than three letters with the policies | Use words from the policy, or move to real embeddings in the next lessons |
error CS8803 about top-level statements | Another file in the folder also has top-level code | Keep one Program.cs with top-level statements per project |
| Output order differs from this page | You changed a policy or the question | Expected; rerun and read the scores |
Cost and cleanup
This lesson costs nothing: it runs locally and calls no AWS service. There is nothing to tear down. Delete the TinyEmbeddings folder if you wish.
Exercises
- ๐ข Guided: run the program, then add a fourth policy sentence about cancelling a standing order. Confirm that a question about standing orders now ranks it first.
- ๐ก Challenge: replace the word-count vectors with word-presence vectors (1 or 0) and add simple stemming (strip a trailing "s"). Does "disputes" now match "dispute"? What new mistakes does stemming introduce?
- ๐ด Live problem: a colleague says, "Let's fine-tune a model on two years of call transcripts so it answers like our best agents." Write five questions you would ask before agreeing, covering personal data, erasure requests, consent, evaluation and who owns the model risk.
Assignment
Write a one-page "GenAI glossary for my team" in your portfolio repository. For each of the nine terms in the vocabulary table, give a one-sentence definition and one banking example. Add a final section titled "Five ways this goes wrong" using the table above, with one fictional Harbourline example per row.
Solutions
Add "A standing order can be cancelled by the customer before the payment date." to the policies array and change the question to "How do I cancel a standing order?". The toy matches "standing" and "order" but not "cancel" against "cancelled", so the score is middling. Rewrite the policy sentence to use the exact word "cancel" and the score rises. The lesson: exact-word matching is brittle, which is why real embedding models exist.
Stemming by stripping a trailing "s" makes "disputes" and "dispute" match, but it also turns "address" into "addres" and treats "bus" and "bu" as the same. The point is not to perfect it: any rule-based matcher either misses synonyms or invents false matches. A learned embedding model captures meaning in context, which is why it is the standard approach for retrieval.
Which fields in the transcripts are personal data or special-category data? What is the lawful basis and were customers told? How would we honour an erasure request once the data is in model weights? How will we measure that the tuned model is better than a retrieval assistant on current policy? Who signs off the model under the bank's model-risk policy? Legal and compliance must answer the first two; engineering cannot decide them alone.
Knowledge check
Answer: No. Generation involves sampling, so some variation is normal. Lowering the temperature reduces it but does not remove it. If a process needs identical output every time, use a rule or a template, or validate and cache results.
Answer: The model is told to answer from retrieved passages and name them. A reader can open the source and check the claim, and an auditor can see what the answer was based on. It reduces, but does not remove, hallucination.
Answer: RAG. You update or replace the document and re-index it. Fine-tuning needs new training data and a new training run, and old behaviour can linger.
Answer: A token is a chunk of text, about four characters of English on average. Providers bill per input and output token, and context windows are measured in tokens, so long prompts cost more and run slower.
Answer: Stale or missing knowledge (the fact is not in the model or the prompt), and retrieval failure (the right passage was not found). Both need data and search fixes, not wording.
Interview questions
- What is the difference between RAG and fine-tuning, and which would you start with for a policy assistant? RAG supplies current approved text at question time and can cite it; fine-tuning changes weights. Start with RAG because it is easier to update, audit and delete.
- Why can a model sound certain and be wrong? It is trained to produce likely text, not verified text. Confidence of tone carries no information about truth. You add grounding, citations and review.
- How would you explain a context window to a product manager? It is the model's working desk. Only what fits on the desk can be considered, and a bigger desk costs more and is slower to search.
- What would make you refuse a GenAI use case at a bank? An autonomous decision or money movement without a qualified human, personal data leaving without a lawful basis, or no way to evaluate and monitor quality.
Certification practice
A bank wants an assistant to answer questions about this quarter's updated lending policy, which was published after the model's training. Which approach is MOST appropriate and cost-effective?
A. Pre-train a new foundation model on the bank's documents B. Fine-tune the model every time a policy changes C. Retrieve the current policy text and include it in the prompt D. Raise the temperature so the model explores more possibilities
Answer: C. Retrieval-augmented generation adds current facts at question time. A is far too costly and slow; B works but is slow, costly and hard to keep current; D increases randomness and makes errors more likely, not fewer.
A team notices that a summarisation feature occasionally states facts that are not in the source document. What is this behaviour called, and what is the BEST first mitigation?
A. Overfitting; train for more epochs B. Hallucination; ground the model in source text and require citations C. Data drift; retrain monthly D. Underfitting; use a smaller model
Answer: B. Unsupported statements are hallucinations. Grounding on retrieved text and requiring citations reduces them. A, C and D describe different machine-learning problems and would not address unsupported statements.
Which TWO statements about embeddings are correct?
A. They convert text into numeric vectors so similar meaning produces nearby vectors B. Cosine similarity can be used to rank stored vectors against a query vector C. They guarantee that retrieved text is factually correct D. They must be regenerated every time the user types a question, for all stored documents E. They remove personal data from the text
Answer: A and B. C is false: retrieval finds similar text, not true text. D is false: you embed documents once and embed only the query at question time. E is false: an embedding is not an anonymisation technique and may still allow inference about the source.
Completion checklist
- I can explain tokens, context windows and embeddings to a colleague in two sentences each
- I ran the cosine-similarity program and broke it with a synonym
- I can name five ways an answer goes wrong and which ones prompts cannot fix
- I sorted the banking use cases into good, controlled and not-without-controls
- I understand that this lesson created nothing in AWS, so there is nothing to clean up
Key takeaways
- A language model predicts likely text; fluent is not the same as factual.
- It knows nothing about your bank unless you put it in the prompt: that is what retrieval is for.
- Most failure causes are cured by controls around the model, not by cleverer wording.
- Autonomy raises both value and risk: plain prompt, then RAG, then agent, each needing stronger controls.
- Start every design with the question "what leaves my process, and who checks the answer?"
Further reading
- Amazon Bedrock User Guide: the service this module builds on.
- Models at a glance (Amazon Bedrock): the current model list, context windows and cutoffs.
- Next: Amazon Bedrock basics.
Independent learning resource โ not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Module 16: Generative AI on AWS for Banking
An awareness-level module on using generative AI safely in a bank: Amazon Bedrock models, knowledge bases, guardrails and agents, and AI coding assistants. You learn to call Bedrock from C#, and above all where GenAI fits, where it must not be used without controls, and which risks a bank's model-risk team will ask about.
Amazon Bedrock basics
Read the Bedrock model catalogue, understand model IDs and inference profiles, know how model access works today, compare pricing options and decide where a request's data may be processed.