Generative AI, made clear.
A short, guided read on what generative AI is, how it actually works, and how to use it well at work. Start by picking who you are. The reader shapes itself around you: the chapters on the left, and the further reading inside each one.
New to AI? The four words to know.
If terms like "LLM" are still fuzzy, start here. Four words, and how they nest inside each other.
The umbrella: software that does things we'd call "intelligent", recognizing, deciding, generating.
The main way modern AI is built: learn patterns from many examples instead of being hand-coded. Your fraud models are ML.
ML that creates new content, text, images, code, rather than only scoring. The wave behind ChatGPT and Claude.
The kind of generative AI behind chat assistants: a huge model trained on text that predicts the next words.
Getting oriented
Three things worth being clear on: what generative AI is, how it differs from ordinary software, and how to work with it well. Each point links to a fuller source.
What it is
Generative AI is software that creates original content, text, images, or code, in response to a prompt, rather than only classifying or scoring data that already exists (IBM).
How it differs from ordinary software
Traditional software follows fixed rules. A generative model instead produces the most probable response, works within a limited context window (its short-term memory), reflects a fixed training cutoff, and can state a wrong answer with full confidence, a failure mode called hallucination. It is capable, but not authoritative (Anthropic, AI Fluency).
How to work with it well
Anthropic frames effective use as four habits: delegate the right tasks to the model, describe them clearly, apply discernment to the output, and exercise diligence in verifying results and handling data. Andrew Ng's Generative AI for Everyone covers the same ground for a non-technical audience: how the technology works, effective prompting, and where it fits (DeepLearning.AI).
The family tree
"AI" is one word covering very different things. Most aren't competitors, they're circles inside circles.
An LLM is a kind of generative AI, inside deep learning, inside machine learning, inside AI. Two ideas sit slightly apart: agents are a way of using a model, and neurosymbolic AI is a way of combining a model with logic.
Traditional ML model
The workhorse behind prediction. It learns patterns from labeled examples and outputs a number or a category.
Good for: churn, fraud, and credit-risk scoring, anywhere the answer is a number or a class.
The catch: narrow, needs labeled data, and can't write or explain in language. It scores; it doesn't talk.
Deep learning
Machine learning built from many layers of artificial "neurons." It powers perception, and it's the engine underneath LLMs.
Good for: image and speech recognition, and the modern models built on top of it.
The catch: data-hungry and hard to interpret, a capable black box.
Generative AI
Machine learning that creates new content, text, images, or code, rather than only scoring what already exists.
Good for: drafting, summarizing, and producing first passes.
The catch: it generates the most plausible output, which can be fluent and confidently wrong.
Large Language Model
The language kind of generative AI: a large model that predicts the next words, one at a time, from the context it's given.
Good for: the drafting, summarizing, and question-answering behind chat assistants.
The catch: it hallucinates, and it isn't deterministic, the same prompt can vary. The model itself keeps no memory between sessions; assistants like ChatGPT and Claude now add a memory feature on top, a property of the app, not the model.
Agents
Not a new kind of model, a system. An LLM running in a loop with tools and a goal, taking several steps on its own.
Good for: multi-step tasks where the path varies and can't be hard-coded.
The catch: it can compound its own mistakes and is harder to control and audit. Earn your way up to it.
Neurosymbolic
The hybrid the field is moving toward: neural pattern-matching combined with symbolic logic, rules, and knowledge graphs.
Good for: reasoning across connected records with rules enforced and an answer you can trace.
The catch: newer and harder to build, but it's where trustworthy enterprise reasoning is heading.
Why neurosymbolic works
An LLM is brilliant at language and useless at guarantees. Classic logic is the opposite. Each is strong where the other is weak.
Pure LLMs can't promise they followed a rule, can't show auditable work, and will state a wrong number confidently. Symbolic AI (logic and rules) is rigorous and explainable, but brittle with messy real-world language. Neurosymbolic AI puts them together. A mental model from Daniel Kahneman: the neural side is "System 1" (fast, intuitive), the symbolic side is "System 2" (slow, deliberate). You want both.
The symbolic side, in practice: ontologies and knowledge graphs
Its most popular modern form is concrete, and increasingly the phrasing you'll hear. An ontology is a shared, formal definition of what things are and how they relate (one meaning for "member," "delinquent"). A knowledge graph stores information as entities and explicit relationships (a member owns an account, which is collateral for a loan). Together they're the symbolic half of neurosymbolic.
Beyond RAG: modeling meaning, not just retrieving text
Basic RAG retrieves passages of text that look relevant. The step past it is to model the meaning: define your terms once (semantics), and store how things relate (a knowledge graph), so a system can reason across connected facts and show the path it took. The harness is the operational side of the same move, the layer that runs this with memory, tools, and governance instead of a fresh, forgetful chat each time. You don't have to build any of it to see the point: it's why "get the data modeled" tends to beat "buy a smarter model."
Watch these (~10 min each)
Four short, durable explainers. Tap a card to watch. The two channels that have aged best for clear, non-technical AI teaching.
Cards open on YouTube in a new tab.
What GenAI is, in 90 seconds
Read this first. It's the idea everything else builds on.
A generative AI model is like a very well-read new hire on their first day. It has read an enormous amount and writes fluently on almost anything. But it has no memory of your organization, no access to your systems, and it will answer confidently even when it's wrong. A capable mind with no context and no hands.
Everything useful comes from what you add around it: what you tell it (instructions), what you give it to read (your documents), and what you let it do (tools). The model is the engine. The value is the car you build around it.
A power tool for expertise, not a replacement
Before we go further, the honest version: AI is very good at the mechanical parts of knowledge work, and genuinely bad at the parts that are actually your job.
It drafts, summarizes, searches, and takes first passes tirelessly. It does not know your members, carry your judgment, or hold accountability for a decision. Those stay with you, and in a regulated institution, they have to.
Drafting and rewriting, summarizing long documents, searching across files, first passes, repetitive formatting, explaining jargon.
Judgment, accountability, member relationships, domain context, reading the room, and the final decision.
We've done this before: the word processor, the spreadsheet
Every generation of knowledge work got a new power tool. The word processor didn't replace writers; it freed them from retyping. The spreadsheet didn't replace accountants; it took over the arithmetic and made their judgment the valuable part. The calculator, email, the search engine, each removed drudgery and raised the value of the thinking on top. Generative AI is the newest tool in that line. It changes the mix of your work; it doesn't erase the reason you're in the seat.
The pattern that wins is consistent: the expert who uses the tool well outperforms both the expert who ignores it and the tool used alone. AI automates tasks, not roles.
What AI is actually good at today
Skip the science fiction. Here's the concrete, unglamorous list of what today's AI does reliably, as long as you give it the input to work from.
Condense long documents, call notes, and email threads. A 40-page policy into five bullets a teller can follow.
Member letters, emails, and plain-language explanations. Turn dense jargon into something a member understands.
Sentiment, intent, topic, priority. Read the tone of a complaint, or triage inbound messages to the right team.
Pull names, dates, amounts, and clauses out of messy forms and contracts into clean fields.
OCR: turn scanned forms and PDFs into text and data. Speech-to-text: turn calls and meetings into transcripts.
Grounded Q&A and semantic search: find and answer from your actual policies by meaning, not just keywords.
Also dependable, with a human reviewing: writing and checking code and spreadsheet formulas, drafting data cleanups, and brainstorming outlines, checklists, and options.
Where it still needs a human
The other half of no-hype: the tasks today's AI is unreliable at, or shouldn't do alone. Knowing these keeps you out of trouble.
| Don't rely on it for… | Why | Do this instead |
|---|---|---|
| Exact math & totals | it predicts text, it doesn't calculate | give it a calculator/tool, or check the number |
| Facts without a source | it can invent confident, wrong answers | ground it in your documents |
| Current / real-time info | its training has a cutoff date | use tools that retrieve live data |
| Guaranteeing a rule or compliance outcome | it offers no hard guarantees | enforce with rules and a human sign-off |
| Reasoning across all your messy data | still the frontier (neurosymbolic) | knowledge graphs + human oversight |
| A final, accountable decision | accountability is a human thing | keep a person on the call |
How people actually use it today
The same model shows up in very different places. These are the main ways people work with AI now, from a chat box to your desktop to a developer's editor.
Claude, ChatGPT, Gemini. Ask, draft, summarize, rewrite. The front door for everyone, no setup.
Claude on the desktop. It reads a folder, writes documents, runs analysis, and uses Skills (saved playbooks) and connectors (Slack, Notion). It does the work, not just talk about it. This guide was built this way.
Copilots in Microsoft 365, Google Workspace, and your CRM. AI shows up where you already work.
Cursor (an AI-native editor), Claude Code (agentic coding in the terminal), GitHub Copilot. For people who write code; the AI writes, edits, and runs it.
LangChain / LangGraph and the Claude Agent SDK, wired to your systems over MCP. For teams building their own automations on top of a model.
Pick your focus
The concepts don't change. What changes is the question you're trying to answer. Jump to yours, or read all three.
You're the expert. This is your assistant.
Think of AI as a sharp, tireless assistant who drafts and fetches, while you stay the one who checks and decides.
You've spent years learning how lending, compliance, operations, or member service actually work. None of that is being handed away. The goal of this focus is narrow and practical: make the expertise you already have go further, and faster, by letting a capable assistant handle the parts that don't need you.
You don't need to be technical. You need a clear picture of what the tool is good and bad at, the kinds of work you can hand it, and how to tell when it's genuinely helping. That's the whole focus.
Here's the path we'll walk
- Four concepts, simply. What the tool actually is, so it stops feeling like magic.
- Three kinds of work. Chat, coworking across your documents, and code, from easiest to most powerful.
- The moving parts. What's happening under the hood, and what to keep an eye on.
- What "success" looks like, so you know when it's working and when to stop and check.
Four concepts, explained simply
You don't need to know how an engine is built to drive well. You do need these four ideas.
1. A very well-read intern, not a search engine
It generates the most likely next words from what it read in training. It's not looking up a fact. That's why it's brilliant at drafting and can also be confidently wrong. Think "smart intern who read everything," not "Google."
2. Its memory is a desk, not a filing cabinet
It can only "see" a limited amount at once (the context window). Like a desk: only so much fits, and it's cleared each session. Newer assistants can carry saved notes between sessions (a memory feature of the app, not the model), but the desk itself still empties, which is why giving it the right document matters. You're loading the desk.
3. It will guess, so you verify
Because it produces the most plausible answer, it will sometimes invent a citation or a number with total confidence. This is hallucination, and it's a property of the technology, not a switch you can flip (IBM explains why). Treat every answer as a draft to check.
4. Ground it, and it stops guessing
The fix is grounding: give it the real source and ask it to answer from that. Now the answer is specific, current, and traceable to a document you can point to.
Check yourself: the AI gives a confident answer on a member's eligibility. What do you do?
Writing a good prompt
The single most useful skill, and the one this track keeps pointing back to. A strong prompt usually has four parts.
| Part | What it does | Example |
|---|---|---|
| Role | who it should act as | "You are a compliance analyst." |
| Context | the material and the situation | "Here is our overdraft policy: [paste]." |
| Task | what to produce | "Summarize it in five bullets a new teller can follow." |
| Format | length, structure, audience | "Plain language; flag anything about ID verification." |
When you can, add one example of a good answer. Showing beats describing.
Same task, weak vs. strong
"Summarize this policy." No role, no audience, no format, so you get a generic summary you still have to rework.
"You are a compliance analyst. Summarize this overdraft policy in five bullets a new teller can follow, in plain language, and flag anything about identity verification." Usable on the first try.
Check yourself: you need a member letter explaining a fee reversal. What makes the prompt strong?
Three kinds of work: chat → docs → code
As you get comfortable, the work climbs a ladder. Each rung gives the AI more to work with, and asks for more care.
Rung 1, Chat. You type, it answers. Draft a letter, summarize a call note, rewrite in plainer language. Fast, low-risk, best place to build fluency.
Rung 2, Cowork across documents. You bring your material, policies, a contract, notes, and work over it: "compare these disclosures and list what changed." The biggest day-to-day lift. Under the hood this is retrieval-augmented generation (RAG) (IBM explainer).
Rung 3, Code & actions. It writes and runs code or uses tools to take steps: crunch a spreadsheet, pull from a system, file a ticket. You describe the outcome; it does the mechanical work. This is where real automation lives, and where oversight matters most.
The moving parts
Every AI interaction is the same handful of parts. Climbing the ladder just switches on more of them.
| Part | Chat | Docs | Code |
|---|---|---|---|
| Your instructions | on | on | on |
| The model | on | on | on |
| Your documents | — | on | on |
| Tools & actions | — | — | on |
| Memory (across sessions) | opt | opt | often |
| Your verification | on | on | on |
So what does "AI success" mean?
Most people get this backwards. Success isn't a smarter model. It's the right context, grounded in the right data, with a human verifying.
Creating value with AI, safely
Your question: where do we invest, what do we build, and how do we keep it governed? Four ideas carry most of the weight.
1. The model is the engine, not the car
Standardize on one model now, but keep the freedom to swap. That comes from design: put prompts, memory, grounding, tools, and rules in a layer above the model (the harness), and connect through the open MCP standard. Then "change the model" is a config change, not a rebuild.
2. Workflows before agents
A workflow follows steps you designed; an agent decides its own. Workflows are predictable, cheap, auditable. Use the simplest thing that works, keep humans in the loop for member-facing work, and earn your way up to autonomy.
3. The data foundation is the real bet
Basic "chat with your documents" disappoints on the questions that connect facts across systems. The durable advantage is a governed, relationship-aware view of your data, consistent definitions and the relationships between accounts, products, and interactions. The model keeps changing; clean, connected data compounds.
4. Governance is the price of admission
The NCUA has no separate AI rulebook: it examines AI through the frameworks you already answer to, safety and soundness, vendor risk, fair lending, BSA/AML. Audit trails, validation history, version control, and board-level ownership are how you show your work under those frameworks. Build observability and guardrails in from day one; you can't bolt an audit trail onto a system that wasn't built for one.
Check yourself: a demo answers your document questions flawlessly. What's the one question to ask?
The data foundation is the moat
The unglamorous work that decides whether AI holds up: defining what your data means, once, so every system and every agent can reuse it. It rarely ships a feature this sprint, which is exactly why it gets skipped.
Define it once, or define it eight times
Without shared definitions, every team, and every AI use case, ends up defining "member," "active," and "delinquent" a little differently. Do the semantic work once, as agreed definitions and a knowledge graph of how things relate, and every consumer, agents included, plugs into the same foundation instead of building its own version of reality.
Semantic debt
Skipping the definitions is like technical debt, except it doesn't appear on any roadmap. It surfaces later, when a second team tries to reuse what you built, or when an agent gets a question nobody anticipated and has no structure to reason with. Data quality then becomes a cleanup project instead of something built in.
Someone has to own it
Because it cuts across every team, the semantic layer easily becomes a shared responsibility that belongs to no one. It needs a clear, accountable owner for that roadmap, not a backlog nice-to-have that each team quietly reinvents in its own corner.
Where AI projects fail
The model is almost never the reason. Projects stall one rung below: on data nobody connected, meaning nobody modeled, and decisions nobody owned.
The 2025 numbers were blunt. MIT's Project NANDA found 95% of enterprise GenAI pilots showed no measurable P&L return. S&P Global found 42% of companies abandoned most of their AI initiatives, up from 17% a year earlier. Gartner predicts 60% of AI projects that lack AI-ready data will be abandoned through 2026. Different studies, same diagnosis: these are foundation failures, not model failures.
The four rungs, and how each one fails
1. Retrieve
A chat interface on a raw model answers from training memory: fluent, confident, and ungrounded. Retrieval (RAG) fixes the worst of it by handing the model your documents. But retrieval alone stops at "find text that looks similar." It cannot connect facts across systems.
2. Link
The systems get connected, the data flows, the demo works. This is where most stacks stop, and where the subtle failures start: linked is not modeled. Each source still defines "member," "active," and "delinquent" a little differently, so a cross-system question comes back wrong, with citations. The pipes carry data; they carry no meaning.
3. Model
Shared meaning: definitions agreed once (an ontology) and relationships stored explicitly (a knowledge graph). This is the rung that gives agents and reasoning engines something real to reason over, and it is the rung the failed-pilot statistics point at. Skip it and every use case re-invents its own version of reality; build it without an owner and it quietly drifts.
4. Decide
The point of the whole stack: a decision a person makes and the institution owns, with lineage an examiner can follow. No tool occupies this rung. If a pilot produces insight but changes no decision anyone is accountable for, it demos well and changes nothing.
The landscape, honestly
Each layer below is real, useful, and routinely oversold as the whole answer. The right question for any tool is which rung it stands on, and what it still needs above it.
| Layer | What it does | What it can't do alone |
|---|---|---|
| Data catalog Collibra, Atlan | Describes what data means and where it lives | A glossary describes; it doesn't answer questions or make calls |
| Pipelines & fabric Fivetran, streaming fabrics | Move and link data across systems | Linked is not modeled; a connection carries no meaning |
| Semantic layer & knowledge graph dbt, Neo4j, GraphDB | Model what things mean and how they relate | A model nobody governs drifts, and meaning is still not a decision |
| Reasoning engine Prometheux, Stardog | Infers new facts by applying rules over the graph | Inference is only as good as the modeled meaning beneath it |
| Agent harness Claude Agent SDK, LangGraph, LlamaIndex | Runs the loop: tools, memory, guardrails | On ungrounded data it automates guesswork, faster |
| Evals & observability LangSmith, Arize, your own eval set | Prove behavior and catch drift | Measurement can't supply meaning |
| DataOS The Modern Data Company | The operating system across the rungs: grounds in your catalog, models meaning as governed, ontology-registered data products, and runs agents and inference partners on top | The decision stays human; DataOS exists so the person making it works from governed meaning, and the call is owned, auditable, and compounds |
Check yourself: a vendor demo answers cross-system questions perfectly. Which rung is it standing on?
The model zoo: which model, and whose?
"AI model" isn't one thing. Models come in different sizes and shapes, and you can rent them or run your own. Matching the model to the task is where a lot of cost, speed, and privacy is won or lost.
By size and shape
The big generalist. Best at hard reasoning, drafting, and open-ended Q&A. Most capable, most expensive per use. Your default for complex language work.
A frontier model that "thinks" in steps before answering (GPT-5 thinking, Claude with extended thinking, Gemini thinking). Markedly better where a wrong intermediate step ruins the answer; slower and much pricier per call. The classic cost mistake is routing everything through one.
Compact and fast (millions to a few billion parameters). Great for narrow, high-volume jobs: classify, extract, route. Cheap, low-latency, and can run privately or even on a laptop or branch server. Examples: Phi, Gemma, Ministral.
Understands images and text together. For reading documents, IDs, checks, screenshots, and charts. The engine behind smart document processing. Frontier models read images natively; a dedicated VLM matters when you run your own.
The quiet workhorse. Turns text into vectors so you can search by meaning. Small and cheap; it's what powers "answer from your documents" (RAG).
By ownership: rent the frontier, or run your own
Anthropic, OpenAI, Google. The most capable models, easiest to start, pay-per-use API. Data leaves your perimeter (mitigated by enterprise no-training terms). Fast to adopt; cost scales with usage; some vendor dependency.
gpt-oss, Qwen, Llama, Mistral, Gemma, Phi. Run in your own cloud/VPC, on-prem, or at the edge. Data stays in your control; fixed infrastructure cost, no per-token fee. You own the operations, and the best open models still trail the frontier on the hardest tasks.
Reading the leaderboards
Three kinds of scoreboard, measuring three different things. Arena ranks models by blind, head-to-head human preference votes. Artificial Analysis charts measured quality against speed and price, hosted and open-weight side by side. LiveBench refreshes its questions monthly so models can't have memorized the answers, and SWE-bench tests whether a model can resolve real GitHub issues end to end. Links below.
Inference, edge, and choosing per task
Two words you'll keep hearing, then a simple guide to which model to reach for.
Inference
Running a trained model to get an answer, as opposed to training it in the first place. This is where your ongoing cost and speed live: every request is an inference, and volume times model size drives both the bill and the latency. Bigger model = smarter but slower and pricier per call.
Edge AI
Running a model close to where the data is, on a branch server, a laptop, a kiosk, instead of sending it to a data center. Small models (SLMs and VLMs) make this practical. The wins: privacy (data never leaves the device), low latency (no round trip), and it works offline.
When to reach for what
| The job | Reach for |
|---|---|
| Hard reasoning, drafting, member Q&A over policy | Hosted frontier LLM (Claude / GPT / Gemini) |
| High-volume classify / extract / route, cost- or latency-sensitive | SLM (often self-hosted) |
| Reading documents, IDs, checks (images) | VLM |
| Sensitive data that can't leave, or offline / branch use | Local open-weight model in your VPC or on-device |
| Search over your own documents (RAG) | Embedding model + an LLM |
Harnesses at the edge
If you're going to run AI on your own turf, for privacy, latency, or examiner comfort, the durable design is a layered stack where the data stays inside your perimeter and every layer is swappable.
The core question for a credit union isn't "which model is smartest." It's "where does the data go, and can I prove it never left." Every choice below flows from that one constraint. Treat the layers as durable and the specific tools as replaceable.
The three durable rules
Small-model-first, with a policy gate. A cheap local model handles the bulk of the work; only genuinely hard reasoning escalates to a frontier model, and the gateway is where you enforce which classes of data may never cross that line.
Prototype simple, ship for load. Running a model on a single box is fine to prototype. Anything member-facing needs a production inference server that handles many requests at once.
Log at the gateway and the harness. That log, every prompt, response, and tool action, is your audit trail. In a regulated shop it's not optional, and it's far easier to build in than to add later.
The edge toolkit
The real, current tools for each layer. The names will change; the layers won't. This is a map, not a mandate.
Ollama / LM Studio for prototyping and single-user tools; vLLM / SGLang for production load; NVIDIA NIM for a vendor-supported, air-gap-capable box. (llama.cpp / GGUF runs underneath.)
LiteLLM is the common self-hosted, OpenAI-compatible gateway: routing, failover, budgets, virtual keys, and central logging in one choke point.
NeMo Guardrails, Guardrails AI, and Llama Guard: redact PII on the way in, classify and filter on the way out, log both.
pgvector if you already run Postgres; Qdrant if you don't; Milvus/Weaviate for large scale; Chroma for prototyping. Retrieval stays in-perimeter.
Orchestration/harness frameworks that tie it together: the Claude Agent SDK, LangChain, and LlamaIndex, with tools exposed over MCP.
Open-source models for banking
In 2026 you can build a credible on-prem stack without ever calling a frontier API. For a regulated buyer, the license and the deployment story usually matter more than the last benchmark point.
The most deliberately enterprise/finance-positioned open family, trained on curated business data with a finance domain, and paired with safety and governance tooling. Best for "open, but with an accountable vendor and a lineage story."
Llama 3.x 8B/70B is still the installed-base workhorse with the biggest fine-tuning ecosystem; Llama 4 is the current, natively multimodal generation. License is permissive-but-not-Apache, so have counsel confirm terms.
Mistral and Qwen (Apache 2.0) are clean, efficient choices; Qwen's vision variants are excellent at documents. Phi (MIT) is tiny and cheap for extraction. Gemma is capable under a custom Google license. OpenAI's gpt-oss (Apache 2.0, 20B/120B) is a mainstream on-prem option, the 120B runs on a single 80GB GPU. DeepSeek (MIT) is highly capable, but its Chinese origin draws US regulatory scrutiny; most US regulated institutions pass.
Not a single model but finance-specific data pipelines and LoRA recipes on open bases, a useful reference for sentiment, extraction, and instruction tuning. BloombergGPT is not open, don't plan a build around it.
The expensive mistake: fine-tuning to teach facts
Fine-tuning changes a model's behavior, its output format, vocabulary, and tone. It does not reliably store facts, and the model tends to forget what it knew. So don't fine-tune to inject your policies. Put changing facts in RAG, where they're grounded, current, and citable, and where your data stays on-prem.
Reach for LoRA / QLoRA when you need a consistent output format at scale (say, structured credit-memo JSON) or a house voice that prompting can't hold. QLoRA tunes an 8B–70B model on modest local GPUs, and tuning locally is itself a privacy control. The 2026 sweet spot: a QLoRA-tuned open model with RAG on top; a few thousand good examples plus retrieval captures most of the value.
Banking workflows, and the governance around them
The point of all this is the work. Here's the use-case catalog with the model and pattern for each, and the governance that keeps it examiner-ready.
| Use case | Model + pattern | Human? |
|---|---|---|
| KYC / document extraction (IDs, statements) | Local VLM · IDP extraction | review exceptions |
| KYC/AML + adverse-media screening | Local SLM · RAG + agent | yes (SAR calls) |
| Fraud / dispute narrative | Local SLM · summarize (detection = ML) | yes |
| Credit memo / underwriting narrative | Fine-tuned SLM + RAG | human decides |
| Adverse-action reason phrasing | Local SLM · phrases an explainable model | yes |
| Member service / agent-assist | Local SLM · RAG + next-best-action | rep in loop |
| Compliance & policy Q&A | Local SLM · RAG with citations | optional |
| Call summarization | Local SLM · summarize | light review |
Rule of thumb: default to a local small model; escalate to a frontier model only for open-ended reasoning or messy multi-document synthesis. Every consequential decision keeps a human committer.
Governance you can't skip
Model risk. In April 2026 the Fed, OCC, and FDIC replaced the long-standing SR 11-7 guidance with a more principles-based standard (Fed SR 26-2 / OCC Bulletin 2026-13). Written for the largest banks and not adopted by the NCUA, it is still the framework examiners reach for. It treats generative and agentic AI as "novel and rapidly evolving" and out of formal scope, but supervisors and internal audit apply model-risk discipline to LLM systems by analogy. The prudent posture: treat your AI systems as governed models, with validation, monitoring, version pinning, and a decision record.
NCUA. No separate AI rulebook. It maps AI onto existing frameworks, safety and soundness, third-party/vendor risk, and fair lending, and its 2026 priorities center on credit risk, fraud, and BSA/AML. A board-approved AI policy and a retrievable decision record for any AI touching lending, BSA/AML, or member routing are the prudent posture, one examiners will recognize even though no rule names it.
Texas note. TRAIGA, the state AI law effective January 2026, mainly regulates government use of AI and bans intentionally discriminatory AI; a federally insured institution following existing banking law is largely carved out. Know it exists, don't build your program around it.
Accelerating the work, honestly
The first thing to be clear about: generative AI changes the work around your model, not the model itself.
Take churn. A gradient-boosted or survival model on good features still beats asking an LLM "who's going to churn." What LLMs change is the surrounding work: exploratory analysis, drafting and documenting feature engineering, generating and reviewing code, explaining output for a business audience, turning a vague request into a testable spec. Treat the model as a fast, occasionally wrong analyst beside you, not the algorithm.
Two pages follow: where AI actually fits in your pipeline, and the one thing that quietly decides whether any of it works.
Where it fits in your pipeline
A data-science project is a pipeline. AI speeds up specific stations, and you verify at each one.
Profiling a table, proposing hypotheses, drafting the first pass of a notebook.
Generating transformations, and reviewing yours for bugs and edge cases.
Turning model output into plain language for a business owner, caveats intact.
Letting an agent iterate on a query, with evals and a human check on the conclusions.
Where you let an agent iterate, keep evals and a human check on the conclusions, not just the code. Fast and wrong is still wrong.
Check yourself: AI writes a whole transformation in seconds. What comes next?
The data still decides it
The thing that decides success has nothing to do with which LLM you pick. It's the data.
Churn is a relationship problem: risk lives in the connections across accounts, products, transactions, channels, and service history, and each source defines "member" and "active" a little differently. Reconcile those definitions and the modeling is easy. Skip it and inconsistent features break the project quietly, especially when a feature is computed one way in training and another way in production.
This is your stake in the Builder's semantic layer. The same governed definitions that make an agent's answer defensible to an examiner are the ones that keep "active member" computed the same way everywhere your features are built.
Check yourself: an LLM writes a clean churn feature in seconds. What's the risk?
The library
The explainers, lectures, courses, and papers behind every concept in this reader, organized so you can go as deep as you want on any of them.
One foundation, three focus areas
The domain expert learns to use it well. The builder learns to deploy it safely. The analyst learns to accelerate honestly. All three depend on the same thing.
The model at the center keeps changing, and it will keep getting better. What compounds underneath every focus is the same: clear inputs, grounding in trustworthy data, and a human accountable for the result. Get those right and the technology serves the work. Get them wrong and no model will save you.