What you should take away
- Retrieval does not teach the model your business; it supplies the relevant passages at the moment of answering.
- In regulated answers, citation is the point: an answer a supervisor cannot trace is not usable.
- Retrieval quality, not model quality, is usually what limits autonomous resolution in banking.
- Retrieval adds a largely fixed cost line plus a variable context cost on every single call.
A capable model knows a great deal about the world and nothing about your fee schedule, your forbearance policy, your product variants withdrawn in 2019 but still held by 40,000 clients, or the wording your complaints team is required to use. Retrieval-augmented generation is the mechanism that closes that gap: at question time, the system finds the relevant passages from your own documents and gives them to the model alongside the question.
For a financial institution, the important consequence is not accuracy in the abstract. It is traceability. A retrieval architecture produces answers with sources attached, which means a supervisor, an auditor or a complaints handler can reconstruct why a client was told what they were told.
What it costs, structurally
| Component | Nature | Cost driver |
|---|---|---|
| Ingestion and chunking | Project then recurring | Document count, format variety, PDF quality |
| Embedding | Per document, per re-index | Corpus size × refresh frequency |
| Vector storage and search | Fixed with steps | Index size, replicas, latency target |
| Refresh pipeline | Recurring engineering | How often terms, rates and policies change |
| Access control at retrieval | Recurring engineering | Entitlement model complexity |
| Context tokens at answer time | Variable | Passages per answer × calls per interaction |
Structure is general; values depend on your corpus and latency requirements.
Retrieval quality is the resolution-rate lever
When an agent fails to resolve a banking query, the cause is usually not reasoning. It is that the right passage was never retrieved, or that three contradictory versions of a policy exist and the system had no basis to prefer the current one. That is a content and governance problem wearing an AI costume, and it is why document estate hygiene often returns more than a model upgrade.
Interactive model
What retrieval investment has to buy
Retrieval spend is justified by the resolution rate it unlocks. Raise the retrieval and data line, then raise resolution, and see where the trade turns positive.
Total cost per successful outcome
£1.20
AI-attempted workflow cost
£6.03m
Human escalation cost
£4.48m
Modelled annual operating difference
£16.37m
Where the cost sits
- Human escalationCalculated£4.48m74%Calculated
- Implementation, annualisedCalculated£500k8%Calculated
- Model inferenceCalculated£420k7%Calculated
- Retrieval & dataCalculated£180k3%Calculated
- Failures & retriesCalculated£180k3%Calculated
- Platform, evaluation & monitoringCalculated£150k2%Calculated
- Tools & APIsCalculated£120k2%Calculated
Largest modelled component: Human escalation (£4.48m). Outcome measured per successful autonomous resolution.
Illustrative assumptions. A retrieval programme that lifts resolution by a few points usually pays for itself many times over; one that does not is data tidying with a licence fee.
Questions to ask before funding a retrieval programme
- Which documents are authoritative, and who owns the answer when two disagree?
- How quickly must a rate, fee or policy change appear in answers, same day, or next release?
- How are withdrawn products and historic terms handled for clients who still hold them?
- What entitlement model governs retrieval, and how is it tested?
- How will we evidence, a year later, which version of a document produced a given answer?
If those five have owners and answers, retrieval is an engineering exercise. If they do not, the retrieval programme will discover them for you, slowly, at production cost.
Apply this to your own workload
The figures above are illustrative assumptions. A BillingEngine assessment replaces them with your company's numbers and shows which assumptions decide the answer.