Models & Controls
Build the models a firm depends on, and the controls around them
For the risk, finance, or operations lead at a UK bank or financial services firm who is asked to show a control works. Two kinds of work, and they are not the same thing. A model produces an estimate that did not exist before: a score, a forecast, a probability. It is judged on how it was fitted and what it can be shown to do. A control decides or checks something and is judged on whether it is documented, owned, tested, and traceable to the obligation it implements. Most firms need far fewer models than they are sold. Both are delivered with the documentation a supervisor, an internal auditor, or the firm’s own validation function can read without us in the room. Fixed fee, agreed before we start.
Built by practitioners who have done this work inside banks and regulated firms.
Risk Audit Flow
LiveData inputs
Origin & lineage
Model behaviour
Logic & drift
Controls
Approval paths
Decision check
Override available
Output review
Audit trail
Review area 1 of 5
An audit is only worth its conclusion when every area was examined. An area nobody reached is reported as unknown, never as cleared.
The shift
The model became the decision
Lending, pricing, affordability and detection decisions are now made by estimates rather than by people reading a file. That moved model risk out of the quantitative team and into the list of things a board is asked to evidence control of.
The gap
The model is documented, the control is not
Firms hold model documentation because a regime asked for it. What they are usually missing is the smaller, harder document: a written statement of what the control is meant to catch, and evidence that it does — including the day it was tested and did not.
The standard
A control you can fail on purpose
The test we apply to our own work: state what the control claims, define what failure looks like, then produce the case that breaks it. A control nobody has ever seen fail is a control nobody has tested, whatever the documentation says.
Models
Where the question genuinely requires an estimate
A model earns its cost only where the question requires an estimate and no rule can produce one. Each is fitted, documented, and delivered with the record of how it was built. Credit models are on the Credit Risk page.
Fraud and financial-crime detection
Behavioural, beneficiary and network features, with the operating point chosen against the capacity of the team that works the alerts. A model generating more alerts than can be worked has not been deployed, whatever it scores.
Controls
Most controls need no model at all
Controls make up most of what a firm is judged on, and they are judged by a different test from models. Deciding which of the two a problem needs is usually the first piece of work, and it is done before anything is built.
Controls with no model under them
Thresholds, limits, reconciliations and exception routes. Often the right answer, and saying so is part of the job. A firm that buys a model where a threshold would do has bought monitoring it did not need, and a decision it now understands less well than before.
Model and control inventory
What exists, what tier it is, who owns it, when it was last tested and what it depends on. The unglamorous one, and the first thing asked for when somebody senior wants to know how exposed the firm is.
How a control is judged
Six dimensions decide whether a control is defensible
The same six apply whether a fitted model sits underneath the control or a threshold does. Where a dimension cannot be shown we say so, and never record it as sound — a missing record is not the same as a working control.
01
Purpose, stated
What the control is for, in a sentence, and what would count as a use beyond it.
02
Failure, defined
What it would look like for this control to be wrong, written before the testing rather than after.
03
Tested against that
A record that it was tried against the failure definition rather than demonstrated against a happy path.
04
Boundaries declared
The range it was built and estimated over, so operating outside it is visible rather than silent.
05
Owned
A named person accountable for it and for the decisions it shapes. Not a team. A person.
06
Re-testable without us
The firm can re-run the test after we have gone. A control only we can verify is a dependency, not a control.
The deliverable
The firm owns the model, the code, and the record
Not a licensing preference. A firm that cannot run, inspect, and change its own model without us cannot show it is in control of it, and we would have become a dependency inside a regulated decision. Everything below is delivered in a form somebody else can pick up.
Purpose and approved use
What the model is for, and what would be a use beyond it.
Sample and exclusions
The sample, the windows, and every row that did not make it, counted.
Assumptions, named
Including the ones nobody can verify, with their effect shown rather than absorbed.
Operating boundaries
The range the parameters were estimated over, so extrapolation is a flagged condition and not a silent one.
Limitations
What the model cannot see, written by the people who built it.
Tier, and the monitoring it demands
Materiality and complexity, and the cadence of review that follows from them.
How engagements are shaped
Start with a review of what already exists
Most firms have more controls than they have evidence about them. The first step reads what is already there and says where it stands, on documentation alone — no production data and no system access, which keeps it small enough to approve without a procurement exercise and outside the need for a data protection impact assessment (DPIA).
Start here
Control or model review
Fixed fee and timetable agreed before we start, after a twenty-minute call.
- A handful of controls or one model
- Scored against the six dimensions
- Written verdict, with what is missing named
- Documentation only: no production data, no system access
Then
Build
Scoped on the problem, not the technique
- Whether it needs a model is part of the work
- Built, tested against a written failure definition
- The full development record, delivered with the model
- The firm owns the model, the code, and the record
Ongoing
Monitoring support
On the cadence the tier demands
- Performance read on a set cycle
- Boundary breaches surfaced, not smoothed
- Re-test when the data moves
- Independent review stays with the firm or a third party
What we cannot be
We cannot build your model and be its independent validation
PRA SS1/23 Principle 4 — the Prudential Regulation Authority’s supervisory statement on model risk — makes independence the control, and extends it to the independent review of monitoring reports. A validation performed by the party that built the model does not satisfy it, and neither does a monitoring reading performed by them.
So a firm buying a build from us has bought a model and the record of how it was made — not its own independent validation, which has to come from its validation function or a third party. We say it here rather than in an engagement letter, because a letter is read after the decision. Where a model was built by someone else, our automations read its monitoring record as an outside instrument, and that is the side of the boundary they sit on.
One scope question worth settling early: SS1/23 binds firms with internal model approval for regulatory capital. A standardised-approach lender sits outside it and is welcome to adopt the principles voluntarily — which changes who signs, and what an unmet expectation means.
And where this work stops: designing the control and showing that it works is this page. Running it — the workflow it lives in, the process rebuilt around it, the monitoring that watches it daily — is AI & Automation. A control and the pipeline that executes it fail in different ways, and are usually bought by different people.
Seeing one
Worked models are shown by invitation, not published
Each one fits itself in the browser over constructed data, so what you see is the arithmetic running rather than a screenshot of a result — and none of it is any firm’s data. Every enquiry is treated in confidence.