Models & Controls
A model is the easy half. The record of how it was built is what gets examined.
Scorecards, forecasting models and detection models for UK financial firms, built by someone who has spent fifteen years building them inside one — and delivered with the development documentation that has to survive a supervisor, an internal auditor or a challenge from a firm’s own validation function reading it cold. The same discipline covers the controls those models sit inside, and the controls that never needed a model at all.
Risk Audit Flow
LiveData inputs
Origin & lineage
Model behaviour
Logic & drift
Controls
Approval paths
Decision gate
Override check
Output review
Audit trail
Review area 1 of 5
The shift
The model became the decision
Lending, pricing, affordability and detection decisions are now made by estimates rather than by people reading a file. That moved model risk out of the quantitative team and into the list of things a board is asked to evidence control of.
The gap
The model is documented, the control is not
Firms hold model documentation because a regime asked for it. What they are usually missing is the smaller, harder artefact: a written statement of what the control is meant to catch, and evidence that it does — including the day it was tested and did not.
The standard
A control you can fail on purpose
The test we apply to our own work: state what the control claims, define what failure looks like, then produce the case that breaks it. A control nobody has ever seen fail is a control nobody has tested, whatever the documentation says.
How a control is judged
Six dimensions decide whether a control is defensible
The same six whether a fitted model sits underneath it or a threshold does. Where a dimension cannot be evidenced we report it as unevidenced, never as sound — missing evidence is not the same as a working control.
01
Purpose, stated
What the control is for, in a sentence, and what would count as a use beyond it.
02
Failure, defined
What it would look like for this control to be wrong — written before the testing, not after.
03
Tested against that
Evidence it was tried against the failure definition rather than demonstrated against a happy path.
04
Boundaries declared
The range it was built and estimated over, so operating outside it is a flagged condition and not a silent one.
05
Owned
A named person accountable for it and for the decisions it shapes — not a team, a person.
06
Re-testable without us
The firm can re-run the test after we have gone. A control only we can verify is a dependency, not a control.
What gets built
Six kinds of work, one discipline
Some of this is six months and a fitted model. Some of it is a threshold, a reconciliation and somebody named to look at the exceptions. Deciding which is usually the first piece of work.
Application and behavioural scorecards
Credit risk at origination and through the life of the account. Weight-of-evidence scorecards where the points have to be explainable, with a gradient-boosted challenger fitted alongside so the cost of that explainability is a measured number rather than an assumption.
Affordability and income resilience
The question a scorecard does not answer. CONC 5.2A names two risks — the risk to the firm that the customer will not repay, and the risk to the customer of not being able to. A score answers the first. This answers the second, over the term, on rate and inflation paths.
Revenue and balance-sheet forecasting
Driver-based by business line, with an interval on every figure and the point where the short and long horizons meet checked rather than smoothed over. A five-year point forecast with no interval is not a forecast anybody can defend.
Fraud and financial-crime detection
Behavioural, beneficiary and network features, with the operating point chosen against the capacity of the team that works the alerts. A model generating more alerts than can be worked has not been deployed, whatever it scores.
Controls with no model under them
Thresholds, limits, reconciliations and exception routes. Often the right answer, and saying so is part of the job — a firm sold a model where a threshold would do has bought monitoring it did not need, in a decision it now understands less well than before.
Model and control inventory
What exists, what tier it is, who owns it, when it was last tested and what it depends on. The unglamorous one, and the first thing asked for when somebody senior wants to know how exposed the firm is.
The deliverable
The firm owns the model, the code and the record.
Not a licensing preference. A firm that cannot run, inspect and change its own model without us cannot evidence control of it, and we would have become a dependency inside a regulated decision. Everything below is handed over, in a form somebody else can pick up.
Purpose and approved use
What the model is for, and what would be a use beyond it.
Population and exclusions
The sample, the windows, and every row that did not make it, counted.
Assumptions, named
Including the ones nobody can verify — reject inference above all — with their effect shown rather than absorbed.
Operating boundaries
The range the parameters were estimated over, so extrapolation is a flagged condition and not a silent one.
Limitations
What the model cannot see, written by the people who built it.
Tier, and the interval it demands
Materiality and complexity, and what monitoring cadence follows from them.
How engagements are shaped
Start with a review of what already exists
Most firms have more controls than they have evidence about them. The first rung reads what is already there and says where it stands, on documentation alone — no production data and no system access, which keeps it below the procurement and DPIA wall.
Start here
Control or model review
Fixed fee, scoped on the count
- A handful of controls or one model, read cold
- Scored against the six dimensions
- Written verdict, with what is missing named
- Documentation only — no production data, no system access
Then
Build
Scoped on the problem, not the technique
- Whether it needs a model is part of the work
- Built, tested against a written failure definition
- The full development record, handed over
- The firm owns the model, the code and the record
Ongoing
Monitoring support
On the cadence the tier demands
- Performance read on a set cycle
- Boundary breaches surfaced, not smoothed
- Re-test when the population moves
- Independent review stays with the firm or a third party
What we cannot be
We cannot build your model and be its independent validation.
PRA SS1/23 Principle 4 makes independence the control, and extends it to the independent review of monitoring reports. A validation performed by the party that built the model does not satisfy it, and neither does a monitoring reading performed by them.
So a firm buying a build from us has bought a model and the record of how it was made — not its own independent validation, which has to come from its validation function or a third party. We say it here rather than in an engagement letter, because a letter is read after the decision. Where a model was built by someone else, our automations read its monitoring record as an outside instrument, and that is the side of the boundary they sit on.
One scope question worth settling early: SS1/23 binds firms with internal model approval for regulatory capital. A standardised-approach lender sits outside it and is welcome to adopt the principles voluntarily — which changes who signs, and what an unmet expectation means.
And where this work stops: designing the control and evidencing that it works is this page. Running it — the workflow it lives in, the process rebuilt around it, the monitoring that watches it daily — is AI & automation. A control and the pipeline that executes it fail in different ways, and are usually bought by different people.
Seeing one
Worked models are shown by invitation, not published
Every one of them fits itself in the browser over a constructed population, so what you are looking at is arithmetic running rather than a screenshot of a result — and none of it is any firm’s data. Every enquiry is treated in confidence.