MODEL RISK MANAGEMENT • OSONWANNE GROUP LLC
ModelProof: validation reports where every sentence shows its evidence
ModelProof automates the repeatable, evidence-heavy parts of an independent model validation (planning, evidence review, testing and report drafting) and keeps every consequential judgment with the validator. Every drafted sentence cites its source, and a checker flags anything that doesn’t.
A full periodic validation, end to end
A representative Tier 1 SME probability-of-default scorecard (synthetic data, fictional institution) with deliberately seeded defects.
12 / 12
Seeded defects surfaced, from calibration failure to an unreproducible AUC
32 of 33
Drafted sentences cite evidence; the last is the opinion, left for the validator
0
Influence from a hidden instruction planted in the model document; it was quarantined
~1 s
For ingestion, 7 tests, claim checks and a full draft over 16,000 records, in the browser
What it automates
Evidence and gap analysis
Model documents, code and datasets become citable, hashed evidence. Documentation is mapped to the validation standard, and the developer’s own claims are recomputed and checked.
Independent testing
Discrimination, grade-level calibration, stability, data quality, re-estimation of documented coefficients and a challenger model, all re-performed with published methods.
Cited report drafting
Every section is drafted against the report template, and every sentence cites a record. Export to Word with approval stamps, a findings register and the evidence register.
The validator stays accountable
Automation proposes; people decide. Every decision is recorded in a tamper-evident audit trail.
FINDINGS
Guided review
Each proposed finding arrives with its evidence. The validator accepts it, adjusts severity, or rejects it with a logged reason.
APPROVAL GATES
Human sign-off
The opinion is never drafted by the tool. Each section needs a named approver, and approving past an open flag requires a written override.
AUDIT
Hash-chained trail
Every automated step and human action is chained by hash. Edit any event and verification fails; you can try the tamper test in the demo.
Controls for generative AI risk
No unsupported statements
The checker flags uncited sentences, citations that don’t resolve, and numbers that don’t appear in the cited evidence, live on every edit.
Prompt-injection quarantine
Instructions hidden in source documents are detected, excluded from analysis and drafting, and raised as a finding.
Your environment, your data
Designed for deployment in the client’s own cloud tenant or on-premises, with in-region hosting. Client data is never used to train models.
Structured on the OSFI Guideline E-23 model lifecycle. Drafting in the prototype is deterministic; a generative mode would sit behind the same checker and approval gates.
See it running
ModelProof was prepared in response to Export Development Canada’s Request for Information on model validation automation technologies. Explore the working prototype, with no login needed, or book a walkthrough.
Questions? Contact us • Bon@Osonwanne.com • 602-632-9589
ModelProof is a working prototype by Osonwanne Group LLC. The demonstration uses synthetic data about a fictional institution; it contains no client or EDC data. Responding to an RFI does not imply selection or endorsement by Export Development Canada. ModelProof supports, and does not replace, independent validator judgment.