Financial institutions rely heavily on quantitative analysis and models in most aspects of decision making. Organizations routinely use models for a broad range of activities, including underwriting credit, costing and pricing risk, determining capital and reserve adequacy, and many other activities. In recent years, models are applied to more complex situations with more ambitious scope, such as enterprise-wide risk measurement, while the markets in which models are used have simultaneously broadened and changed. Changes in regulation have spurred some of the recent developments, particularly the U.S. regulatory capital rules for market, credit, and operational risk based on the framework developed by the Basel Committee on Banking Supervision. The evolution of model risk management continues to be a key focus of financial services regulators and institutions as CCAR (Comprehensive Capital Analysis and Review) and DFAST (Dodd-Frank Act Stress Tests) rules have matured.
While the financial services industry and governing regulatory bodies continue to focus a large amount of attention and dollars on the creation of better quantifiable algorithms and models to predict and protect from the next crisis, there lies beneath these mathematical complexities a sleeping giant that if not adequately addressed by organizations, will leave these intelligent calculations ineffective. The heavy hitter regulations such as Basel and Dodd-Frank concentrate on the model data and results themselves, but the quiet force known as model risk management, is addressed most directly through guidance rather than prescriptive rules (such guidance is issued by the Fed, Supervisory Letter SR 11-7 and the OCC Bulletin, 2011-12, as well as bank examiner and other discreet supervisory practices).
As guidance is just a means of assisting, it allows every organization to choose a sensible framework that best meets its particular culture and organizational environment. However, guidance also leaves the door open to interpretation and that leaves an organization potentially vulnerable to regulatory scrutiny when determining how best to manage its model risk. This level of exposure leaves many risk managers concerned when approaching what can seem a daunting task of instilling a sense of control and process into the art of model development. Our experiences have shown model risk management compliance can be both manageable and highly beneficial to a bank’s overall risk management structure.
Setting up a model governance structure is analogous to establishing a system development lifecycle (SDLC). It is a necessary standard that offers a level of process and control, while facing similar organizational challenges when managing adoption and adherence.
Organizations that integrate model governance policies and procedures with existing IT SDLC processes, have a significantly higher rate of adoption and compliance success.
By adhering to an integrated approach when setting up
a model governance framework, organizations can drive
effective oversight, reduce model risk and move from
mere compliance to business advantage.
The following are key operational elements to consider
when establishing a model governance structure:
Assess your Model Risk Governance
It is vital to match the needs of the business and the
organization’s risk appetite with the goals of the
framework before developing and implementing a model
risk management structure. To identify the critical gaps
in your Model Governance structure, assess the maturity
with the following preparatory questions:
- What is the gap between the regulatory guidance
and existing model development, validation and
documentation policies and practices?
- Has your organization embraced Three Lines of
- Do you have a consistent and agreed upon definition
of model and model use?
- Do you have a well-understood and robust Model
- How will the model governance process integrate with
geographically diverse operations?
- Where do you have to live with some redundancies
and where can you gain efficiencies?
- Is there an existing inventory of models that accurately
reflects what models exist across the organization and/
or in key business functions?
- Can decentralized committees and oversight groups
manage model based decisions or is a centralized
validation approach necessary?
- Can you address ever-changing business needs (e.g.,
need for regular changes, Test & Learn, prototyping,
machine learning, big data), while maintaining an
appropriate level of control?
- Do you have the right talent
A “Rapid Prototyping” approach
can help to maintain controls while
- The business and the modeling teams can partner
to create unique prototype models tested in a
low-impact but real-world environment by limiting
the user-base, financial and reputational exposure
and by rapidly iterating the model design through
- The risk teams can provide conceptual soundness
validation on these prototype models, establish
controls and limits on the use, and complete their
full review ahead of transition from a prototype to
Design a Model Governance Roadmap
Following the assessment of your Model Risk Governance,
time should be spent with key business, modeling, data,
risk, audit and regulatory stakeholders to create
a roadmap that lays out the path and timeline to develop
the model risk framework so sponsorship is established
at the onset of the transformation. Consideration as to
whether the model risk management framework is to be
centrally managed by one function vs. decentralized down
into the business areas should be established up front.
This will enable clear roles and responsibilities to
be delegated as the roadmap is designed.
Aligning corporate goals to the framework objectives will
help direct organizations down the most appropriate path
between these critical decision points. Additionally, the
acknowledgement of the corporate culture’s strengths
and weaknesses will help determine how to implement
the plan. For example, will the corporate culture accept
a big bang approach to address a change required
by a regulatory finding or will the organization need
to take more time to execute through incremental
Define Clear Roles & Assemble the Team
A talented team is at the center of model governance.
It is important to consider the strengths required for each role and include resources who bring project management, technical documentation, audit, and business analysis skills in addition to deep quants to form a well-rounded team that is adept at addressing all the components of a successful risk management framework. The following are examples of key roles that all contribute
to sound model governance practices:
- Model Owner - Primary model development team lead accountable for a model through its life span, from idea through sunset. Manages and implements model changes, including maintaining documentation and communications with affected parties. The Model Owner can reside in each related business unit or can be a shared function that serves all relevant business areas.
- Model Governance Operations/Support - Assist the Model Owner in the management of model development and change, including prioritization of model development and change activities, maintaining model documentation, overseeing governance, performing regulatory reporting and tracking metrics. This can be a centralized team or federated into business areas.
- Model User - The Model User is typically the owner of the business process that uses the model, rather than the individual(s) who may have access to the model or its output. Business areas that directly use a model or that use model output and can request model changes to the Model Owner as business needs transform.
- Model Validation - An independent team which provides “effective challenge” during the model lifecycle, executes model conceptual and technical review and provides approval for model use and model performance thresholds.
- Model Governance Committees - Governance committees can facilitate efficient discussions around Modeling decisions and to solicit diverse perspectives across stakeholders. Make sure to align decision making authority (committee vs. chair vs. individuals) with your organizations culture and decision making framework.
How the team is deployed is an equally important consideration as you set up the model group. The organizational culture is key when determining the best deployment structure.
Understand which structure, whether it be centralized, decentralized or federated, will be met with the most acceptance across the organization to allow for ease of adoption and achievement of cross-organizational support of the new framework and team.
Develop a Comprehensive Model Framework
Model governance ensures that models are built, maintained and used in a repeatable manner that meets both the business objectives as well as the regulatory compliance requirements. Implementing an end-to-end model framework supports well-organized governance by defining the step-by-step process to build, test, document, implement, train users, monitor performance and operate these critical model lifecycle components. A standard and repeatable framework starts with comprehensive policies, procedures, and controls and facilitates common understanding across all model stakeholders.
It is important to also integrate detailed documentation and robust testing throughout the lifecycle. Regulators are focusing their attention not only on the governance process, but also on how comprehensive and usable model documentation and testing is to ensure repeatability, traceability and transparency across the organization. Model documentation conveys a model’s purpose, assumptions, key design decisions and limitations to users, facilitates a faster validation process, and greatly reduces key person dependency concerns. A formal testing structure ensures issues and errors are found early
and reduces the risk of inaccurate model results.
The following are some best practices to consider when
setting up a model framework:
- Standard Model Definition - Agree early on the
definition of a model. This may take longer than
expected depending on the number of key stakeholders
and business lines that need to be involved and the
degree at which standardization is implemented across
the organization. The agreed upon model definition
will set the stage for model governance policies and
processes and what is included in the framework
so be sure this critical step is well documented and
understood by model constituents who will be asked
to follow these new rules once in place.
- Consistency Across Policy, Standards & Procedures - Ensure consistency between Corporate Policy &
Standards and the divisional procedures and practices.
While the divisional approaches may have nuances,
it is critical to maintain the traceability to the corporate
requirements to minimize future misunderstandings in
expectations between the modeling teams, second line
teams and audit functions.
- Use Checklists - Develop a checklist of procedures
and templates that modelers and documentation
specialists agree upon. Both modelers and
documentation specialists should follow this checklist
routinely in building, testing, and documenting models.
- Use of Back-Testing - When setting up a model
performance monitoring framework, establish a backtesting
function. SR 11-7 defines back-testing as “one
form of outcomes analysis; specifically, it involves the
comparison of actual outcomes with model forecasts
during a sample time period not used in model
development and at an observation frequency that
matches the forecast horizon or performance window
of the model.” A back-testing team will help strengthen
the model development unit by building a quality
assurance (QA) function within the model development
group. This QA function should include some individuals
with audit or Sarbanes-Oxley backgrounds who would
work side-by-side with quants to document and test
models as they are being built.
- Simplify Tools - Streamline and digitize your tools (model documentation, modeling needs/requirements documents)
to reduce time wasted on documentation, business case
development, generating time for value-added activities.
Consider workflow tools, corporate wiki solutions to build a
modeling community, and best practices.
- Create a Fast Track - Enable accelerated but wellcontrolled
execution of model changes and new model
development (i.e., fast-track path). For example, a Fast-
Track process provides significant efficiency to both
business and validation teams, in cases where model
parameters are being updated, quick bug-fixes are
implemented, or routine model assumptions are being
Establishing a well-designed
model offers benefits beyond
- Enables risk management personnel to be
more efficient quickly if they have to move
to a new business unit.
- Makes the validation process easier and
- Allows the integration of data management
requirements easier as the critical data
elements for the organization are more
- Provides cross-functional project plans that
accurately forecast development, testing,
and implementation timelines and avoid
constraints for non-fungible resources.
- Eradicates bugs in a controlled environment
early in the development process and prior
- Allows business users to test early,
ensure process functionality, and produce
- Increases user adoption as tangible results
are demonstrated through clear milestones,
reliable handoffs, and predictable and timely
Focus on Data
Well managed model development extends beyond the
model itself. Defining a consistent process for managing
information as inputs and outputs of models is also critical
for achieving strategic business objectives and ensuring
models accurately reflect logical outcomes.
As detailed in Clarendon Partners
white paper “Compliance Driven
Business Value”, regulations such
as Basel 239 are requiring financial
institutions to demonstrate that
their data governance policies
and procedures also align with
organizational practices such as
information security, legal and
This requirement for effective management of data
across the organization requires a holistic view of how
the Enterprise Data Management (EDM) structure ties in
to the model framework in order to provide the required
transparency from reporting all the way through to the
data elements themselves. As previously stated, a key
step in accomplishing this task is to involve enterprise data management stakeholders early in the model
governance framework development. This will ensure
governance policies are in synch and buy-in is gained to
aid in integration across the organization.
Enabling Organizational Growth
A governance standard that is adopted across the
organization will improve the understanding of the model
by the business and allow the model in part or whole to
be reused for other business purposes. This reusability
helps the organization make better informed decisions,
improve risk management practices and gain competitive
advantage. Additionally, a well-planned approach will
result in better models and thus avoid big losses which
would more than offset the cost of good governance. For
example, while regulators may agree with a bank’s model
structure, if they are not comfortable with the controls
around the governance process, they may reject the
required filings or worse, fine the organization for a lack
of control. By making strategic and tactical investments
in a model governance framework, leaders can transform
organizations to not only reduce enterprise risk, but to also
progress the success of the overall business goals.
Share this page: