Session 026 IF - Model Risk Management Moderator: Yimin Yang Presenters: George Alvites Charlie Anderson, Ph.D. Gang Ma, FSA
SOA Antitrust Compliance Guidelines SOA Presentation Disclaimer
Model Risk Management Insurance Practices Presented by: George Alvites
Agenda 1. Audience Profile 2. Enterprise View to Model Risk 3. Focus of Insurance Practices: - Assumptions - Documentation - Independent Validation This is a journey, not a race
Enterprise View to Model Risk Model Risk Framework Governance
Examples of Key Processes Risk Management
Design, Development, Implementation, Use
Identification Inventory
I. Methodology Design
II. Assumptions
Risk Assessment
III. Development Life Cycle
IV. Documentation
Independent Validation
V. Ongoing Monitoring
VI. Change Management
Policy and Procedures
Roles & Responsibilities
Awareness (Training)
Project Management
Oversight & Reporting
Huge effort, big reward, long journey
Assumptions Practices Identification & Processes
Risk Management
- Assumptions link to Model Inventory & Purpose
- Effective oversight and challenge
- Key data elements (sound research; methodology rationalization)
- Outcome analysis – reperformance; benchmarking; analytics
- centralization of common assumptions - Challenge forum: independent team or formal oversight committees
Risk Assessment - Uncertainty: relevance of data vs. judgment - Sensitivity analysis and impact assessment
- Change management
- Long term – enterprise sensitivity analysis
Challenges - Interconnectedness: other assumptions: economic, investment, consistency of forecasting - Strategic: enterprise view of key data elements (consistency, accuracy, timeliness) - Operational: implementation practices
Strategic Risk - Decision making facilitated by Assumption Management
Documentation Practices Key Elements
Objectives
Inputs, Assumptions, Calculation, Output
- Sufficient content for model functionality
- Intended use; model methodologies
- Risk based commensurate with inherent risk and model risk profile
- Model testing, uncertainties; limitations - Conservatism: compensating controls - Controls to assess sensitivity analysis; - Interconnectedness risk – upstream and downstream. - Output review controls (backtesting, benchmarking, overrides management) - Model findings – risk ranked and materiality assessment
- Replicability principle Challenges - Version control – model code - Consistency – standardized documentation - Independent challenge review - Cost vs. benefit: implementation effort
Documentation creates an environment of sustainability
Independent Validation Practices Structure
Challenges
- Independent reviewers – SR 11-7 also requires competence and incentives
- SR 11-7 expertise is limited in Insurance industry.
- Evaluation of conceptual soundness and implementation testing
- Competition for talent within regulated entities (Limited PhDs and FSAs)
- Ongoing monitoring plan – detective vs. preventative (change management, changes in products, adjustments, redevelopment, benchmarking, override management)
- Cost vs. benefits: Cost to maintain effective independent program. Benefits seen over time.
- Outcome analysis (comparison of outputs to expected outcomes or range or outcomes; statistical tests or quantitative measures, expert judgement/overrides testing, assumption sensitivity testing)
- Timing of compliance with requirements. - Cultural shift from traditional collaborative review to independent risk. - Risk profile and learning curves are steep
Validation Processes Creates Effective ERM over time
Model Risk Management in Insurance Investment Management 2017 SOA Annual Meeting & Exhibit
Session 26: Model Risk Management
Gang Ma, PhD, FSA, CFA, FRM VP, Investment Risk Management and Quantitative Analysis RGA Reinsurance Company/St. Louis/Investments Oct 16, 2017
Overview of Investment Models
Models for Insurance Investment Management Functions
Models
Inputs/Assumptions
General Management
Investment planning
A/L cash flow projections, expected trading activities, yield forecast, FX forecast
Cash flow projection
Vary by assets
Asset allocation
Returns or yields, risks, correlations, constraints such as duration, convexity, capital
Total return
CFA Institute's GIPS methodologies
Total return attribution
Methodologies to attribute excess return over benchmark to asset allocation/trading and relative risk positioning (e.g. duration)
Investment income attribution
Lack commonly accepted methodology
Portfolio Management
Investment Performance
Trading
Security valuation (Bloomberg, Security attributes, secondary models broker, proprietary valuation models)
Lending
Underwriting, return/risk analysis
Attributes of the borrower or the investment opportunity
Credit rating
Quantitative/qualitative analyses of the issuer or the borrower
WARF
Ratings along with a mapping methodology
C1 capital (credit risk)
Prescribed factors or company's own models
Credit VaR, Conditional Credit VaR
Credit loss distributions, correlations
Market/credit sensitivities
Security valuation models
Liquidity
Estimates of liquidity demand and supply
Stress testing
Scenarios, market shocks, correlations
Asset valuation
Mark to model based on the observable market price of comparable (Level 2); No observable market prices (Level 3)
Investment income
Accounting principles
Investment Risk Management
Investment Accounting
3
Nature and Challenges of Investment Models Nature of Investment Models
Challenges
Most investment models are external
External models may lack transparency and end-user control. Access to external models may be limited due to license cost
One model may depend on one or several other models
Complex modeling structure, ripple effect
Investment models could be highly technical
Modeling results could be misinterpreted or misused if not carefully communicated or fully understood
Investment decisions based on modeling results could have an immediate financial impact
Some investments require long-term commitment. The long-term financial impact of an investment may not be fully known in the near term
4
Examples of Investment Model Risks
Ratings of New Bonds Do Not Vary With Maturities1 Sector AUTO AUTO ENERGY ENERGY P&C P&C RETAIL RETAIL RETAIL RETAIL TECHNOLOGY TECHNOLOGY
Company GM GM BP CAPITAL BP CAPITAL ALLSTATE ALLSTATE AMAZON AMAZON COSTCO COSTCO APPLE APPLE
Rank Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured Sr Unsecured
Coupon 4.20 5.40 1.77 3.28 3.28 4.20 1.90 4.05 2.30 3.00 1.50 3.75
Issuance Date 08/02/2017 08/02/2017 09/14/2017 09/14/2017 12/01/2016 12/01/2016 08/15/2017 08/15/2017 05/09/2017 05/09/2017 09/05/2017 09/05/2017
Maturity Date 10/01/2027 04/01/2048 09/19/2019 09/19/2027 12/15/2026 12/15/2046 08/21/2020 08/22/2047 05/18/2022 05/18/2027 09/12/2019 09/12/2047
Year to Maturity 10 30 2 10 10 30 3 30 5 10 2 30
Moody's Rating at S&P Rating Issuance at Issuance Baa3 BBB Baa3 BBB A1 AA1 AA3 AA3 ABaa1 AABaa1 AAA1 A+ A1 A+ Aa1 AA+ Aa1 AA+
Same credit rating for shorter- and longer-term corporate bonds at issuance because of the comparable expected credit losses modeled A longer-term credit outlook is inherently less certain, in particular for the fast evolving sectors. To what extent do the ratings reflect this? 1. Data from Bloomberg
6
Credit Loss Charges vs. Actual Impairments Insurance companies use credit loss charges for various applications including cash flow testing, asset adequacy analysis, investment relative value analysis, and deal pricing Insurance companies may assess credit loss charges based on rating migration, default, and recovery data1 How do your company’s credit loss charges compare with the actual impairments?
Annual Impairment of Bonds (bps) Year 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Average 2007-2016 Volatility 2007-2016 Average 2010-2016 Volatility 2010-2016
Median of Sample Insurance Companies* 13 130 95 30 22 14 6 2 6 7 36 46 13 11
* Data based on companies' 10K
1. Annual Default Study: Corporate Default and Recovery Rates, 1920-2016, Moody’s, 15 February 2017
7
Forward Swap Rates Tend to Overestimate Spot Swap Rates of the Same Future Date
Diff
3Mo Fwd
Spot Rate
Diff
6Mo Fwd
3/6/17
3/6/16
3/6/15
3/6/14
3/6/13
3/6/12
3/6/11
3/6/10
3/6/09
3/6/08
3/6/07
3/6/06
3/6/05
3/6/04
3/6/03
3/6/02
3/6/01
3/6/00
350 300 250 200 150 100 50 0 -50 -100 -150 -200
3/6/99
9 8 7 6 5 4 3 2 1 0
3/6/98
12/8/16
12/8/15
12/8/14
12/8/13
12/8/12
12/8/11
12/8/10
12/8/09
12/8/08
12/8/07
12/8/06
12/8/05
12/8/04
12/8/03
12/8/02
12/8/01
12/8/00
12/8/99
12/8/98
250 200 150 100 50 0 -50 -100 -150 -200
12/8/97
9 8 7 6 5 4 3 2 1 0
Diff (bps)
5Yr Swap Rates - 6Mo Fwd vs. Spot of the Same Future Date
Rates (%)
5Yr Swap Rates - 3Mo Fwd vs. Spot of the Same Future Date
Diff (bps)
Forward swap rates appear to be a poor estimator of the actual spot rates of the same future date How much may be influenced by the central banks’ monetary policies and market liquidity?
Rates (%)
Difference (Forward Swap Rate - Spot Swap Rate) 3Mo 6Mo 1Yr Average difference (bps) 16 32 65 Average difference (%) 7.8% 15.9% 32.6% Volatility of difference (bps) 49 71 98 Freq of positive difference 65% 68% 74% Freq of negative difference 35% 32% 26% Avg given positive difference (bps) 43 69 109 Avg given negative difference (bps) -34 -45 -62
Spot Rate
8
Forward FX Rates May Be Better Estimator of Spot FX Rates of the Same Future Date Difference (Forward FX Rate - Spot FX Rate) CAD/USD GBP/USD Average difference ($) $0.00 $0.01 Average difference (%) -0.25% 1.10% Volatility of difference ($) $0.07 $0.15 Freq of positive difference 48% 53% Freq of negative difference 52% 47% Avg given positive difference ($) $0.05 $0.12 Avg given negative difference ($) -$0.06 -$0.12
Certain forward FX rates outperform forward swap rates as an estimator of the respective spot rates of the same future date Better market liquidity and less central banks’ intervention?
Diff
1Yr Fwd
Spot Rate
Diff
1Yr Fwd
9/8/17
9/8/16
9/8/15
9/8/14
9/8/13
9/8/12
9/8/11
9/8/10
9/8/09
9/8/08
9/8/07
9/8/06
9/8/05
9/8/04
9/8/03
9/8/02
9/8/01
9/8/00
9/8/99
-$0.40 9/8/98
$1.00
$ Per £1
-$0.30
Diff ($)
$0.60 9/8/17
-$0.20
9/8/16
$1.30
9/8/15
-$0.20 9/8/14
$0.70 9/8/13
$0.00
9/8/12
$1.60
9/8/11
-$0.10
9/8/10
$0.80
9/8/09
$0.20
$0.00
9/8/08
$1.90
$0.90
9/8/07
$0.10
9/8/06
$1.00
9/8/05
$0.40
9/8/04
$2.20
9/8/03
$0.20
9/8/02
$1.10
9/8/01
$0.60
9/8/00
$2.50
9/8/99
$0.30
9/8/98
$ Per £1
$1.20
Diff ($)
GBP/USD FX Rate - 1Y Fwd vs. Spot of the Same Future Date
CAD/USD FX Rate - 1Y Fwd vs. Spot of the Same Future Date
Spot Rate
9
Other Examples of Investment Model Challenges Forecast of inflation • TIPS implied vs. Econometric models
Models involving human behavior • mortgage and credit card prepayment models used for RMBS and ABS
Correlation of asset returns • may vary by the collective risk appetite of investors
10
Investment Model Risk Management
Ways to Manage and Mitigate Investment Model Risk
Inventory models Rank models based on potential financial impact Ensure to have adequate model accesses and modeling expertise Be knowledgeable about the models, modeling process, assumptions, inputs, constraints and limitations For external models that require user inputs, establish robust processes and procedures to ensure that the inputs are up-to-date and provided according to the procedure When making investment decisions based on modeling results, look at a range of possible results instead of a single point Couple modeling results with sound judgment and experiences Conduct periodical review and validation of critical models Disclose and communicate model constraints and limitations clearly
12
