Feb 1, 2013

Weathering Model Risk: What Tropical Storm Debby Can Teach Financial Institutions


James JockleJames J. Jockle, Chief Marketing Officer at Numerix, addresses the issue of model risk and how a lack of model uniformity can present significant risks.

If you go back twenty years ago, the most distrusted individual on the evening news was the weatherman (unless of course you live in San Diego where it’s always sunny). If the weather man called for a light dusting of snow, you could get rain or a blizzard.  The umbrella was a standard tool in the briefcase.  But we’ve advanced. Now you can go to your mobile device punch up weather.com and get the day’s forecast in 15 minute increments or for the next 10 days – and it’s right.

Well, not always.

This summer I found myself in Florida for the arrival of tropical storm Debby.   For many, Debby, like many storms came and went barely noticed, but it was sizable. The storm featured sustained 65 mph winds, spawned 25 tornadoes and dropped more than 20 inches of rain. Now, this is not a blog about storms and milibars (low pressure was 990 mbar, by the way) but a statement about model risk.


When watching the weather channel, the meteorologists had a long discussion on the storm’s path.  What was unusual for this storm was the disagreement between the two main predictive models – the American and European models.  In terms of disagreement – one model had the storm going to Miami, the other to San Antonio.  I watched the coverage with keen interest because I didn’t know if I needed my golf clubs or to go to the grocery to stockpile bread, water and peanut butter.


For those who are interested in the divergence of the models, here is the brief synopsis from redOrbit Meteorologist Joshua Kelly.

“Tropical Storm Debby will be remembered as being a challenging storm to forecast for as computer-derived weather models had this storm hitting anywhere along the Gulf Coast over the weekend. As time wore on  the models started leaning together pushing the storm on a more northward track which it presently is on now. Two features that lead to this challenging forecast were an upper level trough which was moving through the Eastern Seaboard and if Debby got close enough it would be picked up by the trough and sent northward. A second scenario was for Debby to miss the trough and get picked up by a High pressure area that was moving in from the west. This was a reason behind the models thinking the storm would go westward and become a Hurricane, because of the longer time over the warmer waters. A third option was also seen and it looks like as of this morning this option might be what Debby is following and that is stalled between both features and just kind of beginning to drift slowly in and not show much forward progression and that is what has been occurring in Debby over the past 6 hours with little to no movement.”

Well, in terms of my own “V VaR” analysis (Vacation VaR), I predictably found myself on the wrong side of the weather. As I enjoyed my peanut butter sandwich, I came to ponder the following question – how can something like the weather that has been become so predictable, be so inaccurate?

Turning to Derivatives, even the ‘steady as she goes’ ‘well performing’ pricing and risk model can become suddenly inaccurate.  Parallel to the Debby example, even changes in market environments (volatility, volumes, etc.) can impact model performance.

Managing model risk is centered on three areas:

  1. Knowing and understanding the assumptions behind the models
  2. Problems with the data quality that goes into the models
  3. Understanding the results, and interpreting them accurately

Why is this important? Standardization of market implicitly suggests the concept that things are often simplified.  The truth is that in terms of valuation and risk measurement things are getting more complex. Over the past six months, Numerix experts have exemplified many of the new challenges in a standardized world. For example, the growing complexities of managing collateral and ‘cheapest to deliver’, or curve construction and the impact on the valuation process with OIS discounting.  What about pricing non-linear instruments and the extension of models to the multi-curve framework? - taking a deterministic approach has significant implications on performance and can produce significant pricing differences.

As market analysts continue to talk about the convergence of front-to-back operations and the growing importance of understanding funding or the economic value adjustment of a trade at the point of pre-execution, having a centralized analytic platform, coupled with a strong model validation process, continues to become a center of importance. Moreover, lack of model uniformity can present significant risk to mispricing and hedging, and model quality and transparency is essential to minimize capital charges. But, this is not new.  There has been much written on this topic for 10 years.  What is new, is that technology has advanced to provide the front office the flexibility required to manage its business while giving the organization product and model control throughout the infrastructure.

Lessons learned?  Never assume just because it’s right today, it will be right tomorrow. Market conditions and weather patterns can have a profound impact on model performance.  Finally, to effectively manage ‘V VaR’, bring an umbrella.

Need Assistance?

Want More From Numerix? Subscribe to our mailing list to stay current on what we're doing and thinking

Want More from Numerix?

Subscribe to our mailing list to stay current on what we're doing and thinking at Numerix

Subscribe Today!