A recent decision by Florida insurance regulators to reject Risk Management Solutions' short-term catastrophe model illustrates the controversial nature of the science of prediction. Catastrophe modeling expert Tom Stone spoke with Claims' to shed light on the science behind the models, how recent catastrophes have affected them, and accusations that insurers manipulate models to unfairly raise rates.
How much science is there to predicting catastrophic losses?
A tremendous amount of science and engineering is associated with catastrophe loss modeling. To date, however, the models are based on statistical interpretation of data collected over the last 120 years or so. But information collected more than 50 years ago is often sparse, incomplete, and inaccurate, which makes statistical interpretation problematic. This leads to a significant amount of uncertainty in the meaning of those statistics and how they should be applied to predicting future catastrophes. In addition, the climate is in a constant state of change, so it may not be appropriate to assume what happened in the past will be repeated in the future. As the models evolve and more information is collected, data relevant to current conditions will be used to better predict near-term risk.
Models are most effective when used to assess loss to geographically distributed portfolios with large numbers of locations. They are also most effectively applied to assess risk — or the probability that certain levels of loss will occur — rather than to predict losses from single events. Application for the use of risk assessment attempts to account for all possible outcomes from all possible catastrophic events. Due to unique characteristics of any particular event, models can forecast losses that are significantly above or below what is actually experienced.
Insurers should be aware of these and other model limitations and strike a balance between complete dependence and not trusting them at all.
How did the losses experienced in 2004 and 2005 affect catastrophe modeling?
Every major catastrophe in the past 15 years has had a major impact on catastrophe modeling. Because these events are rare, each event contributes a substantial amount of data for scientific and engineering review. Three major changes occurred following the 2004 and 2005 hurricane seasons. First, the predictions of loss associated with major events were sharply increased to account for a surge in the demand for labor and materials, increased litigation, heightened crime activity, and other non-modeled losses. The models had this feature included before, but they were not well calibrated.
Second, emphasis was placed on the necessity of collecting quality exposure data. The models will calculate losses even though the exposure data is incomplete and inaccurate. As a result, users can be lulled into a false sense of security about their risk measurement capabilities.
Finally, some of the uncertainty associated with the models was revealed with a new feature allowing users to view the impact of changing the hurricane frequency assumptions. Users are now being forced to understand these scientific intricacies and make judgments on appropriate application. This change in particular opened the modeling companies up to intense scrutiny from affected stakeholders, including regulators, insurers, and policyholders.
Some have accused modelers of being in bed with insurers. Do you agree?
Those claiming that insurers use catastrophe modeling to justify rate increases need to consider the scientific and engineering complexity that the models embody. Historically, the models have tended to under-predict losses for many of the larger events. This has led to the changes discussed above.
Nevertheless, the overriding issue is that the statistics underlying model assumptions depend on data that is still in need of enhancement. Until the scientific community can arrive at a consensus regarding natural catastrophe modeling, those impacted most will always be able to claim a bias in the results.
To ensure quality and objectivity in exposure data, it is essential that companies implement an independent audit or review program. Loss aggregation and risk measurement is just as important as keeping sound financial records. Many aspects of the modeling issues over the last couple of years can be attributed to problems with data quality controls rather than a failure of the models themselves. It is crucial for companies to objectively evaluate their data collection and interpretation procedures so that their risk measurement procedures are reliable.
© Arc, All Rights Reserved. Request academic re-use from www.copyright.com. All other uses, submit a request to TMSalesOperations@arc-network.com. For more information visit Asset & Logo Licensing.