Are insurance carriers putting too much weight on the results ofcatastrophe models in underwriting risks and managing theirconcentration of exposures? Have insurers allowed rating agenciesand reinsurers to bully them into an overreliance on the models,and do users understand the uncertainties inherent in suchpredictive analytics?

Those are some of the criticalquestions that have been raised over the past couple of years byone of the pioneers in the cat modeling industry–Karen Clark,founder of the first catastrophe modeling firm, Applied InsuranceResearch (which later became AIR Worldwide Corp., after itsacquisition by the Insurance Services Office in 2002).

Ms. Clark is currently president and chief executive officer ofKaren Clark & Company in Boston, a consulting firm that helpscompanies with risk management processes, including interpretingand using cat model results.

This is not a new controversy, but it is one that has yet to besettled. Indeed, as far back as April 2008, Ms. Clark, speaking ata gathering of the Association of Professional Insurance Women inNew York, said the industry had grown too dependent on cat modelsand “stopped thinking about risks independently.”

She stressed that models are not absolute truths, but rathertools that offer generalized best estimates. They can containuncertainties, limitations and even inaccuracies, she warned,insisting they are not designed to replace underwriters or be thefinal word on which risks are acceptable to an insurer.

Catastrophe modelers that spoke to National Underwriteragreed their products and systems are essentially supporttools.

Explaining his view of what models can do for insurers, JayantaGuin, senior vice president of research and modeling at AIRWorldwide, said that “models provide a robust framework todetermine, 'What is risk?'” He said the models help give an insurera full probabilistic view of its exposure, and act as a tool tohelp the company make a determination on how to minimize risk.

Models have also “introduced a standard into the industry whereeveryone is speaking the same language,” Mr. Guin added.

Particularly, he said, “models are great tools for[companies] looking to expand,” since models can help companiesidentify areas for growth.

The issue, then, is not so much what the models are, but ratherhow insurers are using them, and then interpreting the results.That, according to Mr. Guin, is up to the insurance carriers. “Howeach individual company makes decisions is largely up to them,” hesaid. “Models are tools.”

But to make those decisions, Ms. Clark told NU thatinsurers need to understand the models and the uncertaintiescontained within them.

She said companies should ask whether the models can be reliedupon to produce credible information. The answer, she said, is a“resounding yes” in many areas, but other areas are morequestionable.

For example, insurers sometimes use a model's “location levelloss estimates” to optimize their portfolios by cutting off theirexposure in areas where the loss estimates are high, Ms. Clarknoted.

But at the location level, model estimates are subject to “verywide swings,” she warned, pointing out that a change to a detailedmodel assumption of just 10 percent could cause a 100 percentchange in a location level loss.

Mr. Guin acknowledged potential unknowns in the models. “As wetry to quantify risk, there are many sources of uncertainty,” hesaid.

He reiterated, however, that it is up to the insurer todetermine how it applies the model results in its underwriting,given those uncertainties. “When a model produces an estimate withuncertainty–and we can help people understand uncertainty–it's upto the company how [it uses] that information,” Mr. Guin added.

Robert Muir-Wood, chief research officer of another modelingfirm–RMS–said modelers are working on the representation ofuncertainty and helping clients see “where are we making steps ofinference and how do we think about that?”

The uncertainty in models is all about the data, Ms. Clarksaid–or in many cases, the lack thereof. “Modelers are limited bythe lack of scientific data,” she explained.

Modelers are able to obtain their best data about the damage agiven catastrophe will do in a certain area when such an eventactually occurs, she said.

During the active hurricane seasons of 2004 and 2005, forexample, Ms. Clark said modelers were able to “slice and dice” theresulting claims data and fine-tune damage functions. Essentially,major events that produce a lot of claims can allow modelers tobetter calibrate the models.

But some modeled catastrophes have little historical dataassociated with them.

For example, Ms. Clark said the New Madrid Seismic Zone produceda series of earthquakes in the central United States in 1811 and1812, but much is unknown about the intensity and resulting damagefrom those quakes.

Some newspaper accounts, she said, refer to church bells ringingand cracked sidewalks, but ultimately there is little data, andtherefore there will be uncertainty in the model results for asimilar event today.

Mr. Guin said data quality is not the same across all perils andregions. “One rule that can be applied,” he said, “is phenomenasuch as hurricanes, that occur more frequently, are betterunderstood.”

Also using the New Madrid Seismic Zone as an example, he saidthe lack of loss experience data translates into estimates that aremore uncertain than those for earthquakes in California, wherethere is more data.

However, for a catastrophe or region where there is not a lot ofhistorical data to draw on, modelers can build on catastropheexperience around the globe, according to Mr. Guin. AIR, he noted,builds models for over 60 countries, and that collective experiencecan provide guidance.

In addition, he said, other scientific processes that are notdata-driven can be used. For an earthquake along the New MadridSeismic Zone, Mr. Guin noted, paleoseismology–examining rocks forsigns of previous earthquakes–can help in modeling futureoccurrences.

The engineering component of the models, he said, are also usedto determine how modern buildings would respond to an earthquake.“A lot has changed since 1811,” he noted.

Speaking to hurricanes, Mr. Muir-Wood of RMS said modelers havea good handle on Florida, where there have been many storms fromwhich to draw data. He noted that when Hurricane Wilma struck in2004 and threatened Miami, modelers were able to gather data on howa major urban center with high-rise, multistory buildings respondsto a windstorm.

As for the hurricane risk in other areas, Mr. Muir-Wood saidbuilding stock north of Virginia remains largely untested.

While Ms. Clark said modelers still struggle with gathering alot of wind-speed data from hurricanes, Mr. Muir-Wood said there isa good understanding of what wind speed looks like. He also saidmodelers have a good handle on storm surge and river floodingconsequences. “It's building stock in areas that may be unknown,”he warned.

The question remains whether insurers fully understand theseuncertainties when they apply model results to theirunderwriting.

Mr. Muir-Wood, and other experts, told NU that insurersare making a lot of progress in developing a better understandingof the models and how to interpret the results.

He added that RMS spends a lot of time teaching people “what wethink are best practices.” RMS offers a certification program, henoted, that allows customers to train on the skills necessary todemonstrate a reasonable understanding of the science andtechnology used in the models.

Jayant Khadilkar, a partner at TigerRisk Partners–a privatelyheld reinsurance broker and risk/capital management adviser–said heworks with companies so they can make sound decisions on how toapply the information obtained from cat models. (In July 2009,TigerRisk entered into a partnership with Karen Clark & Companyto help insurance companies manage catastrophe risks.)

Mr. Khadilkar said there has been progress within the industrywith respect to putting the model results into proper context.“Things are moving in the right direction,” he asserted.

A key change is that understanding catastrophe risk is not areinsurance-biased decision anymore, according to Mr. Khadilkar,who said CEOs and CFOs are now taking a direct interest indetermining their catastrophe risks. “The companies we're workingwith,” he said, “we're working with the CEOs and CFOs and helpingthem understand.”

He added that insurance companies are “trying to deploy theircapital in the best way possible,” and catastrophe risk “is onepiece of the puzzle they have to understand.”

Ms. Clark, too, conceded that “insurers are gaining more insightinto the uncertainty underlying the models.” She said her companyhas been working with insurers to help them understand why there isuncertainty, and how there is little modelers can do to limit thatfactor.

“Once insurers have that insight, they become more sophisticatedmodel users who can put [the model output] in better perspective,”she said.

However, it is still a challenge for insurers to get away frommaking decisions on narrow point estimates, such as “1-in-100″ and“1-in-250″ year losses, she added, explaining that one reason forthis is that rating agencies require such information fromcarriers, and ask for these point estimates directly from the model“and even suggest that one model should be used.”

Insurers, then, feel compelled to use those point estimatessince that is what the rating agencies require. “So that'ssomething we need to change,” Ms. Clark said.

“There is a pretty good understanding that [the models] are justtools and there is uncertainty,” according to Ms. Clark, but shesaid putting that knowledge into practice–using the modelinformation properly and not just going by point estimates–is wheremore work needs to be done.

“[Insurance] companies have become more knowledgeable, but whatdo they do at the end of the day?” Ms. Clark said.

Mr. Muir-Wood and Mr. Guin said their models are only gettingbetter as new data and technology are employed.

Mr. Muir-Wood said hurricane models are now in their fourthgeneration since Hurricane Andrew in 1992, adding that modelershave learned a lot from recent hurricane activity. In 2004 and2005, for example, he said modelers learned the effects of“clustering,” where one hurricane follows the track of another.

In 2008, he continued, Hurricane Ike tested Houston and modelerswere able to determine that building stock there was weaker thanpreviously anticipated.

There has also been a lot of investment in earthquakeengineering, according to Mr. Muir-Wood, including buildingsimulation models and constructing realistic buildings atuniversities, and using shake tables and computer simulations tosee the impact. “That's the only substitute for actual experienceon the ground,” he said.

Mr. Guin said AIR models performed “extremely well” in 2004,2005 and 2008, but those years also served to highlight theimportance of data quality provided by insurers. He explained thatthe data used to assess risk is “equally important as the modelitself.”

Mr. Muir-Wood agreed, noting that recent hurricane yearsrevealed some differences between information collected on insuredrisks and what those risks actually were. He cited an example ofstructures in Mississippi during Hurricane Katrina that were,according to data provided, concrete hotel structures but werereally floating casinos.

Lessons were learned by both insurers and modelers on collectingaccurate data for risks. Mr. Muir-Wood said modelers have createdtools and capabilities to measure exposure information.

“Insurers can test what they think [a risk] is versus what weknow it is,” he added.