Reminiscent of the scene in Alfred Hitchcock's “The Birds,” whenthe sky turns black with attacking avians, I often feel as if I'mbeing attacked by throngs of bad statistics when I read today'sinsurance and business press. “The Birds” was just a movie, but badstatistics can draw real blood. Bad statistics can lead to badstrategies and bad decisions. For proof, look no further than oureconomic malaise. While some people rely on statistics, othersavoid statistics at all costs. Some people believe BenjaminDisraeli's famous saying, “There are three kinds of lies: lies,damned lies and statistics.” I would qualify to say that badstatistics are damned lies, while good statistics are key tointelligent management and a huge competitive advantage. This makesit essential to discern between the two. Sometimes advancededucation is required to identify the good from the bad, butusually common sense will suffice. Here are some examples:Check the data source
I recently examined aPowerPoint from an agency consultant. I was jealous of hisbeautiful graphs that were so artistically crafted. After lookingmore closely, though, I realized they were just art. There was noreal data behind them. The consultant had started with how hewanted the charts to look, then created the data that wouldgenerate his desired charts. He had not gathered any real data.The lesson: Always check the data source. The bestdata is random. If it's not random, suspect it. Be particularlycautious if the organization that created the study also is sellingsomething because its study is more likely to be biased.

Beware of averages
Studies presentingonly averages are of little use because the information regardingwhat creates better or worse performance is absent, as is variance.Averages are not usually adjusted to the median, so a few scorescould greatly skew the result. And even if you see averages showingthe numbers for the top performers or worst performers, that dataoften is meaningless because the numbers are just averages. Manyfactors affect averages–without more information, how do you knowwhat to do about it?

For example, an agency was comparing its producers with a studyshowing the average producer's book was $300,000. Was its $250,000producer a poor performer? Possibly, except she made $250,000 inhalf of the time. How about another producer with $350,000? Is hegreat? Perhaps, except his book was given to him. Possibly thebiggest problem with averages is the general assumption that everystatistic can be plotted along a normal curve. Some sophisticatedusers exacerbate this mistake by assuming normal levels ofconfidence and variation at either tail. The truth is not allcurves are normal and we know that in reality, especially infinancial markets, even when using a normal curve, the extremetails do not always work the way a normal curve predicts.The lesson: When a study only shows averages, verylittle weight should be placed on the results if one is trying todetermine cause and effect. Context iscritical
Changing the context enables people topurposely mislead others. For example, if I wanted to pump up anagency owner, I could expound on how his agency's 88 percentretention rate was great, without mentioning that most of hiscompetitors are doing even better. At 88 percent retention, thereality is the agency is likely doing something wrong. Anothergreat example is, “Our producers are awesome! They each wrote morethan $200,000 in new commissions last year!” Are the producers newor established? Is this a small or large agency? Does this includeprogram business? And most importantly, what difference does itmake how much new business the agency writes if all of it goes outthe back door? I've met awesome new business producers who couldwrite the equivalent of 30 to 40 percent of their books in newbusiness each and every year. Of course, their retention rates werearound 65 percent. New business only counts if it is retained.These are important issues that greatly affect theterrific-sounding “$200,000 in new business” statistic. Thelesson: Check the context of the statistic. Do factorsexist that might mitigate the statistic's usefulness?Mixing & matching
Be cautious of studiesthat compare apples to oranges. A common and very misleadingmismatch is to use EBITDA (earnings before interest, taxes,depreciation and amortization) to compare companies growingorganically to those growing by acquisition. The problem is whileEBITDA excludes almost the entire cost of acquisition growth, itdoes not exclude the cost of organic growth. I do not have space togo into all the details here, but suffice it to say that whencomparing the two types of growth using EBITDA as a measure, theacquiring firm's EBITDA is virtually guaranteed to look hugelybetter. The key is to look at cash flow. When firms groworganically, their cash flows and profitabilities usually are veryclose to the same. When a firm grows by acquisition, cash flowoften is much less than profits. But because too many people do notunderstand the implications of EBITDA versus cash flow, a lot ofbad acquisitions, growth decisions and even loans have beenmade.

The lesson: The statistic or measure must matchthe purpose. Do not compare apples to oranges!

Assuming cause and effect
A statisticalcorrelation does not guarantee a cause-and-effect relationship. Forexample, a recent study found the shorter life span of Russianscorrelated with industry privatization, which was killing Russiansat an earlier age. Further research revealed that whileprivatization was highly correlated with shorter life spans, theactual cause is vodka. With privatization, Russian workers candrink as much as they want and are drinking themselves todeath.

An example closer to home is revenue per person and profits.Many in the industry assume greater revenue per person leads togreater profits. The truth is, it doesn't. In fact, there is zerocorrelation between revenue per person and profits. Take a look atbest practices and you will find that the agency sizes with thehighest revenue per person have the worst profitability.

The lesson: Don't assume cause-and-effectrelationships. Are there other factors that may be causing theresult? Incorrect assumptions
The bestexample of this is the assumption that insurers pay more for largerbooks. It's true that bigger books of business generally result inlarger contingency checks. However, do companies really pay morefor larger books? In dollar terms, larger books generally result inbigger bonus checks. But on a percentage basis, the effect of alarger book is very small (except for books less than approximately$1.2 million). Based on my analysis of dozens of contingencycontracts, once a book gets past about $1.2 million in premium, thebonus percentage does not increase much in most contracts forbigger volumes. For example, at $3 million, the bonus percentagemight be 2 percent, and at $8 million it might be 2.1 percent. Thebonus is much bigger: $168,000 versus $60,000. All but 0.1 percentof this is due to the bigger book–it has nothing to do with thecompany paying more money. The percentage is the key here, not thedollar amount. An agency often is advantageous not building abigger book with a specific company just to earn morecontingencies, because it will not, unless the company is one ofthe few that pay a materially higher percentage for bigger books.The lesson: Always check the underlyingassumptions. Are you or is the study making any assumption thatwould cloud the true results? Statistics easily are manipulated tomislead. When statistics are done well and used correctly, they canprovide huge competitive advantages and opportunities. I am notrecommending a formal education in statistics. I am recommendingyou think about the statistics you see, apply common sense, andwork with people who will present an accurate and unbiasedperspective.