Filed Under:Carrier Innovations, Information Security

Big Data is Blowing Up

Many insurers have chosen to take a wait and see attitude on the subject of unstructured data, which they somewhat nervously refer to as “big data.” But if insurers are to use this data effectively, particularly with business intelligence (BI) tools, they need to begin making plans.

“That’s the one coming on like gang busters,” says Karen Pauli, research director in the insurance practice at TowerGroup.

Unstructured data is coming in at the policy and claims levels for some carriers in the form of telematics and the convergence of social media.

Pauli doesn’t believe big data should be a tremendous worry for carriers, but she issues the caveat that insurers have to make plans on what to do with it.

“It is going to come on fast and furious, so if you are not prepared from a technology perspective you are going to get your butt kicked,” she says. “You have to be in the planning stages right now. If you’re not into a solution by the end of 2013, you’re going to have too much work to do.”

Frank Petersmark, CIO Advocate for X by 2, points out that big technology companies such as Google and Microsoft are still trying to figure out the unstructured data issue, so insurers shouldn’t feel alone.

“I’m not aware of any good solutions for the problem right now, but I think something will come,” says Petersmark. “But frankly insurance companies are having enough problems with structured data. It’s a creek that’s going to have to be crossed at some point, but for the next few years I believe the focus will stay on the structured data. In the meantime, there will be vendors working on this to come up with a solution for unstructured data.”

Only the very large carriers are utilizing big data sources—such as telematics or clickstreams—to incorporate or analyze that data, according to Martina Conlon a principal in the insurance practice with the research firm Novarica.

“I have not had a conversation with one midsize or smaller carrier that is incorporating that large unstructured data into their analysis or their everyday operations,” she says. She points out that as telematics become more popular, carriers of all sizes are going to need to support and analyze that type of data.

For now, though, Conlon doesn’t believe big data is a business priority in terms of the value carriers will receive from analyzing data that is available in social media.

“The benefit hasn’t become significant yet and the cost hasn’t become low enough yet,” she says. “I think a lot of midsize carriers have priorities elsewhere such as straightening up their back-end systems, cleaning up their core, thinking about mobile, increasing customer service and taking advantage of new technology. [Big data] isn’t the priority yet. There are tools and technologies that support the analysis of big data, it’s just a matter of making the business case.”

Pauli points out that two companies that offer BI and analytics tools, SAS Institute and IBM, have launched real time visual analytics, so the solutions to this headache are on their way.

“It is amazing because of the technology behind it takes the analytics world—which now sits with some sophisticated math people and those who can deal with analytics technology—that you can drive the analysis down to the business side,” says Pauli. “A business analyst sitting in the underwriting unit can actually deal with the data.”

Big data is more than just large volumes of data, explains Ben Moreland, senior analyst at the research and advisory firm Celent.

“When you want to find patterns and insights from different dimensions you are going to have to go to a vendor that can scale up to the dimensions you need,” he says. “We’re finding a lot of hype with big data and believe it will start seeing a lot of success. But we believe there is a lack of understanding of what they are going to gain from it. The skill sets also are different. Carriers need to understand what it means to leverage this data.”


While carriers are focused on their more traditional data today, they continue to seek better ways of extracting actionable information to make business decisions that are supported by the data.

To do this, carriers need tools that help them pull the most important information from the data to improve decision-making. Some insurers are confused by the use of business intelligence and predictive analytics to define somewhat similar tasks.

Conlon defines business intelligence as a broad capability and platform where insurers have consolidated, integrated, enterprise-wide data available in a tool so carriers can genuinely analyze their data to support business decisions.

“The types of things you do in a BI environment tend to be operational in nature,” she says. “You could have dashboards running off the BI environment that are based on workflow, statistics, and processing claims or new business, which is clearly not predictive analytics type of work.”

Moreland says he can’t get vendors to agree on the difference between business intelligence and predictive analytics, but in his mind BI is the presentation of business insight to decision makers—when, where, and how they need it.

“I think business intelligence is a little more about looking at what is current and analyzing the here and now,” says Pauli. “Predictive analytics is more what we should do in the future and modeling.”

Insurers also can perform decision support data mining on their book of business to understand the characteristics of areas such as who are the carrier’s profitable agents or analyzing the book of business to determine possible risks or upsides.

“Within your BI environment you are doing predictive analytics in order to come up with predictive models,” says Conlon. “You are doing the analysis and the statistical modeling in order to develop predictive models that will score underwriting risks, fraud scoring for claims or severity scoring for claims. Predictive analytics really is a subset of the types of analyses and reporting you will do in a business intelligence environment.”

Some analysts and business users believe predictive analytics is a part of business intelligence, but Pauli feels business intelligence has a stronger relationship to business process management (BPM), whereas predictive analytics is related more to modeling.

“In terms of how our customers use those technology features, that’s where we’re seeing the most traction,” says Pauli. “It makes sense. BI seems to generate BPM and BPM needs BI, so they come together. It also has something to do with the BI vendors and the BPM vendors, who have morphed into each other’s space.

Petersmark believes it is important for a CIO to think about business intelligence as the end game, not a product.

“It’s the sum of all those other parts,” he says. “What you want to provide to your organization is the aggregate of all the work IT and the business areas are doing around data, information, analytics, and modeling and put that into something that resembles business intelligence.”

Petersmark feels business intelligence gives the business leadership the ability to take action.

“It has to be decision-making, actionable information that is informative to executives,” he says. “It’s not worth the effort and investment if you can’t get it to where the carrier can earn some first-mover advantage in a particular market, geography or product, or get some sort of expense advantage or fraud advantage by identifying trends early on.”


Midsize carriers are just now operationalizing policy systems for predictive analytics and predictive modeling, points out Conlon.

Unfortunately, Petersmark doesn’t believe insurance carriers are following that advice.

“Part of the problem is where do you start with this,” he says.

He believes business leaders think if you apply some analytics or build a data warehouse it will evolve into something special. The way many carriers started with business intelligence was to get the information organized and then figure out what to do with it.

Today, business people are more savvy around the notion of the big picture, explains Petersmark.

“They want to be able to look at the three top level strategic objectives,” he says. “Then they want to know what you have to do to accomplish it.”

For too many, though, data information is a mess and there’s no single source of the truth, according to Petersmark. This means they need to build a data mart and apply some advanced analytics and also make the CRM systems more sophisticated.

“The trend now is moving toward the bigger picture,” he says. “For a while it was just an IT thing. IT owned the data and knew how to massage it. Today there is a trend toward joint ownership. IT works in a collaborative role with the business to determine the end game.”


Pauli believes the biggest driver in business intelligence is the distribution system.

“If you are direct to consumer, it’s a lot more around managing your customer and your channel. If you are direct to consumer that information is absolutely everything,” she says. “If you are an agent/broker carrier you are probably more focused on things like pricing and operational issues because service is really where it’s at if you are using the agent/broker channel.”

Pauli also sees more BI being done in the personal lines area and with the small commercial lines because insurers have more data in those lines of business.

“Once you have data, you are anxious to synch up trends around pricing and distribution,” she says.

Field force efficiency, claims, and customer retention are three areas Celent has identified as being areas where business intelligence is used effectively, according to Moreland, but he doesn’t believe a good BI strategy should be limited to those areas.

“Carriers that have a good understanding of their data know how to use it,” he says.

Field force efficiency allows carriers to not only determine who their top performers are, it develops best practices on what makes them the best performers so they can share that information across the entire field staff, whether it’s the process they use or the technology they have available, explains Moreland.

“It raises the bar for the entire sales staff,” he says.

As for claims, typically about 60 percent of the total IT budget is spent in this area, according to Moreland, and much of that money goes toward fraud protection.

“There’s a lot of use of analytics and business intelligence to identify higher risks before an event as opposed to after the event,” says Moreland. “If you have to try and recover your money [after the event] there is always an expense with that. Getting that information beforehand will save carriers money.”

Finding the best use of business intelligence depends on the carrier’s business model and their lines of business to determine the biggest impact, explains Conlon.

“Obviously a direct carrier isn’t going to get much benefit out of analyzing the distribution channel versus a carrier that works with independent agents and potentially other channels,” she says.

Generally there is a lot of benefit from BI in the claims area, adds Conlon. Carriers recognize its value in terms of identifying fraud, looking for opportunities for subrogation, and predicting litigation outcome.

“Claims is an area where both predictive analytics and business intelligence have been heavily used and continue to bring benefits,” says Conlon. “It’s being used more in underwriting to develop automated underwriting rules or manually applied to increase the quality of the book of business.”


Conlon feels large and midsize insurers have made huge progress in getting their data out of silos. A vast majority of carriers have integrated data warehouses in place that they can use to access and analyze enterprise data.

“Smaller organizations often don’t have something like this in place,” says Conlon. “But it needs to be a priority for smaller p&c carriers to put that in place this year.”

Petersmark doesn’t believe the issue of siloed data has been settled among insurers.

“There is recognition that the data needs to be someplace other than where it is now,” he says. “It has to be some place that is much more accessible to the people that are going to use it. The business users should have their own tools and work with the data directly. There are some good products out there, but the problem is, in my mind, the data is not accessible enough.”

Petersmark feels quite a few carriers are scrambling to get their data better organized in a couple of repositories and out of any proprietary formats. They also are trying to pull it out of legacy systems to match it to what business people need.

“Overall the trend is moving in that direction, but we’re far from accomplishing it,” he says. “Those carriers that have that behind them should get some competitive advantage.”

The state of data within a carrier’s systems is improving because insurers are more focused on core systems. But even with the improvement, though, there remains a long way to go, according to Pauli.

“You’ve got different strategies for dealing with data,” she says. “There are a lot of jumbo carriers that are not doing core system replacement; they are doing wrapping, so they can get their data in order that way.”

Pauli also points out there are many vendors now that will come in and make sense out of a carrier’s data.

“If you are going through a transformation project or core system replacement, you might as well have someone clean up your data for you,” she says.

Conlon believes the quality of data for most carriers is higher than some would generally expect. One reason for this is much of the data that is utilized in transactions is exposed to the policyholder and agents, so it is external-facing.

Carriers no longer get all their data on new business from prospects; it is coming from third-party providers and the level of consistency and quality is much better.

“Overall, we don’t have a huge quality issue in the industry, but you always have baggage with your older systems,” says Conlon. “When you are looking at building in a business intelligence environment, you have to look at each system and determine a business case for incorporating the data into the right environment. If the older systems don’t make the business case, you’ll find that carriers won’t incorporate that data into the business intelligence environment. It’s not just a quality issue; it may be an issue of cost and the lack of development of getting the data out.”


Petersmark believes there are carriers in the middle of that thought process right now.

“It depends on your book of business, your claims history, and a regulatory/legal perspective as well,” he says. “With workers’ comp in particular, since they are mostly one-year policies subject to renewal, we often made a decision [while at Amerisure] that converting the data would be a nightmare so we would go new and renewal, build the information as we went, and put that into a BI platform or analytics.”

Many carriers are creating separate, historical data stores where they can put the old data and IT will build an interface to it when needed.

“If history needs to be inquired upon, [users] go to this repository,” Petersmark says. “It still involves some scrubbing, but it’s mostly static information.”

There has been a lot of progress in the battle for data quality, but Moreland believes it is still the number one issue for insurers.

“Garbage in and garbage out still applies,” he says.

Insurers invest time and money to clean data and to make sure the new data coming in is clean to begin with.

“In the past we saw tens of millions of dollars spent on master data management where they were trying to build one data warehouse or data mart,” says Moreland. “Where once they had all the information, now they could have all the business insight they needed. But these failed because of the sheer weight of what they were trying to do.”

Today, companies are working to leverage the data for known business needs and they can expand with additional data for more business needs, points out Moreland.

Carriers can’t abandon old data, particularly if it relates to a customer’s policy, but Pauli reports she has seen some carriers elect not to clean old data.

“They renew a policy into it if they can, but you can’t just abandon your history if it relates to your rate,” she says.

Featured Video

Most Recent Videos

Video Library ››

Top Story

5 ways to defend your family business against fraud

Companies with less than 100 employees are more likely to experience scams.

Top Story

Do you qualify for NU’s Excellence in Cyber Security Risk Management Award?

Gain your cybersecurity risk management program the recognition it deserves. Nominate your program for the NU Excellence in Cyber Security Risk Management Award today!

More Resources


eNewsletter Sign Up

Carrier Innovations eNewsletter

Critical news on the latest tech solutions, information security, analytics and data tools and regulatory changes to help decision-makers at insurance carriers keep their business thriving – FREE. Sign Up Now!

Mobile Phone

Advertisement. Closing in 15 seconds.