After sitting in on usability testing conducted by the IT team on website design, JoAnna Carey, marketing development manager for Foremost Insurance Group, came away wanting more information than the traditional testing sessions could provide. To determine whether its independent agency force was using the carrier's portal efficiently, Foremost would invited agents to look at sketches of what the site might look like and asked for feedback on the user experience.

"I kept hearing questions about how to quantify this information, but that was not something we were able to do," she says. "I began looking for solutions with electronic testing capabilities. I had a specific idea in mind to show an image of the website and provide a picture for agents that would ask them where they would click in a particular scenario."

Carey knew about heat map technology where testers would show an image to the audience, the audience would then click on the image, and the testers could determine what caught the user's attention.

"That would give us a general idea, but we still wouldn't be able to quantify the responses," says Carey. "We were doing one-on-one interviews with agents, but there was always the question of how many agents made that response. Was it that person's opinion or anecdotal information?"

What Carey wanted was to capture a larger group and quantify the information.

"If we survey a thousand agents, how many would click on the exact spot we wanted them to click?" she asks.

Carey came across the software solution provider Qualtrics and its Research Suite solution, which would allow Foremost to post an image for the audience to view.

"We could show agents a static image—one frame of this new website—and ask them a question such as: If you wanted to make a payment, where would you click? We would see where the agent would click and out of a thousand agents we could learn how many would click exactly where we needed them to click," she says. "We could see where they were clicking that would take them to the wrong place and with a visual picture we could show the web design team to find a way to address any problems."

Foremost Insurance is a business unit of Farmers Insurance, but unlike Farmers, Foremost works with independent agents. Farmers reported approximately $19.1 billion in gross written premium in 2012 with Foremost accounting for about 10 percent—or $1.9 billion in GWP—of the enterprise. Foremost offers specialized programs for auto, manufactured home, specialty homeowners, landlord or rental property, seasonal or vacant property, motor home, travel trailer, motorcycle, off-road vehicle, snowmobile, boat, personal watercraft, collectible auto and flood. The current number of independent agent active locations for Foremost is 85,695.

Usability testing is both an art and a science, according to Carey, and Foremost uses a blended approach, but the Qualtrics solution gave the insurer a way to examine at the user interface (UI) and see where the agents were falling.

"This allowed us to test a specific page that we were using and it was just one specific snapshot to see if we were moving in the right direction," she says. "We use the comments we receive from our agents as well when they call the help desk about the website or other issues."

The survey, conducted in 2012, showed high accuracy in the scenario Foremost presented, according to Carey.

"The UI offered a different look and feel than they were used to," she says. 'We wanted to know if they could find the things we were trying to make very obvious. In most cases they were. They were clicking in areas that would take them in the right direction so they could find what they were looking for."

Carey did not receive much feedback from the agents that took part in the testing, but she reported receiving one email from an agent who expressed thanks for being included in the test. That agent had done usability testing before, but not in an electronic manner.

"I think it was a positive experience for the agents because they didn't have to sit with us for an hour and talk through the information," says Carey. "They were able to see it and provide us some direction in just the 10 minutes that it took to complete the survey."

As for the future, Carey would like to build in follow-up questions—which the software allows—so if an agent clicks an area during subsequent testing a follow-up open-end question will allow the design team to determine what the users are expecting.

That would allow us to get some of the qualitative information without taking up too much of the agent's time," says Carey. "There will always be the need for qualitative research because we have to hear the agents in their own words, but this gives us a more specific number, so we can use that to guide us in our future discussions to pinpoint the areas where we need to have more in-depth interviews with our agents."

What pleased Carey was there was no large learning curve with the Qualtrics software.

"I watched a one-hour demo and would have been able to get off the demo and program it," she says. "There was little to no training needed. Someone on our team designed the survey and we looked at the back end of the reports to determine what more we needed. It was point and click. Any questions we had, the support team was there."

As for the drop rate on surveys, Foremost tracked how many clicked in to try and take the survey and how many dropped out and if they dropped out where were they in the process.

"We found that among those that clicked in there was probably a curiosity factor so if they dropped out it was at the first screen," says Carey. "These are independent agents that work with our website and our competitor's websites, so they are comparing us to the other sites they are on. That's why it was nice to hear from them to learn how we compare."

Foremost wanted to keep the survey short to show the insurer was respectful of the agents' time. The test site would update every time an agent responded and the heat map was building before their eyes.

"As fast as we gathered enough information, we were able to share it with our design team along with the percentages they were requesting," says Carey.

In the short term, Carey believes this was a big win for the research group because it created a way to solve the business need.

"There is something exciting when the technology catches up to the business needs and when you can provide something that is exactly what was requested," she says. "Plus, it gave immediate information to the design team to help them think through some of the questions they had."

In the long term, Carey believes there are many other areas Foremost can use the technology.

"We are thinking of ways to use this tool creatively in other business situations," she says. "It's opened our perspective to know that the solution may not present itself in exactly the way you think it will. We have to be open-minded enough in our thinking to find long-term solutions when we are using tools like this."

Special Reports /

Pop Quiz for Agents

Related Stories

Recommended For You