A decade after its introduction, agile development hasestablished a strong following. Forrester, noting that agile has“rapidly joined the mainstream of development approaches,” found itto be a primary method in use at 35 percent of organizations in a2010 study. Novarica, also in a 2010 report, reported that the vastmajority of insurers were using agile in at least some of theirdevelopment, primarily around new applications.

Yet the question on the minds of many insurers is whether—orwhen—agile is right for them.

Agile's purported advantage lies in the literal meaning of itsname. Agility is derived through an iterative development process,a continuous feedback loop that connects business and IT at everystage, and incremental deliverables.

“If I'm a customer and I want certain features to be done on myproduct on day one, and then on day three I read a blog postingthat makes me change my mind by day four, that's what softwaredevelopment teams need to be able to deal with. Agile allows you todo that by providing the framework to help software developersrespond to that rapid pace of change,” says Michele Sliger, ownerof Sliger Consulting Inc.

“Agile allows business users opportunities to interact with theevolving system,” says George Grieve, CEO of CastleBay Consulting.“By breaking down the scope of a project into 30-day deliverables,users get early and frequent opportunities to verify that what theyare getting is what they asked for.”

Grieve claims he has never been involved in an agile projectthat has failed. “I've seen projects where the budget and timelinecertainly came under stress, primarily because the company didn'thave strong enough change-control to separate simple changes fromtruly new requirements. However, every agile project has deliveredsuccessful results that have been well received by business,” hesays.

BECOMING AGILE

Agile proponents contrast the approach with “defined”development methods, the best-known of which is waterfall.Waterfall, where one stage of development has to be completedbefore moving on to the next, offers a sequential and structuredapproach that has been used successfully on countless IT projectsover the past 40-plus years; however, proponents of agile point toits waterfall's limitations.

“In waterfall, the focus tends to shift away from gettingworking software to the company to making sure everything iscompleted on a checklist before moving forward to the next stage,”Sliger says.

“As a result, you can spend months in the design phase beforeanyone even starts coding,” she explains. “Ultimately, because ofall the bureaucracy around waterfall, the development team is oftengiven a document that tells them what to do but that they've had noinput into. As a result, they have no understanding of whatcustomer needs are. Many no longer even see or talk to a customerdirectly.”

The sequential nature of defined development assumes that thebusiness side can completely articulate its needs, which may not bethe case.

“In the past we would spend months if not a year building specs,then deliver something that didn't actually meet users' needs,”says Gary Smith, manager of application development for MichiganMillers.

Tom Mellor, who began his career at State Farm in the claimsdepartment in 1992 before migrating to the development world in theearly 2000s, saw this limitation first-hand.

“It was often futile for the business to try to develop completerequirements upfront, and as a result there were many problems thatresulted from people not getting what they needed or wanted. I knewthere had to be a better way than to think we could look into acrystal ball in new product development,” he says.

Waterfall development also assumes thatbusiness needs, even if well articulated, will not change. “When Istarted working with project development, I was amazed with howunpredictable the work was, yet we were expected to bepredictable,” says Mellor.

“From the business point of view, change is happening all thetime that impacts the scope of the technology they need,” saysGrieve.

Among the different flavors of agile, the most popular is Scrum.Mellor, who was instrumental in introducing agile to State Farm in2004, is a certified Scrum trainer and agile coach at the insurer,serving as project manager in business application developmentareas.

Seven years after its introduction, agile has had a markedimpact on State Farm's development environment, including in thelayout of its headquarters in Bloomington, Ill. In an initiativedubbed Systems@Work, three floors utilized by its development teamswere converted from cubes and walled offices into open spaces andoffice “hotels” where no employee—even a manager—has a designatedworkspace.

“My office is a backpack and I have a locker I put stuff in. Igo where I'm needed. There is open seating and honeycomb workareas. It works well and is very collaborative,” Mellor says.

AGILE SUCCESSES

State Farm's venture into the world of agile began, in the wordsof Mellor, “surreptitiously.” A business unit saw an opportunity toget a product to market to take advantage of new health savingsaccount legislation, but needed a system up and running in 60 daysin order to do so.

“I had explored a bit about Scrum and believed it would work. Ipromised the business and technical sponsors we could get theproject done, but it would have to be run under the radar,” Mellorsays.

The project, requiring about 2,000 development hours, came in ontime and on budget. Based on that result, State Farm used Scrumsuccessfully in a 25,000-hour project later that year.

“After we had completed the core work on that [second] project,we still had money and people left. We asked [the business] if theyneeded anything else delivered and they were flabbergasted. Afterthat, word got out. There was a lot of buzz going on,” Mellor says.That led to additional Scrum projects, the certification of Scrummasters, and experimental projects with other flavors of agile,such as Extreme Programming (XP).

State Farm's start-small approach is one that Grieve recommends.“From a practical point of view, the first couple of projects acarrier does should be small to control the sheer scope and riskand to allow time to learn the methodology as they go along. Themethodology isn't flawless, and there are challenges that arisebecause of the continuous feedback loop of agile,” he says.

“The difference of addressing those challenges with Scrumcompared to traditional waterfall development is that everyone isaware of problems, such as when projects are running late. We alsoused Scrum philosophy and framework to create solutions to timeoverruns,” Mellor says.

“What caught the eyes of a lot of people was that we weredelivering along the way, we recognized early on when problems werearising, and we could respond with simple, interim solutions,” hesays. “That wouldn't have happened naturally in waterfall, becausein waterfall there are no provisions for that kind ofadaptation.”

Yet even with a high level of project successand the environmental changes that occurred at State Farm throughthe Systems@Work initiative, the insurer's development teams stilluse waterfall for the majority of projects.

Out of over 700 projects in progress in 2010, fewer than 100involved agile methodology. “We use agile primarily for new productdevelopment and applications where there is a lot of uncertaintyaround requirements,” Mellor says.

State Farm encourages project teams to select a developmentapproach based on three factors: the relative certainty ofrequirements, the types of technology platforms involved, and thepreferences of the customer. Projects with a high degree ofcertainty around requirements or that involve legacy or exclusivelyinfrastructure development are more likely to be addressed withdefined development approaches such as waterfall.

The third factor is most important. “If the customer wants toinspect a product as it is being developed, including delivering itincrementally, then Scrum is a great fit. But if they don't want itdelivered that way, or don't want to be involved in the developmentprocess, its usefulness becomes compromised and we likely won't useit,” Mellor says.

Whereas State Farm started small, Michigan Millers took on thedevelopment of a new policy administration system, dubbed Matrix,for its first venture into agile development. Michigan Millers wascommitted to developing the system, incorporating a multi-variantrating engine, in house rather than deploying an off-the-shelfplatform.

“We looked at buying products, then we looked at outsourcing thedevelopment. Ultimately, we believed we could provide a betterproduct that we could obtain elsewhere by doing it ourselves,” saysJim Wieber, Scrum master for the Matrix project.

“We also showed that time frame and cost were in favor ofin-house development,” says Smith. “We calculated we could get thedevelopment completed in half the time it would take if we wentoutside.”

Michigan Millers believed that agile development was essentialto meet that goal. “Our old methodologies weren't going to meet ourtimeframe, particularly because we had always had trouble nailingdown business requirements. In the past, we'd get bogged down inrequirements, and sometimes we didn't deliver because we'd work toohard on minute details that weren't important. We needed a betterway,” says Wieber.

The move to agile was driven by the development team. “Theypresented a proof of concept to our senior management, explainingthe concepts and how we wanted to run it, and the need to partnerwith the business unit to deliver,” Smith says.

Although Grieve wouldn't recommend starting the journey to agilewith such a large project, he says that core administration systemsare a perfect fit for the development methodology. “In anyenvironment there are variables you can't control—regulatorydevelopments, changes in business opportunities, changes in themarket. During a lengthy core system project, many things happen inthe outside world which impact the project, and which the projectdoesn't have control over. Agile responds best to that,” hesays.

Michigan Millers created development teams and broke downphysical cube walls. They also leveraged the expertise of vendorsfor other platforms that were using agile in their own development,such as in StoneRiver's billing system. “We noticed that afterStoneRiver began utilizing agile development for code releases, wewere able to accept the releases more quickly,” Smith says.

Most important, the project team got the necessary resourcesfrom the business side. “Our personal lines unit gave up one oftheir best people to be the product owner. That was a hugecommitment, but it was essential,” Smith says.

“There was a steep learning curve for both sides at thebeginning, but the project team immediately saw the benefit. Thebusiness side saw that in every sprint we were developing what wesaid and that it met their needs,” says Wieber. Those sprintsstarted at four weeks, and evolved to two as the launch dateneared.

Smith explains that Scrum put projectprioritization in the hands of business, where it belonged.

“The business knew exactly what our velocity was—how much workwe could do. So if they knew we had the resources to do 100 storypoints but they wanted 150, they could either reduce the number ofpoints or change the delivery date,” he says.

The policy administration system was rolled out to the personalauto line in March, 2010. “We hit the target for time, budget,feature set, and adoption rate with our independent agents,” Smithsays. “It far exceeded everyone's expectations for a best-casescenario.”

AGILE CHALLENGES

Despite the successes enjoyed by agile proponents, there arechallenges to its adoption. “It's not problem-free,” Grievecautions. “Not only does agile break down the rigid developmentstructure in terms of sequencing of tasks, but it breaks down thetypical project management hierarchy. That's the biggest changecompanies have to be prepared to deal with.”

Sliger points out, “Often times the adoption of agile triggers acultural change. Some companies have a 'values mismatch' and willfail at utilizing it, while others actually embrace it thechange.”

The most important point is the original team you put together,explains Smith. “They are going to lay the foundation foreverything else going forward,” he says. “You need to be sure theyhave the personal commitment and buy-in to their Scrum team. Thathas been hard to do—they have to realize that they succeed as ateam and not as an individual. That's not something you can teachpeople, so you have to choose wisely.”

Agile requires a “flat” structure where project teams operatewith a higher degree of autonomy than in directive-driven waterfalldevelopment.

“Ideally with agile development, you want autonomous teams thatcan make decisions about the best course for making the product.But sometimes, people are fearful of pushing that authority down,”Mellor explains.

He stresses that top-level support is needed for the culturalchange—an observation that is unexpected given the under-the-radarmanner in which agile took hold at State Farm.

“You can start [agile] at a grass roots level, but eventuallyyou'll run into too many impediments as projects grow in scope,”Mellor says. “You need full commitment and resources.”

“You have to have a champion or multiple champions, and it hasto be more than just a checkbook commitment,” Sliger stresses. “Youhave to be willing to remove the organizational roadblocks the teamuncovers.”

Failing to provide high-level support can lead to “minidictatorships” on one hand, or a directionless, hands-off approachon the other.

Sliger adds that insurers need to understand that agile is morethan a checklist of steps to follow. “It's easy to focus too muchon the set of practices rather than the philosophy and mindset thatare needed with agile,” she says. “I've seen people try to takebits and pieces of the practice like it's a menu. You will get someof the benefits, but you don't understand the principles drivingthe value systems that these practices help to bring about. Withoutunderstanding that, agile is not going to work as well foryou.”

“A company either does or does not have an agile culture,”Mellor says. “It does not 'do' agile.”

While the structure of projects is less rigid and daily stand-upScrums add informality to project management in agile, there is aheighted focus on execution due to the increased number ofiterative deliverables.

“Some of the developers thought Scrum wasessentially 'cowboy coding,' where they documented a little bit anddid what they wanted. But they soon realized that Scrum actuallyintroduced more engineering processes and discipline than we hadbefore,” Smith says. “They also realized that we are now expectedto be partners with the business in an iterative and collaborativeprocess.”

Managing that collaboration can be a challenge for companies andhas given rise to a new breed of project management tools targetedtoward agile.

Forrester points out that agile collaboration eventuallyrequires automation because sharing status is time-consuming,particularly when a team is dispersed. Agile practices aroundtesting, architecture, and build also require automation to be mosteffective.

Michigan Millers uses ScrumWorks Pro by CollabNet for projectworkflow, and the open source CruiseControl framework for acontinuous build process and automated code deployment acrossdifferent environments. “ScrumWorks was installed at the start ofour first agile project,” Smith says. “We use the Web reportfeature to publish reports to a project status page. From there,everyone in the company can see the current sprint burndown chart,product backlog, velocity history, and more.”

CHOOSING TO BE AGILE

Whether agile is the right choice for a company or particularproject should be driven by business needs. “If waterfall isworking for you, don't change it, because it doesn't make anylogical sense to do so,” Sliger says.

Defined development may well be the right choice in projectswhere requirements are stable or where speed-to-market is notcritical. Waterfall's sequential process emphasizes requirementsand design, and brings structure to the project. Stages havedefined beginning and end points, giving developers clearindication of progress—and a comfort level associated withfamiliarity of process. The benefits of agile development are alsomore difficult to derive when the technology platform itself is notagile, such as with inflexible and legacy platforms that lackconfigurable capabilities.

Nevertheless, agile has established its place in the developmentworld. “Agile will have strong staying power for many reasons, themost important of which is its proven successes,” Sliger says.“Even though agile is a modern philosophy, it is really built onideas of iterative development that make sense to both business andIT and are certainly not new in the business world. I fully expectthe movement to agile will continue to gain strength.”

“Your attitude toward agile should be 'Why shouldn't we dothis?' However, there are a lot of insurance companies that ask'Why should we do this?' instead,” Mellor says.

“From a competitive standpoint, there's nothing I'd like to seemore than other insurers sticking with the old ways ofdevelopment,” Mellor adds. “If you're not focused on continuousimprovement and adapting your development organization to acultural model of efficiency and production, it does nothing butbenefit us as your competitor.”