The common method of functional requirements gathering for a "package" implementation is to employ some version of a gap analysis. This makes sense. After all, why would a carrier buy a third party vendor product and then proceed to ignore what that system does and how the system functions and go off and start requirements from scratch? So the gap analysis technique of requirements gathering is a common practice. Unfortunately it is not always a best practice. In fact the gap analysis phase is often the root cause of many problems that arise on third party software implementation projects. These problems include unrealistic time and cost estimates, incomplete functional requirements, and all the resulting project pathologies which occur when reality finally disabuses the disingenuous.
So is there something inherently inadequate about the gap analysis process that leads to these problems? The answer is "no" but on many of the projects I am aware of it is poorly done for at least two reasons:
Hard to Define Requirements
First, there are several issues concerned with requirements gathering that must be recognized. These include the following: No carrier company knows its requirements well enough to be able to describe them completely and in detail when asked. This is hardly surprising given that business people are too busy during the normal cycle of business operations to actually take the time to define current business processes. If you are busy doing what you do, what is the compelling reason for you to make the time to define how you do what you do? Also, merely documenting how work is currently done does not get at the issue of whether current processes are in need of standardization.
The illusory assumption often made when planning a gap analysis is that the carrier has "standard" processes which are common across the enterprise and which do not vary by teams, departments or regions. Often times there are valid business reasons as to why processes vary. For example, handling an auto collision claim in the Upper Peninsula of Michigan will and should be different from handling the "same" claim in downtown Miami--one of the national centers of auto claims fraud--for obvious reasons. Also, given the degree of state regulatory variation many standard processes and procedures for risk selection, pricing, underwriting, claim handling, and policy and billing administration can only apply within a state, or to a few common states. However, it is just as often the case that departments, teams or individuals do their work a certain way for no better reason than that is how it evolved over time without reflection or reference to how others work.
Next there is the issue that at the detailed level core insurance system implementations need to know exactly how current processes, calculations and rules function. This often requires understanding of how the current legacy systems function, which is usually the domain of a very few, suddenly very "in demand" individuals who have an imperfect knowledge of the current systems, but who know more than anyone else.
Finally, core system implementations are usually about changing and improving business processes, so even if a completely documented current environment can be achieved it is only part of the solution. Figuring out in detail how a business runs today is hard enough; figuring out how it should run tomorrow is harder still and takes time and collective, original thought. So for various reasons, gathering gap requirements is a challenging and usually imperfect process.
Gap Between What and What?
The second major issue that must be overcome is that by definition a gap analysis is an attempt to "fill in the blanks" between what the vendor's base system does and what the carrier needs. In other words, in order to complete the requirements gathering exercise successfully the carrier's business users must gain a substantial understanding of how and what the base system does. Otherwise, how can they define the "gap"?
It is a sobering fact that I see few project plans for third party implementations that allow any substantial amount of time for the requirements gathering team to gain a good working knowledge of how the base system actually performs various transactions. This is a particular problem for those vendors who continue to use the waterfall methodology where (supposedly) all the requirements are defined and documented before the system is configured to act to the carrier's specifications. At least in an agile environment the business users will get several opportunities to see the evolving system and react accordingly.
Yet it is on the basis of this flawed process that the "scope" of the project is defined. And it is on the basis of this inadequate scope statement that the remaining project timeline, resource plan, and budget are defined. No surprise then that so many projects overshoot their budget and timeline goals. And of course such under-estimates come with a great deal of handwringing, helicopter management, and collateral damage.
Help, I?m a Project Sponsor
Of course it takes awhile for the realization to sink in that the project has been underestimated and that the base software has been misunderstood. This usually happens as the result of an initial software delivery where the carrier's staff gets its first real look at the system and realizes it is not what they assumed in significant ways. When this realization has finally sunk in the reaction is usually to "go back" and try and create a more complete estimate before sharing the news with the project sponsors.
Given that this exercise can take weeks and diverts resources it usually does not happen "in secret." So now the sponsors and senior management responsible for the project know that bad news is on the way; often their reaction is to demand the bad news "now," but of course the bad news is not yet available; to demand to know how long the team has been keeping this information from them or some other variant on an unhelpful response. The problem from the sponsors' viewpoint is they have no way of evaluating the state of the project and knowing if there is a rational argument for continuing to support the project team, or whether the project is in deep trouble.
This is where the survival equation comes into effect. The sponsors know only two facts: the first is that they were originally told the project would take a certain amount of time and money, and now they are being told something significantly different; and secondly if they cancel the project they will have (mostly) wasted a large amount of money. Between these two facts lies a wide swath of unattractive options to think about. The real question for the sponsorship group is to try and decide what this new and unwelcome information means. Does it merely mean that the team got it wrong once but have it right now and that the project has a strong chance of success, or could it mean that the team is actually incompetent and the project is headed for failure. Where, for example, is the reassurance that the team will not be back in another six months with a new and more extended timeline and budget?
There are few good options for the sponsorship group at this point. Stopping the project for some period of time in the hopes of figuring out what is really going on is problematic. Unless the sponsors are willing to pay the vendors to sit around and twiddle their thumbs, key members of the project, and their hard-won institutional knowledge, will be lost to the project. Bringing in a third party to assess the project and get an "outside" opinion is also hazardous. Many of the companies that offer such services also run projects and may be influenced in their assessment by their own self interest. Even with the best intent an outside reviewer must still at least disrupt, if not temporarily stop, the project by occupying the time of the key players in order to form an independent opinion of what is going on. And the final conundrum for the sponsors is why should they trust the opinion of an outside consultant any more than they trust their own project team, assuming their assessment differs from the in-house team? And what if they agree with the in-house team? Simply accepting the newly inflated estimates without some form of investigation is hardly good business oversight. So what does that leave?
First, one would hope the project team and its key member would gradually have won some amount of credibility with the sponsorship group that would support their case in the event of a "replan" event. Second, if the project team does its work correctly upon discovery of the inadequate gap statement it should be able to create a plausible explanation of how the project got into the situation it is in and in doing so provide some reassurance to the sponsorship team that they are not experiencing the symptoms of a chronically inept project. Absent these data points the sponsors and the project are in deep trouble.
It is usually the case that first replan events, often born of an inadequate gap analysis, result in approval for the project to continue with greater oversight. In the case where no further replan events are required, the project usually can go forward and will ultimately be deemed successful. The blunt fact is that with software projects--as with all flavors of complex engineering projects--the final timeline and budget often vary greatly from the original plan; ask the good people of Boston. One key issue for the sponsor group at the point of the replan is to be able to stand back far enough to realize that if the project substantially delivers the planned benefits, in five years time no one will care that the project was delivered six months late. They will only care that the project was done in a quality way and delivered results that will continue to benefit the organization.
George Grieve is CEO of CastleBay Consulting. Previously a CIO and still an acting consultant, he has spent much of the past 25 years with property/casualty insurers, assisting them in the search, selection, negotiation, and implementation of mission-critical, core insurance processing systems. He can be reached at 512-329-2619.
The content of "Shop Talk" is the responsibility of the author. Views and opinions are those of the author and do not necessarily represent those of Tech Decisions.
