While there's no arguing that the OBD device allowed autoinsurance innovators to put UBI on the map, many in the industryare beginning to wonder if first-generation technology is the besttechnology. In this disruptive age of self-driving cars,alternative fuel vehicles and shared ridership, warehousing OBDdevice and mailing them to customers for do-it-yourselfinstallation seems a bit archaic. However, while a smartphoneplatform seems to offer a viable alternative, questions lingerabout the reliability and accuracy of data collected. Is it on parwith information collected by OBD devices?

OBD vs. Professional GPS
To answer thisquestion, we first compared the measurements of a mid-range OBDdevice, which reads vehicle speed from a Controller Area Networkbus to a commercial grade GPS navigation system. We collected datafrom 10 cars during the three months. Before comparing the OBDdevice to a smartphone platform, we wanted to see how it matched upagainst the absolute speed standard–the commercial grade GPS.Results are shown below: 

The cars selected for the study varied in model, make and age.Unfortunately, hybrids are not compatible with OBD devices so wewere not able to use the Toyota Prius as part of the test asoriginally planned.

In this initial baseline test, we found that in newer cars, theOBD device achieved an average of 98% accuracy for measuringmileage when compared to the GPS. Accuracy dropped to an average of85% in cars more than 10 years old.

While performance varied by car make, model and year, there wereno major issues with mileage accuracy. Unfortunately, brakingdetection results were not as strong.

To understand the reasons behind braking event inaccuracy, letus first take a close look at how mileage and braking events arecalculated.

Mileage Calculations

An OBD device is alerted that driving has begun when it detectsan electrical spike from the ignition. Ideally, as soon as thedriver turns the key, the electrical spike is detected and the OBDdevice starts measuring mileage. Conversely, when the key is turnedoff, the OBD device stops measuring mileage. Nevertheless, the OBDdevice is always plugged in and slowly draining the car battery.During our study, we found that OBD devices did not always detectthe ignition spikes in cars with older electrical systems,accounting for the deterioration in mileage tracking results inolder vehicles.

Braking Event Calculations
Braking eventcalculations are more complicated. OBD devices take vehicle speedreadings from a car's onboard computer approximately once everysecond. A braking event occurs if a vehicle slows down by at least7 mph per second.

To detect a braking event, the device measures the Speed at TimeInterval 2 minus the Speed at Time Interval 1 divided by the timedifference between the two events and compares it to a certainthreshold. The formula is (V[t2]-V[t1]/(t2-t1).

So, for example, if a vehicle is travelling 50 miles an hour inone second and 40 miles an hour in the next second, there is a dropof 10 miles per hour in a one second timeframe, triggering an OBDbraking event, which is usually defined as 7mph/sec.

However, because the OBD device was not designed forcontinual braking event calculations, this method has some criticalflaws:

1. OBD speed values have no attached timestamps.This means that interval (t2-t1) is actually unknown, although theOBD device assumes it to be 1 second. However, if there are smallvariations, and the interval is actually 0.7 second or 1.3 secondsfor example, it will cause an incorrect calculation and brakingevents may be over or understated.
2. Car computers send OBD speed values as a truncatedinteger (a rounded number). In some cases a 10.5 mph speedvalue may come through as 10.0 mph and a speed of 19.7mph couldcome through as 20 mph. These small variances can add up to brakingevent oversights and in some cases, overstatement.
3. Multiple speed readings. Some vehicles sendmultiple speed readings at the same time, causing even furtherinaccuracy – particularly since there are no timestamps.

Based on initial findings, the OBD device did not appear to beworthy of a "gold standard" status. To take the analysis one stepfurther, we asked, "Can a smartphone perform any better?"

Smartphone Platform Challenges andRequirements
Many questions have been raised aboutthe viability of a smartphone app used as a telematics device,including:  

• What if the driver turns the smartphone off or the battery runsout?
• How can the smartphone differentiate between a braking eventand an event in which the phone is simply dropped?
• How will drivers remember to press a "start" button before eachtrip?
• How will rides on public transportation affect UBIscoring?

Because of these variances, and based on years of extensivetesting and research, we determined that a smartphone app, on itsown, could not compete with the data accuracy of an OBD device.Even a highly developed and sophisticated smartphone app, with autotrip detection, battery-efficiency and no fixed positionrequirements would have difficulty achieving comparable dataaccuracy.

To compete with OBD accuracy, a smartphone platform must be ableto capture all trips, remove unrelated trips and compensate formissed trips. To achieve these three goals requires more than anapp. The smartphone app must be paired with powerful cloudanalytics.

Five requirements for accurate smartphone UBI datameasurements:

1. The smartphone app must feature very reliable automatic tripdetection. The user cannot be relied upon to press a "start" or"stop" button. The accuracy of trip detection has to be above 90%.Our studies showed that only 4% of drivers would remember to press"start" each time they begin a trip if asked to do so.
2. The app must have very low battery overhead so it cancontinuously capture all trips without killing phone functionality.Ideally, it should have battery overhead of 10% or less.
3. The cloud analytics component of the smartphone platform mustinclude unrelated ride filtering capability. It must capture allactivity and then filter out rides in trains, airplanes, boats andtaxis as well as other situations.
4. The smartphone platform must have a mechanism to compensate formissing data during times in which the phone was compromised–leftat home, the battery ran out, or it was just turned off. Periods ofmissing data should not influence the overall UBI scoring.
5. For accurate event interpretation (i.e. to understand thedifference between a dropped phone and a hard brake), all of thesensors in the phone must work together through a process known asSensor Fusion. GPS alone is not enough.

Our findings suggested that if a smartphone platform met thesefive requirements, then the data collected would be on par, and insome ways better than UBI data collected by an OBD device.

Smartphone Platform vs. OBD
To furthertest this theory, we used three test subjects. For the purpose ofbrevity, I will cover the results of one test subject below. Duringthe test period, each test subject's driving behavior was measuredby both an OBD device and a smartphone platform which included asmartphone app combined with cloud analytics.

Test subject activity is shown above. The smartphone platformhad reliable automatic trip detection and on average, it used 7% ofphone battery per day.

The smartphone successfully captured everything that occurredwhile the phone was charged and turned on including passenger ridesvia train, boat, airplane and taxi. Cloud analytics detected andremoved the passenger rides so that only driving activity wasincluded in the UBI calculation.

Furthermore, the cloud analytics detected three days in whichthe phone was compromised (dead battery, left at home, left thephone turned off), accounting for the 9 missed car trips. To ensurethat the overall UBI score was not skewed by missed activity, thethree days missed were subtracted from the total activity and thescoring formula was adjusted accordingly.

Mileage Results

As you can see from the Mileage Results chart above, thesmartphone platform results were on par with the OBD device resultsafter the smartphone successfully flagged and removed unrelatedtrips and accounted for missed trips. Both the OBD device and thesmartphone platform were within 90% of the actual odometer resultsindicating a strong measure of reliability.

Braking Event Results

As mentioned earlier, the detection of braking events is highlyreliant on the collection of accurate second-by-second speedmeasurements. OBD devices were originally designed to diagnoseengine alerts at a moment in time–not on a continuous basis.Therefore, continuous, second-by-second speed measurement by OBDdevices have many inaccuracies.

Likewise, if a smartphone platform uses only a GPS device todetect braking events, it also has data flaws. That's because thetypical smartphone has an average low accuracy of 10 metersmeasured at one hertz, causing many false positives.

Conversely, if a smartphone platform is able to merge data fromthe GPS, the accelerometer, gyroscope and magnetometer, it capturesthe majority of braking events–actually performing slightly betterthan an OBD device. Sensor Fusion also allows the smartphone tointerpret events that occur and differentiate between a phone beingdropped and acute braking.

The UBI Decision Made Easier
Based on thisdata, we conclude that when a smartphone app is combined withsmartphone analytics and Sensor Fusion technology, mileage andbraking event data is on par with OBD data accuracy as long as thefive key requirements are met. Those five requirements areautomatic trip detection; low battery overhead; passenger rideremoval; missing trip accountability; and accurate eventinterpretation.

Now that we've answered the question of data accuracy,executives can look at the big picture factors of program cost,user experience and overall logistics to determine which usagebased insurance approach is most viable for theirorganizations.