Document management system concept with a computer keyboard and office items. Credit: Tierney/Adobe Stock
Ask any underwriter what their day actually looks like, and they'll tell you the same thing: A submission arrives. Before any real underwriting judgment happens — before any risk is assessed, priced, or declined — someone has to find the relevant information buried across a loss run, a property schedule, an inspection report, and a supplemental application.
That work happens first. It takes time. And it adds no underwriting value whatsoever.
This is the problem AI is beginning to solve in insurance. Not the judgment part; not whether a risk is acceptable or what it should cost. The part that comes before judgment: finding, organizing, and surfacing the information that makes judgment possible.
But the carriers seeing real results aren't the ones who simply bought a document AI tool and pointed it at their inbox. They're the ones who understood something important before they started: Getting information out of a document is the easy part. Knowing what to do with it afterward is where the real work begins.
Why simply reading documents is not enough
The promise sounds simple. Feed a document into an AI system and get structured, usable data back. No manual re-keying. No hunting through pages for a single figure. Just clean, organized information ready for the underwriter to act on.
And in a controlled environment, with well-formatted documents and a narrow set of document types, that promise often holds. The trouble starts when the real world shows up.
Insurance documents don't arrive in a standard format. Loss runs look different depending on which carrier produced them. Inspection reports vary by inspector, by property type, by region. A submission from a large commercial broker looks nothing like one from an independent agent. And documents don't always arrive as clean digital files. Some come as scanned images, some as email attachments, some through online portals, some as combinations of all three.
A system that reads documents beautifully in a demo but struggles with the actual variety of what comes through a real submission workflow is not ready for production. It is a pilot that hasn't been tested yet.
The problem no one talks about in the demo
Here is what vendor presentations almost never show: what happens when the system gets something wrong.
In manual review, an underwriter catches most errors naturally. They're reading the whole document, applying context, and cross-referencing details the way an experienced professional does. When something doesn't look right, they notice.
AI systems don't work that way. When an AI misreads a coverage limit, transposes a date, or pulls a figure from the wrong section of a multi-page schedule, the output looks exactly the same as when it got everything right. The error doesn't announce itself. It flows into the underwriting workflow as if it were confirmed fact — where it can affect a pricing decision, a coverage recommendation, or a risk assessment before anyone realizes something went wrong.
This is the most under-appreciated risk in document AI adoption. It's not that the systems are inaccurate. Most are reasonably accurate, most of the time. It's that when they are wrong, the error is invisible — and invisible errors in underwriting have consequences.
The carriers that manage this well don't just measure how often their systems are right. They build deliberate processes for catching and managing the times they're wrong.
What good actually looks like in practice
The implementations that deliver sustained value share a common design principle: AI handles what it handles well, and humans handle everything else — with a clear, deliberate boundary between the two.
In practice, this means that documents coming into the system are processed and sorted into two buckets. The first bucket contains information the system extracted with high confidence — fields that are clear, consistent, and well within expected ranges. That information flows directly into underwriting tools without anyone having to touch it.
The second bucket contains anything the system flagged — a missing field, a value that seems inconsistent with other information in the submission, something it couldn't read clearly. That information gets routed to a person, with a clear note about what needs to be checked and why.
The underwriter's job changes. Instead of spending the first hour of every submission hunting for information, they receive a structured summary with the relevant details already organized — and a short list of specific items that need their attention. Their expertise gets applied to the parts of the process that actually require expertise.
This sounds straightforward. Getting there requires more organizational work than most carriers expect — because the line between "what the system handles" and "what a person handles" has to be drawn carefully, by people who understand the underwriting process, not just the technology.
The real challenge is adoption, not accuracy
Once a document AI system is working reliably, the question shifts from whether it works to whether people use it — and whether it actually changes how work gets done.
The honest answer from real-world deployments is that adoption doesn't come from mandates. It comes from underwriters and their teams experiencing, firsthand, that the system saves them real time on real work. When a team that used to spend two hours organizing a complex commercial submission finds that the structured summary is already waiting for them — accurate, complete, and connected to the tools they use — they become the system's strongest advocates.
The teams that resist adoption are almost never resisting the technology. They're resisting a workflow that wasn't designed with them in mind. If the system extracts data but the output doesn't connect cleanly to the tools the team actually uses, or if the process for handling flagged items is more cumbersome than just doing it manually, the technology will sit unused regardless of how accurate it is.
The carriers that have achieved adoption across multiple business units followed a consistent pattern. They started with one team, one document type, one workflow. They listened to what underwriters actually experienced. They fixed what wasn't working. Then they expanded. The second rollout was faster. The third fastest still. By the time it reached multiple business units, the operational muscle was already built.
What the transition actually feels like
For the underwriters living through this change, the experience is less dramatic than the technology coverage might suggest — and more meaningful.
The stack of documents doesn't disappear. Complex risks still require expert eyes on source materials. Judgment about whether a risk is acceptable, and at what price, remains entirely human. What changes is the time and cognitive load that precedes that judgment.
The underwriter who used to start every morning with an hour of organizing and extracting now starts with a structured summary and a short list of questions. The risk manager who used to spend a day compiling data for a quarterly portfolio review now spends that day analyzing what the data means. The time doesn't vanish — it gets redirected toward work that actually requires the expertise these professionals spent years developing.
That, more than any accuracy number, is the real measure of whether document AI is working.
Before you start: The questions that matter
For P&C professionals evaluating document AI — whether you're in underwriting, operations, or technology — the most important questions aren't about the model. They're about what happens around it.
How does it handle document types that look different from what it was trained on? When it gets something wrong, what does that look like from the underwriter's side — and how quickly can it be corrected? Who decides which extractions go straight through and which ones need a human? How does the system get better over time as your team uses it?
If the answers to those questions are specific and grounded in real deployment experience, you're talking to people who have actually done this. If the answers are vague — "we'll figure that out during implementation" — you're still in the demo stage, regardless of how impressive the demo was.
Document AI is changing how risk review works in insurance. The carriers who get there first won't necessarily be the ones who moved fastest. They'll be the ones who thought carefully about what reliable looks like — and built for that, not for the demo.
Naveen Karakavalasa is Principal Manager of Emerging Technologies,Tokio Marine North America Services. He specializes in the design and deployment of AI systems for underwriting and claims operations. He writes at nkspace.dev.
Any opinions expressed here are the author's own.
(Lead image credit: Tierney/Adobe Stock)
© Arc, All Rights Reserved. Request academic re-use from www.copyright.com. All other uses, submit a request to TMSalesOperations@arc-network.com. For more information visit Asset & Logo Licensing.