Here's a three-step solution:

Build different "sub-agents," each focused on a narrower domain, such as providing details about a state's coverage requirements or how to make a payment.
Create extensive testing tools to eliminate hallucinations and ensure they aren't re-introduced in subsequent versions of each sub-agent.
Finally, build an action framework so that the virtual sub-agent can take actions on behalf of a customer in addition to answering a customer's questions.

Before you start building, consider these keys for success:

Get good at writing prompts. The answers you get from an LLM will be directly correlated to how good your prompts are. If writing prompts is new to you, take the time to learn ongoing and tap subject matter experts, like your insurance agents.
Iterate quickly. Build and test, again and again. GPT-4 can be unpredictable, especially for newer users. Iterate quickly and often to learn its idiosyncrasies.
Invest in testing. Monitoring a chatbot's performance without reading every single message thread is challenging. Build processes that showcase how changes to prompts affect answers to real customer questions. We read 1,000 messages a week and manually labeled responses, then consolidated findings to focus on the highest-priority items. This gave us an intuitive sense of how the LLM responded to prompt changes.
Version control is critical. This enables you to quickly launch versions of sub-agents, test them, and revert to the previous version if necessary.
Start small and grow. We started with a four-person team, including three senior software engineers. We maintained that size through most of the development, and eventually expanded to about 12 people.

Following these guidelines from the beginning will result in a chatbot that has a strong, scalable infrastructure. You'll be able to continue improving your chatbot and growing your AI. capabilities.

Special Reports /

How to overcome AI shortcomings to benefit your insurance business

Related Stories

Recommended For You