Summary: So what do you think of the generation of artificial intelligence? Good or bad, it's an emerging risk that insurers are, as they should be, taking note of for both its potential and its possibility of creating significant risks. Our good friends at ISO held a roundtable discussion that was informative and brought us up to date on the potential risks associated with generative-assisted artificial intelligence (GAI). Here we will share some of what we learned from the discussion, as well as some of our own thoughts on the emergence of this highly volatile risk.
Topics Covered:
GAI has an enormous amount of data to work with, from virtually every industry from every continent. Think about all the knowledge of every scientist, researcher, developer, artist, composer, engineer, stockbroker, president, educator, military professional, and the list goes on and on. Every book, every article, every quote, every word that has been made available through the internet. Think of all the brains of every think-tank organization. Then start brainstorming about how that information can be used for good or for bad.
The GAI potential for good is tremendous. For example, drug discoveries that could eliminate diseases and prevent pandemics by identifying antibodies, being able to diagnose and treat complex health issues by description or pictures at the click of a button, having a robotic chatbot to take patient notes, respond to patient messages and summarize patient data, gather and summarize data that will result in the development of quick and predictable models that will virtually eliminate the guesswork out of assessing insurance, legal and financial issues and risk, just to name a few.
GAI Processes
GAI uses the following process to produce its final output. It takes the available data, either public or from any dataset, and computes it into text, images, video, or code. It applies machine learning techniques to fine tune the data to its useful purpose, termed tailored datasets. This can be done via a transformer or neural network that can be trained for given tasks. The transformer, or network, helps the model predict the next sequence of text and it is fine tuned, over and over, to reach its desired output. It essentially takes the large volume of data and asks it multiples of questions, or prompts, to fine tune the information into a smaller dataset. The model can be produced in an open sourced or closed source platform‒‒open where it can access all of the underlying source code and that data can be accessed publicly‒‒or closed, meaning that the underlying code is not accessible publicly. The algorithm then produces the output based on its prompts, which can be via text, image, video, code or other method. Choosing whether to use an open or closed source platform is important in assessing its risk. For example, an open source platform makes the dataset available for all uses by anyone, but it also increases data privacy issues. However, using a closed source platform hides the data so that it can grow and use data that may not be known about, increasing the potential for privacy violations and possible violation of copyrights.
GAI Liability
There is a lot of enthusiasm for GAI's potential, but there are also a lot of potential liability concerns due to violation of laws of privacy, competition, unfair trade practices, copyrights, bias and discrimination, and more. Training datasets may make use of copyrighted material, outputs may share with others without permissions, leading to liability for creators or downstream issues.
So what could go wrong? Well, there are some known potential flaws and systemic liability risk. The tools are not always transparent in how they work, and it may take some time for the negative impacts to come to light. Some of the known potential flows include IP infringement, personal identifiable information privacy and theft issues, and privacy violation for use of likeness. The dataset could contain leaked or breached data. For example, one person found photos of herself at a doctor's office that were apparently taken from her medical records. There have been suits for face swapping images, with some celebrities' images allegedly having used without their consent, violating the California right of publicity law.
Chatbot hallucination is a known risk. Companies used to define how software was supposed to behave by entering code one line at a time. Now chatbots and other technologies can learn skills on their own from the information they absorb, whether or not that information is true, and a dataset can build and learn based on another dataset which may or may not be accurate. When a chatbot generates a response that is either factually incorrect or is unrelated to the given context, leading to an unexpected outcome, it is known as AI hallucination. An example is when Google's Bard answered questions about the James Webb Space Telescope and incorrectly stated that the telescope had captured the very first pictures of a planet outside our solar system. Or when ChatGPT was asked to tell a fact about George Washington, it generated the response: "George Washington was known for inventing the cotton gin." Actually, Eli Whitney invented the cotton gin
Why is AI hallucination a problem? Well besides the fact that the information may be untrue or irrelevant, it erodes trust in the technology and can also potentially create or perpetuate stereotypes or biases. If used to generate critical decisions in areas such as finance, legal, insurance, healthcare, etc., it can lead to poor responses with devastating consequences. Not to mention the legal liabilities from producing inaccurate, discriminatory or biased information.
Some potential exists in professional and management liability risks. In one example, a company used AI for preparation of legal work that generated multiple lawsuits because the customers claimed that the documents received were substandard. If a company uses GAI to automate corporate workloads, the potential for inaccuracies or omissions could draw negative attention from regulators as well as create liability for failure to disclose, misrepresentations, or other. If errors or other flaws are not caught in GAI output, there could be errors or omissions in advertising, advice, paperwork, etc., that leads to liability for faulty work. If used in insurance or financial audits, it could pull in liability for the auditors for failing to catch errors. Using AI in medical decisions could lead to potential liability for medical malpractice or discrimination. A major US hospital used an algorithm that miscategorized black patients for urgency of care.The researchers calculated that the algorithm's bias effectively reduced the proportion of black patients receiving extra help by more than half, from almost 50 percent to less than 20 percent.
Some ways that GAI can be used in design and manufacturing include designing chips, writing code, manufacturing a car, designing buildings, discovering new drugs, and much more. There is potential for flawed designs or faulty production that could lead to potential liability for negligent design or defective products.
GAI Risk Mitigation
Can the risks be mitigated? Well, somewhat. Over time, GAI capability may improve overall. Human review, product testing, and other vetting can be effective. Using narrower, more specialized datasets could reduce risk. However, not every flaw can be prevented, such as hallucinations. GAI transparency and quality testing are ongoing challenges. AI tools can be in black boxes, and can be so complicated that humans can't explain their processes. Researchers are looking at how to test the tools, but there are challenges, benchmarks are becoming saturated. An AI tool can perform well against benchmarks but still be imperfect. Upstream GAI applications may fly under the radar, although some companies are taking steps to make this less likely. Disclosures may be needed if public AI data is being used in the datasets, which may lead to requirements that disclosures be included in contracts.
Even when AI is working as it should, there are potential concerns when it gets used in the wrong way, or is put into the hands of bad actors.This leads us to cyber concerns.
Many of the existing avenues for cyber crime may be easier when GAI is used. Phishing incidents have shown a 61% increase since 2022, and the attacks have become more sophisticated. Turing tests show a rise in phone scams due to voice clones, with people thinking that they are speaking with relatives and giving out sensitive information when it is only a voice clone. The Turing Test is a method of inquiry in AI for determining whether or not a computer is capable of thinking like a human being. The test is named after Alan Turing, the founder of the Turing Test and an English computer scientist, cryptanalyst, mathematician and theoretical biologist. Turing proposed that a computer can be said to possess artificial intelligence if it can mimic human responses under specific conditions. Mistakes in email and text messages help detect a scammer, but chatbots help the bad actors to make better emails and texts with fewer mistakes.
Then there are deep fakes – videos which look and sound like a real person. These are already out on social media and have a lot of potential to use in video phishing attacks. Attacks increased by 40% since last year. For example, if someone looks and sounds like your manager and they ask for sensitive information or authorize a large cash transfer. These are harder for automated systems to detect at this time. The GAI can put out the code, but the code could be malware.
Demonstrations show that certain prompting sequences can get around automated safeguards. Deep fakes can be used to dupe people and systems to get through, and there is a market for fakes to break biometric security – markers can be changed, passwords and social security numbers stolen. A person's face, voice, and fingerprints can be used over and over. For example, an open sourced AI could be fine tuned to deliver a malware code, and even people with little to no coding experience can use the models to hack a system. If a company adopts the model, even a fine tuned one, it could still be a problem as it could contain malware or give hallucinogenic answers.
Then there are malicious prompts and data poisoning, when the data is skewed by answers to the prompts in the model creation that indicate the answers are correct or incorrect, but the opposite is true. This misleads the model so that the output will be inaccurate or lead to an undesired outcome. It can pull personally identifiable information when it may be only available in a single document among millions of documents and data. Then if a company adopts the model for public use and the data is extracted and used the company will be held accountable.
The model could go off script, intentionally or unintentionally. In February, media published a transcript where the reporter interviewed a chatbot, asked several questions, and eventually the chatbot claimed to love the reporter and suggested they get a divorce.
GAI can help fuel social inflation. There are chatbot lawyers giving GAI assisted legal analysis and there are reportedly already some tools in place for some legal applications. Social inflation makes it easier for people to sue in individual or class actions, to analyze past cases, look for elements in new cases, giving rise to increased frequency of litigation. In some cases GAI has been used by litigation funders to direct capital to certain cases.
GAI could even generate civil unrest – online mis- or dis-information contributes to real world violence, and heightens the risk of civil unrest. It can lead to financial panics, mass unemployment. For example, one chatbot estimated that up to 300 million people could be unemployed by a certain event/time. A financial panic happened on 5/22 when an image of the Pentagon on fire was distributed by social media and the stock market dropped immediately. The picture was phony but it was on a real account. It was temporary, but this just shows the affect that a single image can have. AI generative campaigning or live reporting can lead people astray or put them in a panic. These things could lead to a downturn in the market or affect individual companies or offices.
GAI Emergence
There are potential risks on the horizon. Two open letters of academics highlighted long term risks from either a pandemic or a nuclear event. Systems will continue to improve, as AI will be used more and more, but in each case unexpected action or loss of control over systems could create damage.
Proxy gaming is when an AI system is given a goal, attempts to achieve that goal and in the process creates unanticipated harm. For example, an AI firm experiment created a game where the goal was for the boat pilot (player) was to achieve the most points. AI achieved the goal, gaining 20% better than its counterparts, but did so by repeatedly crashing into other boats, went in the wrong direction and even caught its own boat on fire; all were unexpected consequences.
Emergent abilities and goals are possible when an AI system gains capabilities and creates goals that are unanticipated and harmful. So far, this has been benign, except for more sophisticated mathematical calculations than were anticipated; but the more it learns, this could be an issue. Some of these emergents may not be discovered until after the model has been deployed. Having a surprising result from a chatbot is one thing, but information generated from a law robot is quite another.
GAI could lead to an intelligence explosion, as some domains may have AI superior to humans according to mathematician turned computer scientist, Irving John Good. Now it is domain specific, but there is a push by some developers to exceed capabilities by a continual process of training and recoding across multiple domains, so it keeps the system growing and learning. This process may exceed more functionality across domains, but it is unpredictable, rapid, hard to control, and an emergent risk. If it gets really smart, the system may be more capable and have a broader scope to impact the entire world.
In 1965, Irving John Good quoted the following about the intelligence explosion:
"Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."
Regulations
It is well understood that further research is warranted and that regulations are needed to reign in the potential scope of artificial intelligence competencies and its impact on humanity and individual rights and protections.
To that end, a number of studies and regulations have been put into place or are being considered, at the federal, national and state levels.
Federal
The White House Office of Science and Technology Policy has identified five principles that should guide the design, use, and deployment of automated systems to protect the American public in the age of artificial intelligence. The Blueprint for an AI Bill of Rights is a guide for a society that protects all people from these threats—and uses technologies in ways that reinforce our highest values. Responding to the experiences of the American public, and informed by insights from researchers, technologists, advocates, journalists, and policymakers, this framework is accompanied by From Principles to Practice—a handbook for anyone seeking to incorporate these protections into policy and practice, including detailed steps toward actualizing these principles in the technological design process. These principles help provide guidance whenever automated systems can meaningfully impact the public's rights, opportunities, or access to critical needs.The AI bill of rights represents a comprehensive approach that rests on five principals: safe & effective systems (pre-development testing, risk identification and mitigation, ongoing monitoring of safety and effectiveness based on intended use), algorithmic discrimination protections (no unjustified treatment or impacts disfavoring people for anything protected by law), data privacy (protection for privacy violations through design choices, ensuring that data collection is reasonable and used only for its specific context), notice and explanation (plain language use including clear descriptions of system functions, and notices of key functionality changes), and human alternatives, consideration and fallback (opt out options, access to human consideration, fallback remedies as necessary if system fails, produces an error or if appeal is human requested).
National AI Research Resource (NAIRR) (2023)
In 2020, the National AI Initiative Act called on the National Science Foundation (NSF), in coordination with the While House office of Science and Technology Policy (OSTP), to form a National AI Research Resource (NAIRR) Task Force to investigate the feasibility of establishing a NAIRR, and develop a roadmap detailing how such a resource could be established and sustained. The roadmap would be a shared research infrastructure that would provide AI researchers and students with significantly expanded access to computational resources, high-quality data, educational tools, and user support, which previously such advanced data and computational power would be often unavailable to those outside of well-sourced technology companies and universities. OSTP and NSF formally launched the NAIRR Task Force in June 2021, appointing 12 leading experts equally representing academia, government, and private organizations. In January of 2023, the NAIRR Task Force presented their final report on the suggested roadmap and four-phase implementation plan for a national cyberinfrastructure that would reap the benefits of greater brainpower, plus perspectives and experiences applied to developing the future of AI technology and its role in our society.
National Institute of Standards & Technology (NIST)
The United States and European Union have an arrangement to work together on further research on AI, computing and privacy protection – emergency response – grid optimization – agricultural optimization. A new public working group on generative AI was launched this month. The working group aims to provide essential guidance for organizations developing, deploying and using generative AI to help minimize potential harm from such technologies.
Office of Educational Technology
The U.S. Department of Education Office of Educational Technology's new policy report, Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations, addresses the clear need for sharing knowledge, engaging educators, and refining technology plans and policies for artificial intelligence (AI) use in education. The report describes AI as a rapidly-advancing set of technologies for recognizing patterns in data and automating actions, and guides educators in understanding what these emerging technologies can do to advance educational goals—while evaluating and limiting key risks.
The FTC, EEOC, and other areas have measures in place to monitor AI development and establish guidelines in an effort to prevent unlawful discrimination or violation of privacies.
The National Institute of Insurance Commissioners (NAIC) formed the Innovation Cybersecurity and Technology (H) Committee (formerly Innovation and Technology (EX) Task Force). In 2019, the task force established the Big Data and Artificial Intelligence (H) Working Group to study the development of artificial intelligence, its use in the insurance sector, and its impact on consumer protection and privacy, marketplace dynamics, and the state-based insurance regulatory framework. The NAIC Principles on Artificial Intelligence establish guiding principles for AI actors and systems that address five key areas: accountability, compliance, transparency, and safe, secure, fair and robust outputs.
In 2021 the group began surveying insurers by line of business to learn how AI and machine learning techniques are being used, and what governance and risk management controls are in place. The surveys included a specific definition of AI that is different than a standard algorithm, as it considered a subset of AI that covered all aspects of the insurance life cycle of AI and machine learning (ML). It is set up to recognize patterns without being programmed to come up with a predetermined output. The personal auto surveyed 193 companies, and 169 of those indicated that they were planning to use or explore the use of AI / ML (machine learning). The most uses were for claims (70), followed by marketing (50), fraud and underwriting (18), and loss prevention (2). A homeowners survey is planned.
The same working group drafted model questions intended for use in market conduct examinations. The working group will coordinate with the Innovation, Cybersecurity and Technology (H) Committee to develop market conduct examiner guidance for the oversight of regulated entities' use of insurance and non-insurance consumer data and models using algorithms and artificial intelligence (AI). State Activity
States have been focused on model algorithms and bias.
CO state bill 169 looks at unfair discrimination, primarily on life insurance. Rule making continues; the governance for life insurance is in its second draft. The Department hopes to have draft for life insurance done by the end of the month. It likely won't serve as the basis for P&C. The insurance division is holding a stakeholder meeting on private passenger auto.
DC – The department has obtained data on auto bias, has submitted it to ORCA (in house consultant) and is seeking carrier input.
CA – The state department has issued a racial bias and unfair discrimination bulletin addressing all aspects of insurance. It reminds insurers to treat everyone alike and states that discrimination will be investigated. Companies should avoid both conscious and unconscious bias. A state workshop examined the use of AI and its impact on consumers. Plans are to make a formal rule process ensuring fairness, transparency and privacy. The commissioner is very involved in AI matters.
CT – The state has issued a formal notice to all licensed insurers stating that the use of technology and big data must be in full compliance with all laws, federal and state, and complete a data certification annually.
Filings/Market Conduct
There have been some draft questions on models; they have started to come up although nothing has been formally adopted. Both AI models and bias are on the minds of insurance regulators.
Includes copyrighted material of Insurance Services Office, Inc., with its permission.