Compliance leaders are seeing an explosion of artificial intelligence (AI) based tools designed to support regulatory compliance. It can be daunting to sift through the vendors and in-house solutions to determine where AI can help and what is best left to your in-house team.
These new AI compliance tools can fill glaring in-house needs. Compliance teams are often underfunded, underresourced, and understaffed, while regulatory obligations seem to continue to grow. AI offers a caffeine boost to one’s monitoring capabilities and can help leverage limited resources. On the other hand, compliance officers are appropriately wary of solutions that can create errors at scale, have hallucinations, or are biased.
Guidelines for evaluating AI compliance tools
- Does the compliance task involve standardized repeatable checks? AI is great at automating standardized tasks like determining if signatures on a document match, or if mandated state fees or dollar thresholds are complied with on a loan.
- Is the AI solution transparent? It’s important that your compliance team or a regulator can understand how an AI tool arrives at its recommendation or action. For example, if the AI tool flags a transaction as fraudulent, it should be able to explain the basis of the decision.
- Is the AI solution auditable? You should build your compliance program as if regulators are going to examine your records. To that end, you should look for solutions that provide evidence you can submit to a regulator justifying particular actions. Another key question to ask to a vendor is if they have a robust model governance program for their models.
- Does the vendor stand by the performance of its product with accuracy service level agreements (SLAs) and automation SLAs? Some compliance solutions offer “all hat, no cattle.” Closely scrutinize any offered solution to ensure it works as advertised. We, at Informed, promote both accuracy and automation level commitments. If you’re seeking a tool to automate a compliance process, the vendor should commit to a certain automation rate and prove that out. Similarly, if a process, like data extraction from a document, must be accurate, make sure the vendor commits to a certain level of accuracy.
- Consider taking an incremental approach to deployment: Roll out your AI compliance in stages. Start with a small part of your service coverage, and iron out the kinks and identify the edge cases that trouble the solution. That way, when you roll out across your product ecosystem, you can avoid systemwide failures that could put your company at risk.
- Adding Human–in-the-loop: Consider establishing a human backstop for AI-driven decisions that have consequential consumer impacts. The backstop could be as simple as having a data analyst periodically sample and verify a model’s output.
Human-in-the-loop not only makes practical sense, it’s consistent with the vibe of consumers and regulators. The Bipartisan House Task Force on AI from the U.S. House of Representatives just issued a 250 page report in December 2024. One of its recommendations was “keeping a human in the loop to help monitor and check for quality and accuracy in outputs can help.”
It is inevitable that compliance teams are going to increase their use of AI tools. The tools are getting better and better, and exceed human capabilities in many areas. As you consider your options, remember to do your due diligence.
