4 potential risks from shadow AI use in accounting

1. Accounting handles sensitive data

An accounting department handles various sensitive information. Payroll registers and client tax files contain information that must not end up in the wrong hands. If a user pastes financial information into a personal chatbot account, the financial data is now subject to the provider’s terms, including potentially being used to train their systems.

CFOs must begin by establishing who is using what tool in their department instead of simply sending out a policy document.

Ask employees what AI applications they are using, what data they are putting into the applications and what problems each tool solves, then give them access to an approved tool that can securely solve those issues. Blocking AI tools without providing a new option could cause employees to secretly continue using unsecured tools.

2. AI may not provide enough reasoning during an audit

AI-drafted analyses may seem sufficient, but problems can occur when an auditor asks how an estimate was developed. The person who signed the workpaper may not be able to reconstruct the reasoning, the source material may not have been preserved, the prompt and output may be sitting in a personal AI account, or the model may have inserted assumptions that nobody noticed because the answer “sounded right.”

Issues can also occur during an investigation or litigation because the accounting work done in a personal AI account is likely outside of the company’s collection tools, data retention schedule and legal-hold process.

Accounting departments must preserve the source data, prompt, relevant output, validation steps and documentation of the human judgment that was part of the decision, with all of that information stored in an approved system.

3. Shadow AI may be part of approved software

Blocking the most well-known AI applications can lead to a false sense of security. AI is now becoming part of various business applications, such as the following:

• Browser extensions
• Email
• ERP platforms
• Expense systems
• Meeting software
• Recording and transcription services
• Spreadsheets

An AI feature may be automatically activated during installation or a product update, leading to sensitive information moving to a third party without anyone in the accounting or legal departments knowing about it.

CFOs should work with IT to review the application list for the accounting department, then review the applications’ settings and permissions. Determine which AI features are enabled and then find out what data they can access, the storage location of prompts and outputs and whether the provider uses the data for model training. Pay special attention to meeting recording and transcription services, including those that record online meetings.

4. AI use can lead to accounting compliance problems

Accounting departments are already subject to confidentiality, security and records obligations that predate AI. Accounting must continue to ensure that their operations are compliant.

CFOs don’t need to create a new program – much of accounting compliance involves security principles and controls that are likely already in place. Continue to use existing systems for processes like the following:

• Access control
• Asset inventory
• Change management
• Data classification
• Legal hold
• Offboarding
• Retention
• Vendor review

AI is another system that can access sensitive data, so it must be treated – and monitored – accordingly.

Kevin Beaver is an independent information security consultant, writer and professional speaker with Atlanta-based Principle Logic, LLC. With more than 30 years of experience in the industry, Beaver specializes in performing vulnerability and penetration tests, as well as virtual CISO consulting work.