Best practices for integrating third-party AI with local systems

The vulnerabilities of integrating third-party AI

Relying on outside AI services can reduce work, lower costs and accelerate time to market. This brings valuable benefits to today's fast-moving business environment. However, businesses must carefully integrate external AI into their existing software and systems, which can introduce new problems. Although all third-party integrations carry risks, AI vendors present three specific vulnerabilities.

1. Questionable or unclear data use

AI vendors collect data each time a client uses their models or other software components. AI components might also need access to organization data stores, which the AI vendor might use for analytics and processing. It's critical to determine exactly what enterprise data third-party AI components access and understand precisely how the vendor retains and uses that data. Because a significant amount of enterprise data is confidential, sensitive and protected through compliance obligations, businesses should have definitive oversight and control over how a vendor accesses, stores and uses their data.

2. Unclear or fragmented permissions

Every third-party component or service requires permission to access the business's systems and data. The sheer number of possible third-party components and potential enterprise systems and data stores in service can easily lead to incorrect, incomplete or inconsistent AI permissions -- any of which can result in security vulnerabilities and sensitive data exposure. Apply zero-trust policies and ensure that third-party AI components receive correct, complete and consistent permissions.

3. Poor governance

Businesses must ensure that they -- and all their vendors -- share the same standards of governance and compliance. An AI vendor with weak or absent AI governance can potentially leak, share, resell or expose sensitive data or cause its AI components to perform inadequately. Businesses must ensure that AI vendors establish and maintain suitable AI governance.

Best practices for third-party AI integration

Despite the broad risks involved in third-party AI integrations, it is possible to address and remediate vulnerabilities with careful consideration, planning and preparation.

Operational aspects of AI integration

Addressing third-party AI integration issues often starts with detailed component selection and careful monitoring and validation of each component:

• Selection and planning. Use proof-of-concept projects to test third-party AI components and evaluate their performance, ease of use and compatibility with existing infrastructure, systems and software already operating across the enterprise. Compare behaviors from similar third-party AI components and select the best available offering. Limited third-party choices might mean more work to adapt and integrate the existing environment to the outside AI component.
• Workflow integration. The third-party AI component must operate to support -- and even enhance -- the existing AI workflow. Otherwise, the external AI service might introduce unwanted bottlenecks or impair AI performance, harming CX or limiting adoption.
• HITL validation. Consider how humans will interact with third-party AI components and the overall AI system. Human-in-the-loop protocols might be necessary for complex decisions or mission-critical outcomes. Third-party AI components without these vital human checks can result in serious consequences for the business and AI platform users.
• Monitoring and analytics. Use monitoring to log and evaluate the data sent to and from the third-party AI component. Analyze the component's monitoring data for accuracy and performance, as well as the overall AI system it supports. This can help determine how the external AI service affects the overall AI platform and provide an early warning of potential problems with the third-party AI component.

Security and privacy in AI integration

Third-party AI requires data from the business. It might be something as simple as user prompts or involve large volumes of data for analysis. Regardless of the data type or volume, integration must address data security and privacy in the following ways:

• Security alignment. Evaluate the security features of third-party AI components and ensure they align with the business's security requirements. Apply techniques such as zero trust for third-party AI agents and other components to ensure that third-party AI components can only access the minimum local resources needed to work.
• Data anonymization and encryption. Protect local data stores. Use encryption to safeguard data in flight to and from outside AI services. When third-party AI needs to access sensitive or personally identifiable information (PII), consider applying techniques such as data anonymization to shield PII from potential exposure.
• Permission access. Carefully examine permissions and consider how they map to existing role-based access control or other local access control mechanisms. Minimize access to external services and ensure that humans review and approve any changes to permissions before implementation.
• Secure networking and connectivity. Local data encryption might not be enough on its own. Sensitive or critical AI systems might use secure networks and connectivity with techniques such as network segmentation, virtual private networks or zero-trust network access to enhance secure data exchanges between the business and any external AI components.

Compliance and governance in AI integration

Businesses must meet compliance and governance requirements. They also shoulder the burden of ensuring that outside vendors and service providers meet the same regulatory requirements. Although a business cannot control its outside vendors, several considerations can help validate vendors with suitable compliance and governance:

• Vendor review and assessment. Understand the relevant compliance and governance issues related to third-party AI providers and work with prospective AI vendors to address them. Reconsider AI vendors that cannot directly address compliance and governance issues. Perform vendor reviews regularly to ensure providers continue to meet evolving governance and compliance requirements.
• Contractual agreements. AI providers typically involve a contract or a legally binding agreement. While the provider usually offers a boilerplate terms-of-use document, business clients can often pursue a contractual relationship that clearly stipulates the understandings and requirements expected from the business client. As an example, such agreements might specifically restrict the retention and use of data collected from the business.
• Data usage restrictions. AI providers typically receive and process data from their client users. This can range from LLM prompts for search queries to vast data access for analysis and decision-making. Considering how business data is often confidential or domain-specific, no business wants to reveal sensitive information through AI use. Pay close attention to data usage restrictions in AI providers' terms of use and any contractual agreements.
• AI use monitoring and logging. AI compliance and governance involve both external and internal monitoring. External monitoring involves tools to oversee data sent to third-party AI providers and data received from them, helping ensure that AI services are used as intended and work as expected. Internal monitoring typically involves AI inventory management to ensure that only authorized AI services are being used, thereby identifying and preventing shadow AI across the business.

Approaches to AI integration

Methodology dictates how a business accesses an external service provider and the nature of its interaction. Several important approaches affect access and security:

• APIs. An API is the most common means of integration. For example, cloud-based AI models such as OpenAI and Google Cloud use APIs to support data exchanges in real time. APIs are treated as middleware and managed as a separate software platform -- for example, where APIs are version-controlled and updated.
• RAG. A retrieval-augmented generation (RAG) system connects external models, such as LLMs, to internal data stores. AI models that blend training with local content can typically deliver more accurate and relevant responses while reducing the risk of hallucinations. However, access to local data stores can present greater security and regulatory risks for the business.
• AI agents. Agentic AI relies on external autonomous agents to handle complex, on-demand tasks with little human oversight. Because agents can access data and make decisions, they demand careful security and close monitoring to ensure that agentic AI actions deliver the intended outcomes. Critical decisions might require human-in-the-loop approvals.
• Local integrations. Some third-party AI components can be deployed locally and operate within a business's infrastructure. These components, available from third parties but deployed locally, can support private AI computing with low latency and greater security. This can be a preferred approach for sensitive or highly regulated industries.

Stephen J. Bigelow, senior technology editor at TechTarget, has more than 30 years of technical writing experience in the PC and technology industry.