How data mesh supports AI-ready data architectures

Data mesh principles

The core idea behind data mesh is decentralization. Each department or business unit -- say finance or marketing -- owns, manages and accesses the data it needs on a self-service basis. This is the opposite of the more conventional approach to data architecture, in which all data is centrally managed by a single team of data engineers or analysts.

Implementing a data mesh usually involves combining a data platform -- often based on a warehouse or lakehouse architecture -- with an operating model that enables business domains to manage, govern and share their own data products. To be clear, not all data warehouses and lakehouses are implemented as a data mesh, but many can be configured that way.

The role of data meshes in AI enablement

Data mesh's primary benefit for AI deployment is that it makes it easier for business users and technical teams to access the data they need. By reducing dependency on centralized data teams, business units can roll out AI systems faster with a lower risk of underperformance.

For context, consider a marketing team that wants to deploy an AI tool to help plan and execute marketing campaigns. To perform well, the AI system needs access to various datasets, such as customer databases, sales enablement content, revenue systems and product documentation.

With a data mesh in place, the marketing team can access approved data resources through a self-service process, enabling faster AI deployment while maintaining governance controls. Marketers can also determine how data is prioritized and used -- for example, instructing the AI system to ignore outdated content -- based on their expert understanding of the business's marketing needs and the most relevant data.

Without a data mesh, business units may need to wait for the central data team to give access to the resources required by the marketing AI system. In some cases, not all the necessary data may be available when the system begins operating, leading to subpar results. Limited domain-specific knowledge among centralized data teams further complicates deployment. In this case, abandoning the AI project might be the ultimate outcome if the system fails to justify continued investment.

Can a data mesh benefit your AI strategy?

Whether a data mesh can speed AI adoption depends largely on how significant the data access challenge is within a business.

Centralized architectures are not inherently flawed; they can work well when central data teams respond quickly to requests and communicate effectively with business units.

But if enabling data access or building integrations between data sources and other systems is a common problem -- not just when deploying AI, but during any business project -- it's likely a data mesh can help overcome it.

Optimal data mesh use requires a moderate level of data literacy across the organization, along with clear governance standards and defined accountability for data ownership. These capabilities help business units independently manage data resources while upholding appropriate controls. Otherwise, a centralized data architecture in which a team of experts oversees all data initiatives usually works better than a data mesh.

Limitations of data meshes for AI deployment

Data mesh won't solve every data-related problem that complicates AI projects. On their own, data meshes don't address the following issues:

• Inaccurate, missing, incomplete or otherwise low-quality data that causes AI systems to fail to generate accurate insights.
• Poorly designed or implemented data integrations that cause data to be inaccessible even if it's nominally connected to an AI system.
• Insufficient data security and privacy controls, which lead to security risks in AI systems connected to the data.

Data mesh can be a valuable step toward addressing the data woes that undercut AI deployment, but it only addresses data access issues. A comprehensive AI data strategy also requires strong data quality, governance, security and integration practices.

Chris Tozzi is a freelance writer, research adviser, and professor of IT and society who has previously worked as a journalist and Linux systems administrator.