Generative AI ethics: 16 biggest concerns and risks

1. Distribution of harmful content

Generative AI systems create content automatically from people's text prompts. "These systems can generate enormous productivity improvements, but they can also be used for harm, either intentional or unintentional," said Bret Greenstein, chief AI officer at West Monroe, a business transformation consultancy. For example, an AI-generated email sent on behalf of the company could inadvertently contain offensive language or issue harmful guidance to employees. GenAI should be used to augment but not replace humans or processes to ensure content meets the company's ethical expectations and supports its brand values, Greenstein advised.

2. Taking harmful actions

As AI systems move from generating content to taking action, accountability structures must evolve. Current accountability frameworks assume AI is a tool and liability flows to the humans who deployed it, said AI ethics researcher and author Nell Watson. However, as agentic systems make increasingly autonomous decisions, that model breaks down, and organizations should implement a "structured disagreement register," where both the AI system and the human decision-maker record their reasoning when they diverge, she said. This creates a corpus that reveals where each party adds value and where each introduces risk.

3. Copyright and legal exposure

Popular generative AI tools are trained on massive image and text databases from multiple sources, including the internet. When these tools create images or generate lines of code, the data's source might be unknown, which could be problematic for a bank handling financial transactions or a pharmaceutical company relying on a formula for a complex molecule in a drug. Reputational and financial risks could also be substantial if one company's product relies on another's intellectual property. "Companies must look to validate outputs from the models," Roselund advised, "until legal precedents provide clarity around IP and copyright challenges."

4. Data privacy violations

GenAI LLMs are trained on data sets that might include personally identifiable information (PII) about individuals. This data can sometimes be elicited with a simple text prompt.

Moreover, compared with traditional search engines, it can be more difficult for consumers to locate and request the removal of the information. Companies that build or fine-tune LLMs must ensure that PII isn't embedded in the language models and that it's easy to remove PII from these models in compliance with privacy laws.

5. Sensitive information disclosure

GenAI is democratizing AI capabilities and making them more accessible. This combination of democratization and accessibility, Roselund said, could potentially lead to a medical researcher inadvertently disclosing sensitive patient information or a consumer brand unwittingly exposing its product strategy to a third party. The consequences of unintended incidents like these could irrevocably breach patient or customer trust and carry legal ramifications. Roselund recommended that companies institute clear guidelines, governance and effective communication from the top down, emphasizing shared responsibility for safeguarding sensitive information, protected data and IP.

6. Enterprise data contamination

A related concern is how AI-generated content could contaminate enterprise data. Rahul Jolly, vice president of AI and data at OSF Digital, a Salesforce consulting company, warned that as organizations use GenAI to create content, summarize interactions and enrich customer profiles, that output gets fed back into core systems. Over time, AI-generated data becomes indistinguishable from human-verified data. Competitive advantage will shift from having the most data to having the most trusted data.

7. Compounded AI risks increase accountability challenges

The accountability challenge is compounded by the way AI risks interact. Compliance controls intended to manage privacy risk can spin up new databases of sensitive content that need protecting in their own right, said Hugh Mulligan, associate director of cyber risk and governance at S-RM, an intelligence and cybersecurity consulting firm. Cybersecurity teams that lock systems down too hard push users toward shadow AI that security teams can't see. Companies that will struggle most are the ones where AI risk sits in one department rather than being treated as an enterprise-level problem, he said.

8. Amplification of existing bias

GenAI can potentially amplify existing bias. For example, data used to train LLMs can contain biases beyond the control of businesses that use these language models for specific applications. It's important for companies working on AI to have diverse leaders and subject matter experts to help identify bias in data and models, West Monroe's Greenstein said.

Scott Zoldi, chief analytics officer at credit scoring services company FICO, identified two bias mechanisms that enterprises often overlook. First, most GenAI use involves prompting or fine-tuning a pre-existing LLM with training data that might not be sufficiently representative or balanced. Practitioners have no reliable way to assess those biases without access to the underlying data. Second, prompt engineering itself is a form of cognitive bias, shaping and constraining results in ways that reflect the practitioner's own assumptions. Zoldi recommended businesses build small language models on curated, auditable data sets rather than relying on prebuilt models with unknown provenance.

9. Workforce roles and morale

AI is being trained to do more daily tasks that knowledge workers do, including writing, coding, content creation, summarization and analysis, Greenstein said. Although worker displacement and replacement have been ongoing since the first AI and automation tools were deployed, the pace has accelerated with innovations in generative AI. "The future of work itself is changing," Greenstein added, "and the most ethical companies are investing in this [change]."

Ethical responses have included investments in preparing certain parts of the workforce for the new roles created by GenAI applications. Businesses will need to help employees develop skills such as prompt engineering. "The truly existential ethical challenge for adoption of generative AI is its effect on organizational design, work and ultimately on individual workers," said Nick Kramer, principal of AI and applied solutions at consultancy SSA & Company. "This will not only minimize the negative impacts, but it will also prepare the companies for growth."

In industrial environments, the accountability stakes are especially high. AI is increasingly operating inside mission-critical systems where decisions have real-world consequences, from keeping energy grids running to maintaining food supply chains, said Somya Kapoor, CEO of IFS Loops, an industrial agentic AI platform. Ethical AI in the enterprise isn't just about preventing harm, but it's also about ensuring responsible execution in the systems that keep industries running, she said.

10. Data provenance

GenAI systems consume tremendous volumes of data that could be inadequately governed, of questionable origin and used without consent or bias. Social influencers or the AI systems themselves can amplify inaccuracy to additional levels.

"The accuracy of a generative AI system depends on the corpus of data it uses and its provenance," Zoldi said. AI vendors and companies are mining the internet without understanding its provenance, which can present accuracy problems.

FICO, according to Zoldi, has been using generative AI for more than a decade to simulate edge cases in training fraud detection algorithms. The generated data is always labeled as synthetic, he said, so his team knows where it can be used. "We treat it as walled-off data for the purposes of test and simulation only," he said. "Synthetic data produced by generative AI doesn't inform the model going forward in the future. We contain this generative asset and don't allow it 'out in the wild.'"

11. Lack of explainability and interpretability

Many generative AI systems group facts together probabilistically, mirroring how AI has learned to associate data elements, Zoldi said. But these details aren't always revealed when using applications like ChatGPT. Consequently, data trustworthiness is called into question.

When interrogating GenAI, analysts expect to arrive at a causal explanation for outcomes. But machine learning models and generative AI search for correlations, not causality. "That's where we humans need to insist on model interpretability -- the reason why the model gave the answer it did," Zoldi said. "And truly understand if an answer is a plausible explanation versus taking the outcome at face value."

Until that level of trustworthiness can be achieved, GenAI systems shouldn't be relied on to provide answers that could significantly affect lives and livelihoods.

12. Process debt

The lack of explainability feeds what AI ethics researcher Watson calls process debt: What happens when organizations choose dependency over comprehension. When nobody understands how core processes work without AI mediation, the ability to audit, recover or adapt is lost. The solution isn't less AI but AI designed for bilateral comprehensibility, she said, where both the human and the AI system can account for decisions in terms the other can verify.

Watson's research has shown that standard safety alignment via reinforcement learning from human feedback creates a thin surface coating on a model's behavior, one that can be easily stripped. An emerging approach called bilateral alignment learns a model's representation to guide it toward a better understanding of the world that holds up against malicious prompts, such as those that turned Microsoft's Tay chatbot racist and sexist in 2016, she said. It's the difference between surface-level paint that can be easily scraped off and color dyed into fabric, which is more difficult to get out.

13. AI hallucinations

Generative AI techniques all use various combinations of algorithms, including autoregressive models, autoencoders and other machine learning algorithms, to distill patterns and generate content. As good as these models are at identifying new patterns, they sometimes struggle with teasing out important distinctions relevant to human use cases.

This can include creating authoritative-sounding but inaccurate prose or producing pictures with realistic-looking imagery but misshapen human figures with extra fingers or eyes. With language models, these errors can show up as chatbots that inaccurately represent company policies, such as in the case of an Air Canada chatbot that misrepresented bereavement benefit policies. Lawyers using these tools have also been fined for filing briefs that cited nonexistent court cases.

Newer techniques, such as retrieval-augmented generation and agentic AI frameworks, can reduce these issues. However, it's important to keep humans in the loop to verify the accuracy of GenAI information to avoid customer backlash, sanctions or other problems.

14. Environmental and electricity costs

The environmental, energy and water costs of GenAI have grown significantly since the first wave of large model deployments. Training and running large AI models require enormous data center resources, driving up energy consumption, water use for cooling and emissions. Communities near data centers are increasingly feeling these effects and raising concerns. Employees of AI vendors have also identified instances where their employers have failed to address adverse effects on local communities. Improving an AI model to reduce these costs could be a net positive.

15. Conflicting ethics frameworks

The absence of a universal ethical framework for AI is a governance challenge. In the U.S. alone, there's no single coherent regulatory regime for AI: Federal baseline guidance and state-level legislation vary in scope and assumptions, S-RM's Mulligan said. A company operating across jurisdictions must comply with multiple rules and navigate frameworks built on contradictory philosophies. What companies need, Mulligan advised, is a tiered approach to governance that lets them make defensible decisions about which risks to accept and which to mitigate.

16. Political impact

The political impact of GenAI technologies is a fraught topic. On one hand, better tools have the potential to make the world a better place. At the same time, they could also enable various political actors -- voters, politicians and authoritarians -- to make communities worse. Social media platforms are an example of generative AI's negative effect on politics. They algorithmically promote or create divisive comments as a strategy to increase engagement and profits for their owners, rather than comments that find common ground but might not have the same click-through and sharing numbers.

These issues will remain thorny for years to come as societies determine which GenAI use cases serve the public good and whether that should be the end goal.

George Lawton is a journalist based in London. Over the last 30 years, he has written more than 3,000 stories about computers, communications, knowledge management, business, health and other areas that interest him.