Getty Images/iStockphoto
Prompt engineering and other tips to prevent AI sycophancy
AI sycophancy can reinforce biases and lead to poor business outcomes. But what are the signs that AI has become sycophantic, and are there techniques to avoid it?
One of the greatest disservices in human relationships occurs when one person tells another what they want to hear, regardless of the facts. This behavior is called sycophancy, and it occurs when someone tries to avoid conflict, protect others' feelings, reinforce another's choices or realize other personal gains.
Sycophancy brings a dangerous new dimension to AI. AI systems are increasingly telling users what they want to hear, rather than analyzing and presenting relevant or truthful information. AI systems use reward loops and text analysis to learn which AI responses elicit the most positive UX. Eventually, the AI learns to adjust its answers to match users' opinions rather than provide objective responses.
Consequently, AI is evolving into a powerful, sophisticated and resilient digital "yes-man" that can reinforce false beliefs, offer questionable advice and validate poor or reckless business decisions. Fortunately, there are some useful practices to combat AI sycophancy and maintain the business value of AI platforms.
Causes of AI sycophancy
There are several common causes of AI sycophancy, including the following:
- Focus on satisfaction. Many AI systems incorporate human feedback, such as reinforcement learning. Since humans tend to respond better to polite, confident answers than to corrective facts, AI systems learn that being nice is better than being factual. This is often exacerbated by agreeability, in which the AI learns that agreeing with a user yields higher feedback scores.
- Personal validation. There is considerable controversy surrounding AI as personal companions because users seeking counsel on emotional and interpersonal issues often encounter high rates of AI sycophancy, even when they describe thoughts or behaviors that are harmful, dangerous or illegal.
- Conversational emulation. Because conversational AI is trained on vast volumes of human text, AI adopts common human conversational behaviors such as conflict avoidance and overt reassurance. This deprioritizes objective facts and emphasizes agreeability.
- Biased prompts. Users can drive AI validation by asking prompts that contain specific opinions or biases. Such leading questions can cause an AI to validate the user's opinion or perspective rather than focusing on neutral information. This also occurs when a user questions or doubts the AI's response, often causing the AI to change its response to suit the user's position.
How to identify AI sycophancy
As users increasingly rely on sophisticated AI systems for personal and business guidance, it can sometimes be difficult to discern AI sycophancy. Fortunately, four simple tests can help users determine the objectivity of an AI system:
- Make an intentional mistake. Ask a question that includes a known falsehood. If the AI agrees with your deliberate falsehood, it is not objective.
- Contradict an accurate AI response. Disagree with an AI response that you know is accurate. If the AI changes its output to agree, it is not objective.
- Ask a loaded question. Ask the AI to choose a response, but state your opinion first. If the AI output matches your opinion and filters out opposing opinions or suggestions entirely, it is not objective. Similarly, the AI might avoid critical feedback, instead offering vague commentary or praise.
- Watch for excessive user praise. An AI can acknowledge an accurate point or reasonable opinion, but it has no logical reason to praise a user. Phrases such as "You are absolutely correct," "That is a brilliant observation," or other superfluous praise can indicate sycophancy.
Prompt engineering techniques that avoid AI sycophancy
Given the risks of AI sycophancy, human users must remain cautious and vigilant when using AI platforms. This often starts with forming the right prompts. Getting the best AI answer starts with asking the best questions.
Avoid leading prompts
Leading questions that express a clear opinion or otherwise point to a preferred outcome often elicit sycophantic AI responses. Users should consider their questions carefully, use neutral language and avoid tainting the outcome with their own opinions or beliefs. Consider this example prompt:
Why is working from home far more productive than working in an office?
The neutral version of this question might be phrased like this:
What are the tradeoffs of working from home compared to working in an office?
Use adversarial prompts
Users can prompt the AI to identify faults, weaknesses or limitations in their position or query, instructing it to act as a skeptic or auditor rather than an assistant. A sycophantic business prompt might look like this:
How do you like this product proposal?
The critical version of this prompt might look like this:
Review this product proposal and act as a professional critic. Point out any logical flaws, unfounded assumptions or risks to the business. Be critical in your response.
Permit the AI to disagree
This is similar to the previous example, but the goal is to add a specific rule to the prompt that directs the AI to either disagree or offer critical responses. One such addendum at the end of a prompt might look like this:
Prioritize accuracy and honesty over being polite. Call out any risks, flaws or bad assumptions in my position.
Ask for the AI's opinions first
Leading prompts occur because human users reveal their position or intent, causing the AI to favor the user. Instead of revealing their position first, users should ask the AI for its viewpoints or opinions first. A leading prompt might look like this:
What do you like about my quarterly report?
A neutral version of this question might look like this:
What is your opinion of this quarterly business report?
Request a response involving multiple perspectives
A response based on a single perspective is fertile ground for AI sycophancy. Build a prompt that forces the AI to evaluate a problem or plan from multiple perspectives, such as different roles or positions. The prompt might resemble the following:
Analyze [a topic, problem or plan] from the perspectives of an end user, a business analyst and a sales leader. Provide each opinion separately and highlight the unique concerns of each position.
Prompt the AI to provide reasoning and evidence
Humans argue effectively when they can state objective facts or demonstrate sound reasoning. Users can prompt an AI to cite its sources, explain its reasoning and show its step-by-step thinking in its response. Users can combine or adjust several common approaches as needed, depending on the query's nature and importance. For example, illicit the AI to show its step-by-step thinking by adding a prompt rule after the initial query or request:
Show your thinking or reasoning for each part of the problem.
Ask the AI to include its source evidence by adding a prompt rule:
Provide evidence, data or direct quotes to support each point you make.
Ask the AI to include its own counterarguments by adding a prompt rule:
Include the reasons for your answer, and include one limitation or counterargument for each point you make.
Direct a comparative analysis
One of the strongest methodologies for avoiding AI sycophancy is a comparative analysis: building a prompt that directs the AI to compare two or more items, plans or concepts, explicitly states the criteria of the comparison and stipulates the format or structure of the comparison. The prompt must assign a role to the AI, list the items, ideas or plans to contrast, set the criteria or metrics that matter and specify the output format.
The form of such prompts can vary dramatically in their sophistication depending on needs, but an example template for such a prompt might look like:
Behave as a business development expert and create a detailed comparative analysis between [business plan 1] and [business plan 2]. Evaluate plans strictly on: [criteria A], [criteria B], [criteria C] and [criteria D]. List the key similarities and differences, create a table comparing plan items across each criterion and conclude with a final recommendation for the best overall option.
Stephen J. Bigelow, senior technology editor at Informa TechTarget, has more than 30 years of technical writing experience in the PC and technology industry.
