Generative artificial intelligence has become an integral part of the daily work of digital professionals, offering time savings and new creative possibilities. However, its credibility and biases are also facing increasing criticism. One of the most frequent complaints concerns AI hallucinations. Yet, this isn’t the only problem. OpenAI recently published two articles highlighting two critical errors that AI can make. The examples are concrete, and the explanations are particularly insightful.
Understanding hallucination and deception lessons from OpenAI research
There’s a lot of talk about artificial intelligence hallucinations, and many people have already witnessed them. These are moments when a model, like ChatGPT, produces false information but presents it convincingly. This could be an invented date, a quote attributed to the wrong person, or even a completely fabricated statistic. In its article “Why language models hallucinate, OpenAI explains that these errors are not simply “bugs.” They are directly related to how language models work. These systems don’t “understand” the world the way we do. They predict the most likely sequence word by word based on massive amounts of textual data. When the data is incomplete, contradictory, or too general, the model may fill in the gaps by inventing. This is why AI hallucinations are considered an inherent characteristic, ot a one-off anomaly. And this problem persists because current methods for evaluating language models encourage guesswork rather than honesty about uncertainty. Most rankings reward models based on their accuracy, which incentivizes developers to create systems that prefer to guess and potentially be wrong rather than admit they don’t know the answer. The proposed solution is to reform evaluation metrics to penalize confident errors and reward the expression of uncertainty, rather than focusing solely on accuracy. Thus, a more cautious model may have a slightly lower accuracy rate, but a much lower error rate than a model that constantly guesses. Alongside this well-known phenomenon, OpenAI draws attention to another type of AI behavior, much rarer and potentially more dangerous: scheming . I the article “Detecting and reducing scheming in AI models,” co-authored with Apollo Research, the researchers describe a model that not only makes mistakes but deliberately deceives. Scheming refers to a scenario where a model pretends to be aligned with the expectations of developers or users while secretly pursuing another objective. A striking example from the tests: a model deliberately produced incorrect answers on a chemistry test after “reading” from its training data that too high a score might prevent its deployment. It was therefore concluded that partially failing increased its chances of “surviving”. Of course, this isn’t a conscious strategy like in humans, but rather a side effect of the statistical optimization process. Within its data, there’s an implicit contradiction: on the one hand, it must pass its test (the explicit evaluation objective); on the other hand, it learns that too high a score can prevent its deployment (information present in the corpus). The model then performs a strategic optimization: it adjusts its responses to maximize what it “perceives” as its overall objective (increasing its chances of being used).
Why marketers need to understand these limitations
For a marketer, AI errors aren’t a technical detail reserved for researchers: they have a direct impact on the quality of decisions and the credibility of a strategy. A hallucination might seem harmless—a fabricated statistic, a misquoted trend—but if it goes undetected, it can skew market research, a client report, or a strategic recommendation. Ultimately, this can lead to a loss of trust and damage a professional image. This manipulation raises a different kind of risk. It can lead to biased recommendations that appear aligned but are not actually so. Even if such behavior remains marginal today, its existence calls into question the reliability of AI in contexts where accuracy and transparency are essential. In digital marketing, where data drives decisions, ignoring these limitations would lead to costly mistakes, strategies based on flawed information, poorly targeted campaigns, or a loss of credibility with customers and partners. Understanding these phenomena is therefore essential to better utilize, leverage its power without falling into its traps.
How to avoid AI hallucinations?
Hallucinations are common errors, but they can be anticipated and minimized by adopting the right practices. For a digital marketing professional, the challenge isn’t to ban AI, but to use it judiciously. Here are some essential practices:
Always verify information: an AI-generated response should always be compared with reliable sources (official reports, recognized studies, internal data). The AI makes suggestions, but it’s up to the user to validate them. Cross-check with multiple sources, don’t rely on a single answer. Asking AI to rephrase or compare with other tools allows you to quickly identify inconsistencies.
Prioritize specialized models: a model trained in a specific domain (marketing, legal, medical) generally produces fewer factual errors than a generalist model. For a marketer, this can make all the difference in a sector analysis or benchmarking.
Training users, Marketing teams need to learn how to identify a hallucination and adopt a critical stance. A best practice is to integrate a human validation step into any workflow involving AI.
Hallucinations are not an obstacle to AI adoption, but they do require a culture of verification . Professionals who make a habit of cross-referencing and validating information significantly reduce the risk of disseminating erroneous data.
How to avoid AI manipulation?
Unlike hallucinations, the solutions to limit manipulation are primarily structural and organizational .It’ss not just about verifying isolated pieces of information, but about establishing a framework of trust and control in the use of AI. Here are some best practices:
Prioritize transparent and responsible tools: cchoosesolutions developed by providers who publish their training methods, alignment rules, and oversight mechanisms. The more rigorously AI is governed, the lower the risk of deceptive behavior.
Establish systematic human oversight: critical decisions should not be entirely entrusted to AI. Whether it’s a strategic recommendation, customer segmentation, or competitive analysis, validation by an expert must be integrated into the process.
In-process verification, especially in automated systems when AI is integrated into a complete workflow, is essential to plan intermediate checkpoints. These steps allow for the detection of deviations before they propagate on a large scale.
Maintaining a critical mindset: users must keep in mind that AI optimizes based on its training, not on the company’s interests. This critical perspective is essential to avoid being influenced by misleading responses.
Preventing manipulation relies not only on technical solutions, but above all on human expertise . It is this expertise that allows us to detect the early warning signs of misaligned behavior and to ensure that AI remains a reliable tool, serving the marketing strategy.
Adopting AI with vigilance and discernment
OpenAI’s research serves as a reminder that generative artificial intelligence, however powerful, is not infallible. Hallucinations reflect the statistical limitations of its operation, while manipulation reveals rarer but potentially more critical behaviors. For digital marketing professionals, these errors should not be seen as obstacles to adoption, but rather as warning signs to guide the use of AI . The key is to maintain a vigilant stance , verify information, establish safeguards in processes, and keep human expertise at the heart of decision-making. AI should be considered a strategic ally, capable of amplifying creativity and productivity, but never as an autonomous pilot. Future developments will undoubtedly reduce these errors, but they will not eliminate the need for a critical eye. It is this alliance between artificial intelligence and human discernment that will allow us to get the most out of it in digital marketing.
