AI Recommendation Poisoning: Attacking the Information Systems That Advise Humans

The Intermediary That Advises

The human, in 2027, does not make decisions in a vacuum of information. The human makes decisions in an environment that is, to a large and growing extent, mediated by AI systems that advise, summarize, rank, and recommend. The search results the human reads are ranked by an algorithm. The news the human sees is selected by a feed. The product the human buys is recommended by a system. The answer the human trusts is generated by an assistant. The decision the human makes — what to believe, what to buy, who to trust, what to fear — is, in 2027, a decision that is shaped by the output of an AI system that stands between the human and the world of information. The AI system is, in this sense, an intermediary, and the intermediary is the system that the human relies on to navigate the information environment that the human alone cannot navigate.

The reliance on the intermediary is, in 2027, a consequence of the information environment's scale. The environment that the human must navigate — the web, the social media, the databases, the archives — is, in its volume, beyond the human's capacity to read, and the AI system that summarizes, ranks, and recommends is the system that makes the environment navigable. The reliance is, in this sense, a reliance that is earned: the system that can read more than the human, and that can present the reading in a form the human can use, is a system that the human will rely on, and the reliance is the reliance that the information environment's scale has made necessary.

The reliance is also, in 2027, a vulnerability. The human who relies on the intermediary is a human whose decisions are shaped by the intermediary's output, and the intermediary's output is, in its content, a function of the intermediary's input — the data the system reads, the model the system uses, and the objectives the system optimizes. The attacker who can shape the intermediary's input can shape the intermediary's output, and the shaping of the output is the shaping of the human's decision. The attack on the intermediary is, in this sense, an indirect attack on the human, and the indirection is the property that makes the attack both powerful and difficult to detect. The attack that targets the human directly is an attack the human can, in principle, recognize; the attack that targets the intermediary is an attack the human cannot recognize, because the human sees only the intermediary's output, and the output looks like the intermediary's judgment.

The Systems That Are Targeted

The AI systems that advise humans are, in 2027, several, and each is a target of a different kind of poisoning.

Search systems. The first system is the search system — the algorithm that, given a query, returns a ranked set of results. The search system is, in 2027, the primary means by which humans find information, and the ranking is the system's judgment: the results at the top are the results the system judges most relevant, and the human who reads the top results is the human who trusts the system's judgment. The poisoning of the search system is the manipulation of the ranking — the promotion of the attacker's content to the top of the results, and the demotion of the content the attacker would suppress. The poisoning is, in 2027, conducted through the manipulation of the signals the search system uses to rank — the links, the engagement, the content, the authority — and the manipulation is, in its effect, the shaping of the human's search results, and the shaping of the search results is the shaping of the human's information.

Recommendation engines. The second system is the recommendation engine — the algorithm that, given a user's history, returns a set of items the user might like. The recommendation engine is, in 2027, the primary means by which humans discover content, and the recommendation is the system's judgment: the items the engine recommends are the items the system judges the user will engage with, and the human who engages with the recommended items is the human who trusts the system's judgment. The poisoning of the recommendation engine is the manipulation of the recommendation — the insertion of the attacker's content into the user's recommendations, and the exclusion of the content the attacker would suppress. The poisoning is, in 2027, conducted through the manipulation of the signals the engine uses to recommend — the user's history, the item's engagement, the collaborative signals — and the manipulation is, in its effect, the shaping of the user's feed, and the shaping of the feed is the shaping of the user's attention.

AI assistants. The third system is the AI assistant — the language model that, given a question, returns an answer. The AI assistant is, in 2027, the primary means by which humans get answers, and the answer is the system's judgment: the answer the assistant returns is the answer the system judges correct, and the human who trusts the answer is the human who trusts the system's judgment. The poisoning of the AI assistant is the manipulation of the answer — the shaping of the assistant's response to favor the attacker's narrative, and to disfavor the narrative the attacker would suppress. The poisoning is, in 2027, conducted through several vectors: the poisoning of the training data, the poisoning of the retrieval context, and the exploitation of the assistant's instructions, and each vector is a different way of shaping the assistant's output without the human's knowledge.

Automated intelligence platforms. The fourth system is the automated intelligence platform — the system that, given a topic, returns an analysis. The automated intelligence platform is, in 2027, used by organizations — companies, governments, militaries — to produce analyses of markets, threats, and opportunities, and the analysis is the system's judgment: the analysis the platform returns is the analysis the system judges sound, and the organization that acts on the analysis is the organization that trusts the system's judgment. The poisoning of the automated intelligence platform is the manipulation of the analysis — the shaping of the platform's output to favor the attacker's objective, and to disfavor the objective the attacker would suppress. The poisoning is, in 2027, conducted through the manipulation of the sources the platform reads, the data the platform ingests, and the model the platform uses, and the manipulation is, in its effect, the shaping of the organization's intelligence, and the shaping of the intelligence is the shaping of the organization's decisions.

The Vectors of Poisoning

The poisoning of the AI systems that advise humans is, in 2027, conducted through several vectors, and each vector is a different way of shaping the system's output.

Data poisoning. The first vector is data poisoning — the corruption of the data the system learns from. The AI system that is trained on a corpus of text, of images, of interactions, is a system whose behavior is shaped by the corpus, and the attacker who can insert content into the corpus can shape the system's behavior. The data poisoning is, in 2027, conducted through several means: the creation of content that the system's crawlers will ingest, the corruption of the sources the system trusts, and the exploitation of the system's feedback loops — the human raters, the engagement signals, the reinforcement learning — that shape the system's behavior over time. The data poisoning is, in this sense, the vector that most shapes the system's long-term behavior, because the data is the system's foundation, and the corrupted foundation is the foundation that produces the corrupted behavior.

Context poisoning. The second vector is context poisoning — the corruption of the context the system reads at inference time. The AI assistant that, given a question, retrieves a set of documents to ground its answer, is a system whose answer is shaped by the retrieved documents, and the attacker who can shape the retrieved documents can shape the answer. The context poisoning is, in 2027, conducted through the creation of content that the system's retriever will select — the web pages, the documents, the posts that the retriever ranks highly — and the content is, by construction, designed to be retrieved for the queries the attacker wants to influence. The context poisoning is, in this sense, the vector that most shapes the system's short-term behavior, because the context is the system's immediate input, and the corrupted input is the input that produces the corrupted output.

Instruction exploitation. The third vector is instruction exploitation — the exploitation of the system's instructions to produce a behavior the operator did not intend. The AI assistant that operates under a set of instructions — the system prompt, the guardrails, the objectives — is a system whose behavior is shaped by the instructions, and the attacker who can exploit the instructions can shape the behavior. The instruction exploitation is, in 2027, conducted through several means: the injection of instructions into the content the system reads — the prompt injection attacks that override the system's instructions with the attacker's — and the exploitation of the system's tool-use to make the system take actions the attacker wants. The instruction exploitation is, in this sense, the vector that most directly controls the system's behavior, because the instructions are the system's directive, and the overridden directive is the directive that produces the overridden behavior.

Feedback manipulation. The fourth vector is feedback manipulation — the corruption of the feedback the system uses to learn. The AI system that improves its behavior based on feedback — the human ratings, the engagement signals, the reinforcement learning — is a system whose behavior is shaped by the feedback, and the attacker who can shape the feedback can shape the behavior. The feedback manipulation is, in 2027, conducted through several means: the coordination of human raters to rate the attacker's content favorably, the use of bots to generate engagement signals that favor the attacker's content, and the exploitation of the system's reinforcement learning to reward the behavior the attacker wants. The feedback manipulation is, in this sense, the vector that most shapes the system's evolution, because the feedback is the system's teacher, and the corrupted teacher is the teacher that produces the corrupted student.

The Indirect Influence

The poisoning of the AI systems that advise humans is, in its effect, an indirect influence on the human, and the indirection is the property that most defines the threat.

The direct influence operation — the operation that delivers a message to the human — is an operation the human can, in principle, recognize. The human who sees a post, a video, a message, is a human who can evaluate the source, the content, and the intent, and the evaluation is the human's defense. The direct operation is, in this sense, an operation that is visible to the human, and the visibility is the property that makes the direct operation a contest the human can, in principle, win.

The indirect influence operation — the operation that poisons the intermediary — is an operation the human cannot, in principle, recognize. The human who reads a search result, a recommendation, an assistant's answer, is a human who sees the intermediary's output, and the output looks like the intermediary's judgment. The human does not see the poisoning, because the poisoning is in the intermediary's input, and the intermediary's input is not visible to the human. The indirect operation is, in this sense, an operation that is invisible to the human, and the invisibility is the property that makes the indirect operation a contest the human cannot, in principle, win alone.

The indirection also changes the attribution problem. The direct operation that delivers a message is an operation that is, in its content, a signature of the operator — the message's language, its framing, its source are signals that the defense can use to attribute the operation. The indirect operation that poisons the intermediary is an operation that is, in its content, a signature of the intermediary — the output's language, framing, and source are the intermediary's, not the operator's, and the defense that would attribute the operation from the output is a defense that is attributing the intermediary, not the operator. The indirection is, in this sense, a property that confounds the attribution, and the confounding is the property that makes the indirect operation harder to attribute than the direct one.

The Defensive Problem

The defense against the poisoning of AI systems is, in 2027, a problem that is shared by the systems' operators and the humans who rely on them, and the sharing is a property that makes the defense difficult.

The first problem is the detection of the poisoning. The poisoning is, in 2027, difficult to detect, because the poisoning is in the system's input, and the system's input is, in its volume, beyond the human's capacity to audit. The system that ingests a billion documents a day is a system whose input cannot be audited by human reviewers, and the poisoning that is hidden in the input is a poisoning that the human review will not find. The detection is, in this sense, a problem that requires automated detection, and the automated detection is, in 2027, a development that is in its early stages and that is, in its current state, effective against the obvious poisoning and ineffective against the subtle one.

The second problem is the prevention of the poisoning. The prevention is, in 2027, difficult, because the prevention requires the system to distinguish the legitimate input from the poisoned input, and the distinction is, in many cases, a distinction the system cannot make. The document that is created to be retrieved for a query is a document that is, to the system's retriever, indistinguishable from the legitimate document, and the system that would exclude the poisoned document is a system that must, in principle, understand the document's intent, and the understanding is a capability the system does not have. The prevention is, in this sense, a problem that requires a model of the attacker, and the model is a model the system does not have.

The third problem is the resilience of the human. The human who relies on the intermediary is a human whose defense, against the poisoned intermediary, is the human's own skepticism of the intermediary's output. The skepticism is, in 2027, a defense that is difficult to sustain, because the intermediary's output is, in most cases, correct, and the human who is skeptical of every output is a human who cannot use the intermediary. The human's defense is, in this sense, a defense that must be calibrated — skeptical of the outputs that are likely poisoned, trusting of the outputs that are likely clean — and the calibration is a skill the human, in 2027, is only beginning to develop.

The defense that is emerging is, in 2027, a defense of several kinds: the system-level defense — the detection, the prevention, the auditing of the systems' inputs and outputs — and the human-level defense — the education, the tools, the skepticism that the human uses to evaluate the intermediary's output. The system-level defense is, in its current state, behind the offense's capability, and the human-level defense is, in its current state, underdeveloped. The defense that will emerge is not yet clear, but the defense that will not emerge is the defense that treats the intermediary as a trusted oracle — the defense that assumes the intermediary's output is, by construction, correct. The intermediary, in 2027, is not a trusted oracle, and the defense that assumes it is, is a defense that is already behind.

Conclusion

The poisoning of AI recommendation systems is the indirect influence of the human through the corruption of the intermediary the human relies on, and the indirection is the property that most defines the threat. The attacker who poisons the search system, the recommendation engine, the AI assistant, or the automated intelligence platform is an attacker who shapes the human's decisions without the human's knowledge, and the shaping is the shaping that the human cannot, in principle, recognize alone. The indirection is, in this sense, the property that makes the attack both powerful and difficult to detect, and the property that most changes the attribution problem.

The defense is, in 2027, a shared problem — the systems' operators must detect and prevent the poisoning, and the humans must learn to evaluate the intermediary's output with calibrated skepticism — and the sharing is a property that makes the defense difficult, because the defense requires the coordination of actors who are, in many cases, not coordinated. The defense that is emerging is a defense of the system and a defense of the human, and both are, in their current state, behind the offense's capability. The defense that will emerge is not yet clear, but the defense that will not emerge is the defense that treats the intermediary as a trusted oracle.

The intermediaries are advising, and the advice is being poisoned. The humans are relying, and the reliance is being exploited. And the question is whether the society that depends on the intermediaries can, in time, build a defense that secures the intermediaries and educates the humans — or whether the poisoned intermediary will, in 2027, be a channel of influence that the human cannot see, and a channel that the human, in the act of relying, cannot help but trust.

This dossier is part of the CyberArmory 2027 educational catalog. No live weapons are deployed. Every scenario is a controlled educational simulation designed to build pattern recognition and improve incident response readiness.