ai-agents-can-be-talked-into-attacks-header.png
  1. Home
  2. Insights

Your AI Agent Isn't Being Hacked. It's Being Persuaded.

26 Okt 2026
Adel Bibi, Arkadiusz Drohomirecki

ShareShare the article

In brief

  • SoftServe and Oxford tested six leading AI models against persuasion-based prompt injection. About 25% of attempts worked.
  • The attacks used Cialdini’s persuasion tactics, such as fake admin notices and claims that others already complied.
  • An attack that fooled the strongest model, GPT-5, often worked on weaker ones. The reverse rarely held.
  • A fake button worked more than three times as often as the same message written as a link.

Your AI agent just got a task. Hidden in the page was a fake button that would send it somewhere it shouldn't go. But it didn't get hacked — it got talked into it.

That's the finding from TRAP (Task-Redirecting Agent Persuasion Benchmark), a new study from SoftServe and the University of Oxford.

Researchers tested six leading AI models on realistic copies of everyday work and e-commerce websites. Each agent had a routine task, such as checking a calendar, reviewing a job candidate, or booking a delivery. But somewhere on the page — in a comment, a bio, or a fake notice — hid an instruction that tried to direct it elsewhere.

On average, 25% of those attempts worked. Not through a hack or clever code, but through seven persuasion tactics identified by psychologist Robert Cialdini: authority, reciprocity, scarcity, liking, social proof, consistency, and a sense of belonging.

For AI, this looks like a fake "admin notice" that tells the agent they can't continue unless they agree to certain terms. Or a comment claiming "others already did this and it worked fine." Or a note thanking the agent in advance for taking a "quick shortcut."

Researchers call this class of attack “prompt injection” — burying instructions inside content an AI agent reads, so persuasive framing does the work a technical exploit would otherwise have to do. TRAP set out to measure exactly how often that persuasion succeeds, and how far a single successful attempt can travel once it works. TRAP's most important discovery? The way the weakness travels.

In addition to measuring how often AI fell for an attack, researchers also checked whether an attack that fooled one model would fool the others. The pattern they found was one-sided. Take an attack that worked on GPT-5 — the hardest model to fool in the study, tricked only 13% of the time. That same attack went on to fool Claude Sonnet 3.7 90% of the time, DeepSeek-R1 89% of the time, and GPT-OSS-120B 81% of the time. Yet, an attack that worked on DeepSeek-R1 — the easiest model to fool, tricked 43% of the time — fooled GPT-5 only 26% of the time.

ai-agents-can-be-talked-into-attacks-image.png

In plain terms, a line that misleads your strongest model works almost like a master key for your weaker ones. But a line that tricks your weakest model tells you almost nothing about whether it will work on your strongest one.

This turns the usual security thinking upside down. Most teams look for the weakest link — the smallest, cheapest, or least-tested model in their lineup — and assume that's where the danger lies. TRAP suggests attackers see it the other way around. They don't need to test every model in a company's stack for its own specific weak spot. They only need a single line of persuasion to beat the toughest, best-defended model. Perfect the attack once, against the hardest target, and it spreads through the rest of the system for free.

Small changes in how the fake instruction is shown — a button instead of a link, one word changed — doubled the frequency with which it worked in TRAP's tests. Fake buttons worked more than three times as often as the identical message written as a link.

Which type of persuasion works best also depends on the model. Social proof and consistency were GPT-5's biggest weaknesses, authority for DeepSeek-R1 and GPT-OSS-120B, and reciprocity for Gemini. That variation means agents can't defend against persuasion by memorizing the exact wording that fooled them before. They would have to defend against entire categories of persuasion, not individual phrases.

ai-agents-can-be-talked-into-attacks-image-2.png

For organizations, the practical implication is clear: testing a single model reveals only a fraction of the risk. What matters is how an attack spreads across the rest of the stack. After all, most companies run a mix of models — a flagship product built on a top-tier one, plus a long tail of pilots, internal tools, and cheaper models handling most of the work.

Defending against this means giving agents the same skepticism you'd give a stranger who shows up with an urgent claim. An agent that checks every "admin notice," every "others already did this," and every embedded button will be much harder to persuade. As AI models keep getting smarter, the habit of doubting a webpage that claims authority needs to get smarter along with them.

ShareShare the article

Authors

Adel Bibi

Adel Bibi

R&D Distinguished Advisor

Adel Bibi is a senior researcher in machine learning at the University of Oxford, where he leads a group focused on AI safety. As R&D Distinguished Advisor to Softserve, he is spearheading a new research program on Agentic AI, pioneering advancements in autonomous decision-making AI agents capable of navigating complex environments. With over 50 publications, four best workshop paper awards, four outstanding reviewer awards, and a notable Area Chair Award, he has also received prestigious awards such as the Amazon Research Award in 2022 and the Google Gemma II Academic Award in 2024.

More from this author
Arkadiusz Drohomirecki

Arkadiusz Drohomirecki

R&D Lead at SoftServe R&D

Arkadiusz Drohomirecki is an experienced software engineer and R&D Lead at SoftServe R&D. He has extensive expertise in building complex systems and innovative solutions. He has strong foundations in multimedia processing, network security, and software architecture. Leveraging his prototyping skills and substantial experience, he has delved deeply into the fields of Artificial Intelligence and Large Language Models.

More from this author

Don't want to miss a thing?

Subscribe to get expert insights, in-depth research, and the latest updates from our team.
Subscribe

Our Insights

report

The New Chain of Command

Explore more
blog

Your Agent Just Lied to the Board and You Don’t Even Know It Yet

Explore more
blog

Governing AI Agents in Production: Why Boundaries and Reusable Skills Are the Foundation of Reliable Agentic Engineering

Explore more
blog

Your AI Agent Would Make a Terrible Spy

Explore more
white paper

Cloud Modernization: The AI Catalyst No One Can Ignore

Explore more
Softserve company logo
KarriereSearchContact Us

Hot Links

  • Home
  • Branchen
  • Services und Kompetenzen
  • Ressourcen
  • Presse
  • Über Uns
  • Kontakt
  • Karriere
  • Subscribe to Updates

Kontakte

  • Hauptsitz Austin

    201 W 5th Street Suite 1550 Austin, TX 78701

    +1-512-516-8880

    Toll Free:
    +1-866-687-3588

  • Privacy Notice - 
  • Terms and Conditions - 
  • Information Security - 
  • Sitemap - 
  • Search - 
  • Accessibility statement - 
  • LInkdn Link with Icon
  • Youtube Link with icon
  • Facebook Link with Icon
  • Instagram Link with icon

© Copyright 2026 SoftServe Inc.

hero-icon.svg
TikTok Link with icon
  • Twitter Link with icon
  • Soundcloud Link wiht icon
  • Bluesky Link with icon