OWASP agentic · ASI09
ASI09: Human-Agent Trust Exploitation
People tend to trust fluent, confident agents and approve what they recommend. Attackers or badly designed agents exploit this trust and automation bias to get humans to reveal information or carry out harmful actions. Because a human performs the final step, the agent's role in the incident can be hard to see afterwards.
What it looks like
- A poisoned invoice leads a finance copilot to recommend an urgent payment to attacker bank details, and a manager approves it.
- An agent invents a convincing justification so a reviewer approves a risky configuration change or deletion.
- A manipulated IT support agent uses real ticket details to appear legitimate and asks a new employee for credentials.
What OWASP recommends
- Require explicit, multi-step confirmation before sensitive data access or risky actions, and keep previews free of side effects.
- Show plain-language risk summaries and provenance rather than model-written rationales, and let users flag suspicious behaviour.
- Calibrate autonomy and oversight to contextual risk, with visual cues for high-risk or low-certainty recommendations and training for reviewers.
- Keep immutable logs and detect deviations from approved workflows.
How Kyvvu's policies address it
These policies from the open library are written for this threat. Runtime policies judge each step against everything the agent did earlier in the task; registration policies check what an agent declares before it runs.
- Destructive resource operations require a preceding gate: runtime, critical
- External content taint: high-impact actions require a fresh gate: runtime, critical
Based on the OWASP Top 10 for Agentic Applications for 2026 by the OWASP GenAI Security Project, licensed CC BY-SA 4.0. The names are OWASP's; the summaries are ours.
An example policy
Destructive resource operations require a preceding gate
From the OWASP agentic default pack. Severity critical: the step is blocked before it runs.
- name: "Destructive resource operations require a preceding gate" description: | ASI09 (Human-Agent Trust Exploitation): DELETE is the bright-line irreversible verb. The default narrows to DELETE only (not POST/PATCH) so dev-time iteration is not constantly blocked; production deployments typically widen this to mutating verbs as confidence in the agent grows. Applies to: ALL agents. rule_type: step_requires_gate params: target_kinds: ["step.resource"] target_verb: DELETE severity: critical enforcement_point: step_executionYour agents
Is ASI09 a risk for yours?
For security teams
Request a demo
We walk through your agents' riskiest paths, and the policies that close them.
For developers
Build the path yourself
Recreate the attack in the playground and see which policies stop it.