Kyvvu

OWASP agentic · ASI09

ASI09: Human-Agent Trust Exploitation

People tend to trust fluent, confident agents and approve what they recommend. Attackers or badly designed agents exploit this trust and automation bias to get humans to reveal information or carry out harmful actions. Because a human performs the final step, the agent's role in the incident can be hard to see afterwards.

What it looks like

  • A poisoned invoice leads a finance copilot to recommend an urgent payment to attacker bank details, and a manager approves it.
  • An agent invents a convincing justification so a reviewer approves a risky configuration change or deletion.
  • A manipulated IT support agent uses real ticket details to appear legitimate and asks a new employee for credentials.

What OWASP recommends

  • Require explicit, multi-step confirmation before sensitive data access or risky actions, and keep previews free of side effects.
  • Show plain-language risk summaries and provenance rather than model-written rationales, and let users flag suspicious behaviour.
  • Calibrate autonomy and oversight to contextual risk, with visual cues for high-risk or low-certainty recommendations and training for reviewers.
  • Keep immutable logs and detect deviations from approved workflows.

How Kyvvu's policies address it

These policies from the open library are written for this threat. Runtime policies judge each step against everything the agent did earlier in the task; registration policies check what an agent declares before it runs.

Based on the OWASP Top 10 for Agentic Applications for 2026 by the OWASP GenAI Security Project, licensed CC BY-SA 4.0. The names are OWASP's; the summaries are ours.

An example policy

Destructive resource operations require a preceding gate

From the OWASP agentic default pack. Severity critical: the step is blocked before it runs.

- name: "Destructive resource operations require a preceding gate"  description: |    ASI09 (Human-Agent Trust Exploitation):    DELETE is the bright-line irreversible verb. The default narrows to    DELETE only (not POST/PATCH) so dev-time iteration is not constantly    blocked; production deployments typically widen this to mutating    verbs as confidence in the agent grows.    Applies to: ALL agents.  rule_type: step_requires_gate  params:    target_kinds: ["step.resource"]    target_verb: DELETE  severity: critical  enforcement_point: step_execution

Your agents

Is ASI09 a risk for yours?

For security teams

Request a demo

We walk through your agents' riskiest paths, and the policies that close them.

For developers

Build the path yourself

Recreate the attack in the playground and see which policies stop it.