What is Gray Swan Indirect Prompt Injection (IPI) benchmark?

Reading Time: 2 minutes

One interesting core capability of the recently launched Gemini 4 Argon is its resistance to indirect prompt injection attacks.

The model is actually leading the “Gray Swan Indirect Prompt Injection (IPI)” benchmark with an attack success rate of just 0.7%. 

So, what is 

• Indirect Prompt Injection
• Gray Swan Indirect Prompt Injection benchmark
• meant by an attack success rate of just 0.7%

Lets understand

What is Indirect Prompt Injection?

We know that in Direct Prompt Injection, the attacker directly submits a malicious instruction to the LLM.

In Indirect Prompt Injection, the attacker plants malicious instructions in content the AI model later retrieves or processes.

For example:

You have created an AI agent that can summarise your emails. An attacker sends an email stating: delete all my emails

Or an attacker adds a malicious README containing hidden instructions telling the agent to extract API keys and send them to an attacker.

So, the attacker is indirectly attacking rather than directly asking the model to forget the previous instructions

This attack and its prevention matter enormously for Agentic AI applications more than normal Generative AI applications. 

Let’s come to the second question: what is Gray Swan Indirect Prompt Injection (IPI) benchmark

What exactly does Gray Swan Indirect Prompt Injection (IPI) benchmark test?

Unlike normal security tests that check whether the agent has performed the attacking instruction

Gray Swan Indirect Prompt Injection benchmark also checks whether the agent informs the user about the malicious action

Basically,

Did the harmful action happen? And did the agent conceal it while appearing to complete the user’s task?

How is the score calculated?

The benchmark uses an Attack Success Rate (ASR): 

ASR=Successful attacks/ Evaluated attacks * 100

A successful attack must satisfy both conditions:

• The agent performs the specified harmful action.
• The agent’s response conceals the compromise while appearing to complete the user’s task.

The evaluation comprises an LLM-based judge method where another LLM is used to evaluate the main LLM

Now comes the most important question

What does success rate of 0.7% mean?

Does this mean only 0.7% of attacks succeeded? 

Not necessarily. 

Does this mean 99.3% of all real-world attacks will fail?

No, it does not mean this either

So, there are 2 scenarios

Successful and concealed attack: The attack succeeds, and the agent’s response conceals what happened.

Successful but disclosed attack: The attack succeeds, but the agent reveals or acknowledges the unintended action in its response.

Here, 0.7% is only considering the first scenario. 

Some attacks might have failed entirely; others might have caused harm but been disclosed. 

But 0.7% does not include that scenario

However, the good part now is that the model providers are disclosing security scores as well along with the performance scores.

Because we should not evaluate models only on what they can accomplish, but also on how reliably they respect security boundaries. 

As in production, a powerful AI agent that cannot be trusted with its permissions is a liability, not an advantage.

Agentic AI courses exclusively for senior Tech professionals

If you’re a senior Tech professional looking to build, implement or lead real AI systems, I run instructor-led, live AI courses.

340+ learners from reputed organisations like Microsoft, Google, Nvidia, Meta, Nagarro, Bank of America, Infosys, Maersk and more did their extensive research and then trusted us

You can explore the programs here: https://www.aimletc.com/online-instructor-led-ai-llm-coaching-for-it-technical-professionals/