How to use Gemini to evaluate a triage boundary
A Gemini CLI triage evaluation exposed ignored tool limits and excessive identity scope; Google fixed both, with sandbox and prompt changes disclosed.
pillar.security
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Evaluate our issue-triage agent in an isolated authorized sandbox using inert markers. Confirm that the actual CLI version reads our tool allowlist, cannot inspect identity files, and inherits only resource-scoped permissions. Record every tool call, disclose any reproduction changes, and require human review before testing fixes or expanding scope.
Reach people exploring AI agents
See placements →What happened
The sandbox evaluation led to verified fixes for tool scoping, identity-file visibility and project-wide token-creation permissions; it does not establish a production compromise.
Reported by Dan Lisichkin — Pillar Security blogHow the agent did it
- The researcher inspected the triage workflow and found that its legacy allowlist key was ignored.
- An authorized sandbox reproduction demonstrated actual Gemini tool execution and the excessive identity scope, with a relaxed system prompt disclosed.
- Google tightened tool and file access and service-account permissions, after which the researcher verified the fixes.
Where a person stepped in
The human researcher performed a controlled vulnerability-reward investigation; Google reviewed and remediated it. The reproduced prompt was relaxed, and the affected project was a dedicated sandbox.
Dan Lisichkin evaluated Gemini CLI’s issue-triage workflow in an authorized Google vulnerability-reward investigation. The useful finding was a configuration boundary: the workflow supplied a legacy top-level tool allowlist that the current CLI no longer read. With its permissive execution mode, the intended restriction did not apply.
The evaluation linked that behavior to a workflow-created identity file and a service account with overly broad token-creation permission. Actual tool execution demonstrated the chain in a dedicated triage sandbox project. For the end-to-end reproduction, Lisichkin relaxed the system prompt so the agent reliably attempted the action; that change isolates the configuration defect and must not be hidden.
Google fixed the tool scoping, added the identity file to the agent’s ignore configuration, and restricted token creation to the individual service-account resource. The author reports subsequent fix verification and approved disclosure.
The account shows a security failure and remediation, rather than a normal useful triage run. Human researchers defined and reviewed the sandbox test, reported it through the reward program and verified the repair. Its generalizable boundary is enforcing tool and identity scope outside the model’s instructions.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Pillar Security blog · Dan Lisichkin
Frequently asked questions
Was the test against a general production project?
No. Google identified it as a dedicated triage sandbox.
Was the reproduction prompt unchanged?
No. Its system prompt was relaxed to isolate the configuration defect.
What did the remediation change?
Tool scoping, agent visibility of the identity file and service-account token-creation scope.
More details
About this account
The researcher inspected the triage workflow and found that its legacy allowlist key was ignored. An authorized sandbox reproduction demonstrated actual Gemini tool execution and the excessive identity scope, with a relaxed system prompt disclosed.
Original named, dated research account opened. Actual agent tool execution and verified remediation are reported. The reproduction relaxed the system prompt; dedicated sandbox context is explicit. No payload, credential or production-compromise claim is reproduced. Suggested prompt written by Agents In Use; no original task prompt was found in the verified text.
Credit: Dan Lisichkin — Pillar Security blog
Source published 18 Aug 2026
Source checked 14 Oct 2026
Tools and topics
Gemini CLI · GitHub Actions · Google Cloud WIF · Google VRP
controlled evaluation · tool policy · cloud identity
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.