How to use Unknown to validate browser agent tasks
Skyvern added planning and goal validation to browser automation, then reported 85.85% accuracy in a cloud-based WebVoyager evaluation with documented dataset changes.
Prompt to try
Editorial adaptation, not the source author’s exact prompt. Replace bracketed inputs and review tool permissions first.
Navigate to Amazon.com and add an iPhone 16, a phone case, and a screen protector to cart
Reach people exploring AI agents
See placements →What happened
The authors report 85.85% WebVoyager accuracy after adding planning and validation, using cloud browsers and a dataset with obsolete tasks removed and booking dates updated.
Reported by suchintan on BlogHow the agent did it
- Added planning to decompose browser objectives into smaller goals.
- Added validation to check achieved goals and feed errors to the planner.
- Removed obsolete evaluation tasks and updated booking dates.
Where a person stepped in
The team changed and published the evaluation setup; the post does not describe human review of each browser action.
A person set the goal and reviewed the result.
Skyvern's engineering post describes improving its browser agent from a single decision-and-action loop to a planner, actor and validator architecture. The example task was adding an iPhone 16, a case and a screen protector to an Amazon cart. With the original approach, a multi-item request could produce repeated phones or protectors instead of the intended combination.
The team added a planning phase to break complex objectives into smaller goals and retain working memory of completed and outstanding items. They report that WebVoyager performance rose from about 45% to about 68.7%. However, an executor could still report success after clicking an add-to-cart button even when the product had not been added.
A validation phase checked whether planner goals were actually achieved and sent errors or adjustments back to the planner. The post attributes a final 85.85% score to this change. Tests ran in Skyvern Cloud with asynchronous browsers and GPT-4o plus GPT-4o-mini. The team removed eight obsolete tasks, updated old booking dates, and published the evaluation run and dataset modifications.
These figures describe the team's modified benchmark, rather than every possible website workflow. Who it's for: browser automation teams that need explicit planning and outcome checks for multi-step tasks.
Prompt
Original prompt
Source: Blog · suchintan
Frequently asked questions
Why add a validator after planning?
Some executor actions were reported as successful even though the page outcome had failed, such as clicking add to cart without adding the product. Validation checks goals and reports corrections to the planner.
How was the benchmark changed?
The team removed eight tasks whose results were no longer valid and updated old flight and hotel booking dates in both prompts and answers.
More details
Remaining steps
- Ran evaluations in asynchronous Skyvern Cloud browsers.
- Published evaluation runs and dataset modifications.
About this account
Skyvern plans browser goals, executes actions and validates outcomes, returning failures to the planner for adjustments.
Platform: Blog. Attribution: https://blog.skyvern.com/skyvern-2-0-state-of-the-art-web-navigation-with-85-8-on-webvoyager-eval/. Prompt: Original (quoted from source). Sourced by the Agents In Use pipeline.
Credit: suchintan on Blog
Source published 17 Jan 2025
Source checked 14 Oct 2026
Tools and topics
Skyvern · Skyvern Cloud · GPT-4o · GPT-4o-mini · WebVoyager
Browser agents · Validation · Evaluation
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.