ChatGPT agent tests website conversion journeys
Scholz reports more than one hundred ChatGPT agent tests and 250 website visits, exposing browsing and conversion failures including a mailto button on her own site.
searchengineland.com
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Visit my site and attempt the defined conversion journey. Record the actual pages and controls used, where progress fails, and what requires my login or approval. Stop before final purchase or submission and preserve evidence for my review.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source author
- Jes Scholz
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- Scholz reports more than one hundred ChatGPT agent tests and 250 website visits, exposing browsing and conversion failures including a mailto button on her own site.
- Human involvement
- Scholz designs and analyses the tests; humans intervene at login and conversion decisions. Findings describe the 2025 sample rather than a universal current capability.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Define concrete search, reservation or conversion tasks for the agent.
- Observe its page visits and whether it uses reading or visual browsing.
- Record blocked paths, filter failures and conversion handoffs.
Jes Scholz uses ChatGPT agent mode for an empirical investigation of online journeys, including shopping, reservations and searches for housing and jobs. She observes more than one hundred agentic actions and over 250 website visits rather than assuming an agent experiences websites exactly as a person does.
Her own site supplies a concrete failure: its email-link conversion path cannot be completed in the tested agent environment. Across the observed journeys, blocked pages, redirects, poor filters, overlays and registration placement also interrupt progress. The account records a seventeen-percent conversion rate for this sample, not a current universal rate.
Scholz distinguishes text-oriented reading from visual browsing. This matters when inspecting what content and controls the agent actually sees, and when interpreting analytics. A successful page request does not by itself show that a human’s intended conversion reached completion.
The agent performs the searches and page interactions; Scholz designs the prompts and analyses the results. Humans still take over at important login and conversion decisions. The account establishes a completed investigation and a discovered flaw, not proof that she subsequently repaired every website or that historical product limitations remain unchanged. A useful adaptation is to define one conversion goal, observe the actual path and record the failure point separately from the final user approval step.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Blog · Jes Scholz
Frequently asked questions
What failed on the author’s own site?
Its mailto conversion button could not be completed in the agent environment tested at the time.
Does a page request prove conversion?
No. The account separates page access, usability and actually reaching the intended conversion path.
Is seventeen percent a universal success rate?
No. It is the reported conversion rate from the author’s historical sample of more than 250 visits.
More details
Remaining steps
- Test the site’s actual form or email-link endpoint.
- Interpret request logs alongside completed journeys and human decision gates.
About this account
ChatGPT agent searches, compares options and navigates website tasks, providing observable journeys whose successes and failures the researcher records.
Firsthand account by Jes Scholz; original publication date verified against Original Search Engine Land publication October 8, 2025 at 9:00 am. Outcomes are the author’s reports, not independently reproduced. Suggested prompt written by Agents In Use; not the author’s original prompt.
Credit: Jes Scholz
Source published
Added to Agents In Use
Source checked
Tools and topics
ChatGPT agent mode · Website conversion paths · Browser observations · Request logs
Empirical conversion tests · Agent journey failures · Reading-mode observations
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.