Your experimentation platform assigns a variant to every session that meets the targeting rules. It does not ask whether there is a human behind the session. Increasingly, there isn't — Cloudflare reported in 2026 that more than half of all internet traffic is now non-human, with AI training crawlers alone accounting for 52% of crawler requests, up from 22% in spring 2025.
Classic bot filtering catches most of the obvious cases. The category it does not catch is the one growing fastest: agentic browsers like ChatGPT Atlas and Perplexity Comet, which execute real JavaScript, accept cookies, and present a user agent indistinguishable from Chrome. They enter your tests as ordinary visitors, and they quietly distort every number the test produces.
What Agent Traffic Does to a Test
Four distinct effects, in rough order of how much damage they cause.
It dilutes your measured conversion rate. Agents get counted in the denominator and almost never appear in the numerator. If 8% of your traffic is agentic, an experience that converts real humans at 5.0% reports 4.6%. That alone is survivable — it hits both arms roughly equally.
It inflates the sample size you need. This is the expensive one. Adding non-converting sessions to both arms shrinks the observed effect size without adding information. The minimum detectable effect you set your test up to catch is now measured against a diluted baseline, so the test needs meaningfully more traffic to reach the same power. Tests that were already slow get slower — the exact failure mode described in why your A/B tests take too long.
It can break randomization outright. Agent traffic is not guaranteed to distribute evenly across variants. Redirect-based split URL tests are especially vulnerable: an agent following a task may not complete the redirect the way a browser session would, landing unevenly across arms. That produces sample ratio mismatch, which invalidates the test regardless of how significant the result looks.
It contaminates behavioral metrics. Agents scroll programmatically, click precisely, and never hesitate. Scroll depth, time on page, rage clicks, and heatmap data all absorb this. Any test judged on engagement rather than a hard conversion is reading a blend of human and machine behavior.
Why Your Existing Filters Miss It
Most experimentation platforms and analytics tools filter bots using some combination of user agent matching and known-bot IP lists. Both fail here.
ChatGPT Atlas presents a user agent string matching stock Chrome — not similar to Chrome, identical. There is no dedicated robots.txt token for ordinary agentic browsing, so there is no declared identity to match on. And because these agents run through real browser infrastructure, the heuristics built to catch headless scrapers — missing JavaScript execution, implausible viewport dimensions, absent cookie support — do not fire.
The situation is bad enough that OpenAI's own ChatGPT agent relies on cryptographic signatures rather than user agents for identification, precisely because it presents a standard Chrome user agent from hosting-provider IP ranges. If the operator cannot identify it by user agent, neither can you.
The Four Detection Methods, Ranked
| Method | How it works | Reliability |
|---|---|---|
| User agent matching | Match User-Agent against known tokens | Weak — trivially spoofed, misses agentic browsers entirely |
| Reverse DNS | Verify the IP resolves to the operator's domain | Moderate — fails for agents on generic hosting |
| Published IP ranges | Match against operator-published address blocks | Moderate — good for declared crawlers, useless for browser agents |
| HTTP Message Signatures (RFC 9421) | Agent signs requests with keys published at a .well-known URL | Strong — cryptographic, survives spoofing |
HTTP Message Signatures, often called Web Bot Auth, is the only method that actually holds. Agents carry a Signature-Agent header you can verify against published keys. The catch is that it only covers agents that choose to declare themselves — a growing but incomplete set.
For everything else, behavioral heuristics are what remain: implausibly uniform timing between actions, navigation paths that never backtrack, zero mouse movement preceding clicks, and sessions that request a page's content without ever requesting its images.
Filter at Assignment, Not in Analysis
This is the part teams get wrong, and it matters more than which detection method you pick.
The instinct is to run the test normally and exclude agent sessions when analyzing results. Don't. Excluding sessions after they have been assigned to a variant — using a criterion that may itself correlate with which variant they saw — is a documented cause of sample ratio mismatch. You can manufacture exactly the bias you were trying to remove.
The correct order is:
- Identify the session before variant assignment happens.
- Exclude it from the experiment — serve it the control experience, and do not log it as a participant.
- Still count it in traffic reporting, flagged separately, so you can see the trend.
Serving control rather than blocking matters. An agentic browser session represents a real person who delegated a task. Blocking it costs you a customer, not a bot.
Make Agent Share a Guardrail Metric
The most useful thing you can do is not a filter at all — it is a number on the dashboard.
Track agent-identified sessions as a percentage of total sessions, per variant, and treat it as a guardrail metric on every test. Two things become visible immediately:
- A gap in agent share between arms is an early warning of broken randomization, usually before SRM formally triggers.
- A rising baseline over time tells you your historical results are not comparable to your current ones. A conversion rate that "declined" 4% this year may have simply absorbed more agent traffic.
That second point deserves emphasis. If agent traffic grew over the past year and your filtering did not keep pace, your year-over-year conversion trend is measuring bot composition, not customer behavior.
What to Do This Week
- Measure your exposure. Segment sessions with zero mouse movement, or with a data-center ASN, and see what share of traffic they represent. If it is under 1%, this is a note for next year. If it is over 5%, it is affecting live decisions now.
- Check agent share per variant on your two or three most recent tests. Unevenness is the signal that something already went wrong.
- Verify where your platform filters. Ask your vendor whether exclusion happens before or after variant assignment. The answer determines whether their filtering helps or hurts.
- Add agent share to your guardrail set so this is monitored rather than periodically rediscovered.
The Bottom Line
Agentic browsers are not a fraud problem — they are a measurement problem, and a worse one, because the traffic is legitimate. Real people are behind these sessions, so blocking is the wrong response. The right response is to identify them before assignment, keep them out of the experiment, and watch their share as a standing guardrail.
None of this is exotic work. It is the same discipline as any other data-quality check in experimentation, applied to a category of traffic that did not meaningfully exist three years ago and now accounts for a growing share of every site's sessions.
Platforms that handle assignment, filtering, and guardrails as one system have an advantage here, because the filtering decision happens in the right place by construction. That is how Surface AI approaches it — bot exclusion at assignment time, with agent share monitored as a guardrail on every running experiment.