Bot Traffic Detection: A 2026 Guide Beyond Google Analytics

Bot traffic detection is the practice of identifying non-human visitors and separating them from real users before they pollute analytics, drain ad budgets, or skew bidding algorithms. Standard rule-based methods in 2026 catch under 40% of sophisticated bots, because AI-driven automation now passes CAPTCHAs, spoofs human cursor physics, and rotates residential proxies at session level. ^[1] Google Analytics’ built-in filter only removes traffic on the IAB/ABC declared-bots list. Everything beyond declared traffic, the entire Sophisticated Invalid Traffic (SIVT) category, requires deeper instrumentation at the server and client level.

This guide is for developers and ad-ops teams who need to detect bot traffic past the GA4 baseline. We cover the server-log signals you can read without JavaScript, the client-side signals when you have DOM access, a working detection script you can copy into a project, and where commercial layers add signals you cannot collect alone.

Why Google Analytics bot filtering isn’t enough

GA4’s built-in bot filter is enabled by default and cannot be turned off. It works by comparing incoming hits against the IAB/ABC International Spiders & Bots List, a maintained blocklist of declared crawlers, bots, and spiders. ^[2] The filter is good at what it does: declared bot user agents and obvious crawler IP ranges never reach your reports. The problem is what the filter does not do.

The IAB list is, by design, a declared-bots list. It contains the user agents that bots voluntarily identify with: Googlebot, bingbot, AhrefsBot, and thousands of others that follow robots.txt and respect crawl-delay headers. It does not contain user agents the bot does not want you to recognize. Any automation framework targeting your site will spoof a Chrome User-Agent string, which sits well outside any blocklist.

There’s a structural mismatch between the IAB framework and SIVT. The Media Rating Council’s invalid-traffic taxonomy splits non-human traffic into two categories ^[3]:

Category	What it covers	Caught by GA4?
GIVT (General Invalid Traffic)	Declared bots, data-center IPs, known crawler ranges, repeated identical user agents	Mostly yes
SIVT (Sophisticated Invalid Traffic)	Headless browsers with stealth plugins, residential proxy traffic, click farms on real devices, AI-driven bots, hijacked device traffic, attribution fraud	No

The 40%-catch-rate figure for sophisticated bots applies to standard rule-based filtering across the industry, not just GA4. ^[1] GA4 in particular is even narrower because it operates only on hits that reach the measurement endpoint, with no view of the TLS handshake, no header inspection, and no client-side instrumentation it can act on in real time. Any bot that successfully renders your site and fires the measurement event looks identical to a real user in GA4 reports — we cover the limits in detail in our Google Analytics bot filtering deep-dive.

click fraud detection guide

In our field experience, advertisers who think “our GA4 looks clean so we’re fine” are usually 8-22% over-counted on paid channels, with the gap concentrated in affiliate and programmatic traffic.

Server-log signals (what to look at without JS instrumentation)

The cheapest bot signals are the ones already in your access logs and reverse-proxy data. No JavaScript, no client cooperation, no cookies. If you have nginx, Caddy, Cloudflare, or any reasonable web server in front of your app, you can build the baseline filter from data you’re already collecting. Below are the signals that pay back fastest, in rough order of strength.

User-Agent versus capabilities mismatch

The User-Agent header is the first thing every bot lies about. Detecting the lie is a matter of cross-referencing the declared UA against signals the bot did not think to spoof. A request claiming Chrome/126 on Windows 10 should carry the headers Chrome 126 sends, in the order Chrome 126 sends them, with the cipher suites Chrome 126 negotiates. A Python requests library spoofing the same UA will lack the sec-ch-ua Client Hints, send headers in alphabetical order, and offer a tiny TLS cipher list. The mismatch is the tell.

Practical checks:

• Sec-Fetch-* headers must be present and consistent on modern Chrome / Edge / Firefox traffic
• Accept-Language should be non-empty for browser UAs; bots often omit it
• HTTP version must match the UA’s era: a UA claiming Chrome 110+ requesting over HTTP/1.1 is suspicious
• Accept-Encoding should include br for any modern browser UA

JA3 / JA4 TLS fingerprint

JA3 hashes the TLS Client Hello packet, the first encrypted handshake your server sees. The hash is built from TLS version, cipher suites, extensions, elliptic curves, and EC point formats. ^[4] Real browsers produce a small, well-known set of JA3 hashes per major version. Python requests, Go’s net/http, Node https, curl, and even patched headless Chrome produce distinctive JA3 hashes that rarely match a real-browser fingerprint coherently.

JA4 is the newer evolution from FoxIO, addressing JA3’s stability issues with TLS extension ordering. If your edge supports it (Cloudflare, some HAProxy builds, Suricata), prefer JA4.

The single highest-leverage server signal we see in production is JA3 / UA disagreement: a Chrome 126 user agent paired with a JA3 hash known to belong to python-requests/2.31. There’s no legitimate reason for this combination to exist. It’s automation, period.

ASN classification (data center vs residential)

Every IP belongs to an Autonomous System (ASN). Public datasets (IPinfo, MaxMind, Team Cymru) classify ASNs as data center, residential, mobile, business, hosting, or VPN. A residential user from Comcast (AS7922) is plausible; a “user” from DigitalOcean (AS14061) browsing your e-commerce checkout fraud surface is not.

The standard pattern:

ASN type	Plausibility for organic traffic	Action
Residential (ISP)	High	Allow, score on other signals
Mobile (carrier)	High	Allow, expect mobile-shaped fingerprint
Business	Medium	Allow, common for B2B traffic
Data center / hosting	Very low	Flag, except known good (Apple Private Relay, iCloud)
Anonymizing VPN	Low	Flag, allow with caution
Tor exit	Low	Flag, allow with caution

The exceptions matter. Apple Private Relay (AS714 and related) routes legitimate Safari traffic through Apple’s egress. Cloudflare WARP (AS13335) does the same. Treating all data-center ASNs as automatic bots will burn real iPhone users. The right pattern is to maintain an allowlist of known-good privacy ASNs and treat the remaining data-center pool as high-suspicion.

Request rate and temporal pattern (the 3:17 UTC spike)

Real human traffic follows daily curves that match the geographic distribution of your visitors. Bot traffic frequently does not. The classic tells:

• Spikes at oddly precise times (3:17 UTC every day, exactly 14 requests, all from the same /24 range)
• Constant request rate over hours with zero variance (real humans have variance)
• Traffic at 4 AM local time from an ASN that geolocates to a market that’s asleep
• /24 or /16 ranges producing coordinated bursts within seconds

We once traced a 19% lift in “conversions” for a client to a single botnet hitting the funnel from 47 IPs in a single Romanian /24 at precisely 03:00, 03:17, 03:34, 03:51 UTC. The timestamps were so consistent they looked like a cron expression. They were.

HTTP version and Accept-Language consistency

Two small, cheap, high-leverage checks:

• Modern browser UAs (Chrome 100+, Firefox 100+, Safari 16+) negotiate HTTP/2 or HTTP/3 by default. A request from Chrome/126 over HTTP/1.1 to an HTTP/2-capable server is suspicious.
• Accept-Language is sent by every real browser. Headless setups frequently omit it or send a generic en-US,en;q=0.9 that mismatches the geolocation of the IP (Vietnamese IP, US English only, no other languages).

Neither is conclusive alone. Both add weight to the score.

Citation capsule

In 2024 and 2025, HUMAN Security and Imperva separately documented that sophisticated bot traffic now evades rule-based filters at rates above 60%, with residential-proxy bots routing through real consumer ISPs and patched headless browsers passing single-signal CAPTCHAs reliably. ^[1]

types of ad fraud

Client-side signals (when you have JS access)

When you can land JavaScript on the page (your own domain, a tag manager, a vendor script), client-side detection roughly doubles your catch rate. The browser exposes dozens of attributes the bot cannot fully control without significant engineering effort. These are the signals worth instrumenting first.

navigator.webdriver and headless markers

The cheapest signal in the entire stack. The W3C WebDriver spec requires that automated browsers expose navigator.webdriver === true. Selenium, Playwright, and base Puppeteer all honor this. Unsophisticated bots will hit this flag on the first request.

Sophisticated bots patch it. puppeteer-extra-plugin-stealth and undetected-chromedriver both override the property. So navigator.webdriver is not a moat, but it’s free and catches the floor of the market. Pair it with other headless markers:

• window.chrome should exist on Chrome and Edge; missing means non-Chrome or a stripped-down headless build
• navigator.languages should be a non-empty array; many headless setups leave it empty
• navigator.plugins.length === 0 on a UA claiming Windows or macOS is suspicious
• Notification.permission should equal 'default' in headless; production browsers vary

Mouse entropy and movement physics

Real humans move the cursor in noisy, non-linear paths with micro-corrections, overshoots, and pauses. Bots either skip mouse movement entirely (most common) or generate movement programmatically (Bezier curves, linear interpolation, simulated noise that’s too clean).

Capture mousemove events for the first 5-10 seconds of a session and compute:

• Entropy of (dx, dy) deltas across consecutive events
• Variance of inter-event timing (humans cluster around 8-16 ms, bots produce flat lines or perfect distributions)
• Path linearity measured by the ratio of straight-line distance to actual path length

Across the production traffic we score, real human sessions cluster in a mouse-entropy band roughly 5-10x higher than sophisticated bot sessions on the same page. The gap is wide enough that even attackers using motion-replay tools leave a residual signature.

Scroll velocity distributions

Humans scroll in bursts: scroll, pause, read, scroll again. The velocity distribution is multi-modal with long tails. Bots that need to fire scroll events to appear human typically use constant velocity or sigmoid curves that produce unrealistically smooth distributions. Measuring scroll velocity variance and clustering across a session is a strong supplemental signal.

Touch event physics

On mobile, touch events carry richer biometric data than mouse events. The Touch interface exposes force, radiusX, radiusY, and rotationAngle. Real fingers produce non-zero force, variable radii, and slight rotation between contacts. Emulated touch (from BrowserStack-style farms or device emulators) produces clean zeros or constants across these fields.

WebGL renderer and canvas fingerprint

Browsers expose the underlying GPU renderer string via WebGL:

const gl = canvas.getContext('webgl');
const dbg = gl.getExtension('WEBGL_debug_renderer_info');
const renderer = gl.getParameter(dbg.UNMASKED_RENDERER_WEBGL);
// Real device: "ANGLE (Intel, Intel(R) UHD Graphics 620, OpenGL 4.1)"
// Headless Chrome on Linux: often "Mesa OffScreen" or "SwiftShader"

SwiftShader, llvmpipe, and Mesa OffScreen are software renderers used heavily by headless setups. A consumer-class device should report a real GPU (Intel UHD, AMD Radeon, Apple M-series, Adreno, Mali). Software-renderer strings are a strong bot signal.

Canvas fingerprinting adds a parallel check. Render text and a few primitives to an offscreen canvas, hash the result, and compare against known automation hashes. Many headless setups produce stable, identical canvas hashes that mass-show up across “different” users.

Citation capsule

Cloudflare’s bot management documentation explicitly catalogs navigator.webdriver, WebGL renderer strings, and canvas fingerprint stability as primary bot indicators, noting that no single signal is sufficient but multi-signal correlation drives detection accuracy above 95% on common automation frameworks. ^[4]

A basic detection script

Here’s a working starting point you can drop into a page or extend in a vendor script. It runs in under 5 ms, requires no dependencies, and catches the floor of unsophisticated automation. Treat it as a baseline, not as production. Real production detection requires server-side correlation, async behavioral capture over time, and a tuned scoring model.

// Basic browser-side bot detection — baseline scoring stack
// Returns { isBot, score, signals } for the current request.
// Pair with server-side JA3, ASN, and rate signals for production use.

function detectBot() {
  const signals = [];
  const w = window;
  const n = navigator;
  const d = document;

  // 1. WebDriver flag (cheap, catches the floor)
  if (n.webdriver === true) signals.push('webdriver-flag');

  // 2. Languages list missing or empty
  if (!n.languages || n.languages.length === 0) signals.push('no-languages');

  // 3. Plugins absent on a desktop UA
  if (n.plugins && n.plugins.length === 0 && /Win|Mac/.test(n.platform)) {
    signals.push('no-plugins-desktop');
  }

  // 4. Chrome global missing on a Chrome UA
  if (/Chrome|Edg/.test(n.userAgent) && typeof w.chrome === 'undefined') {
    signals.push('missing-chrome-global');
  }

  // 5. WebGL renderer is software (headless tell)
  try {
    const c = d.createElement('canvas');
    const gl = c.getContext('webgl') || c.getContext('experimental-webgl');
    if (!gl) {
      signals.push('no-webgl');
    } else {
      const dbg = gl.getExtension('WEBGL_debug_renderer_info');
      const r = dbg ? gl.getParameter(dbg.UNMASKED_RENDERER_WEBGL) : '';
      if (/SwiftShader|llvmpipe|Mesa OffScreen|ANGLE \(Google/i.test(r)) {
        signals.push('software-renderer');
      }
    }
  } catch (e) {
    signals.push('webgl-error');
  }

  // 6. Canvas fingerprint stability check (very simplified)
  try {
    const c1 = d.createElement('canvas');
    c1.width = 200; c1.height = 50;
    const ctx = c1.getContext('2d');
    ctx.textBaseline = 'top';
    ctx.font = '14px Arial';
    ctx.fillStyle = '#f60';
    ctx.fillRect(0, 0, 200, 50);
    ctx.fillStyle = '#069';
    ctx.fillText('bot-detect-canvas-0', 2, 2);
    const data = c1.toDataURL();
    // Known automation canvas hashes (truncated examples):
    const knownBotHashes = [
      'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA',
    ];
    if (knownBotHashes.some(h => data.startsWith(h))) {
      signals.push('canvas-known-bot');
    }
  } catch (e) {
    signals.push('canvas-error');
  }

  // 7. Permissions API quirk (headless Chrome legacy bug)
  try {
    if (n.permissions && n.permissions.query) {
      n.permissions.query({ name: 'notifications' }).then(p => {
        if (Notification.permission === 'denied' && p.state === 'prompt') {
          signals.push('permissions-quirk');
        }
      }).catch(() => {});
    }
  } catch (e) {}

  // 8. Hardware concurrency / device memory implausibility
  if (n.hardwareConcurrency && n.hardwareConcurrency < 2) {
    signals.push('low-concurrency');
  }
  if (n.deviceMemory && n.deviceMemory < 1) {
    signals.push('low-memory');
  }

  // 9. Outer/inner dimensions zero (common headless tell)
  if (w.outerWidth === 0 || w.outerHeight === 0) {
    signals.push('zero-outer-dims');
  }

  return {
    isBot: signals.length >= 2,
    score: signals.length,
    signals: signals
  };
}

// Async mouse-entropy sampler — call separately and feed into scoring
function collectMouseEntropy(durationMs = 5000) {
  return new Promise(resolve => {
    const events = [];
    const onMove = (e) => events.push([e.clientX, e.clientY, performance.now()]);
    window.addEventListener('mousemove', onMove, { passive: true });
    setTimeout(() => {
      window.removeEventListener('mousemove', onMove);
      if (events.length < 5) return resolve({ entropy: 0, samples: events.length });
      let totalDelta = 0;
      for (let i = 1; i < events.length; i++) {
        const dx = events[i][0] - events[i - 1][0];
        const dy = events[i][1] - events[i - 1][1];
        totalDelta += Math.sqrt(dx * dx + dy * dy);
      }
      resolve({
        entropy: totalDelta / events.length,
        samples: events.length
      });
    }, durationMs);
  });
}

// Usage
const result = detectBot();
if (result.isBot) {
  // ship the verdict to your server, do not block client-side alone
  fetch('/bot-score', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ signals: result.signals, score: result.score })
  });
}

Two practical notes on shipping this:

1. Never block based on client-side signals alone. Always send the verdict to a server endpoint and act there. A bot that controls the client can also lie about the detection result.
2. Sample, do not gate. Initial detection should observe and score. Hard blocking on score >= 2 will catch real users behind privacy extensions. Score, log, and tune for two weeks before adding any blocking action.

click fraud protection API

When to layer commercial tools

A homegrown detection stack catches the floor of bot traffic cheaply. It runs into a ceiling fast. The signals that cost money and engineering effort are the ones a commercial layer earns its keep on. The honest list of what you cannot reasonably build alone:

• Residential proxy detection at the IP level. Companies like Spur, IPQS, and Censys maintain continuously updated maps of which residential IPs are being sold as proxies. Building this internally requires running honeypots across the proxy marketplaces. The same applies to identifying advertising botnets like Methbot and 3ve, which require continuous threat intel feeds.
• Cross-customer fingerprint reputation. Detection vendors see a fingerprint hit thousands of properties per day. A single hash that just attempted credit-card stuffing on three other tenants is suspicious before it touches your site. You cannot replicate this signal in isolation.
• JA3 / JA4 fingerprint corpus. Knowing that JA3 hash cd08e31494f9531f560d64c695473da9 matches python-requests/2.31 requires a labeled corpus. Vendors maintain that corpus actively as TLS libraries update.
• Behavioral biometrics at scale. Modeling mouse entropy distributions across millions of sessions requires sample volume one site cannot generate. Commercial models trained on the cross-customer distribution outperform anything single-tenant.
• Real-time threat intelligence feeds. Botnet C2 IPs, residential proxy egress ranges, and emerging automation fingerprints are published privately by vendors hours to days before public threat intel catches up.

A reasonable architecture: build the in-house baseline (script above, server-side log analysis, basic ASN classification), then layer a commercial scoring API on top for the signals you cannot collect. The combination is more effective than either alone.

Signal class	In-house feasible?	Commercial advantage
navigator.webdriver, headless markers	Yes	None
Mouse entropy, basic behavioral	Yes (single-tenant)	Cross-tenant model accuracy
WebGL renderer, canvas hash	Yes	Known-bot hash corpus
JA3 / JA4 fingerprint	Yes (with TLS inspection)	Labeled corpus, JA3 / UA mismatch ruleset
ASN classification	Yes (with IPinfo / MaxMind)	Privacy-ASN allowlist maintained
Residential proxy detection	No (without honeypot infra)	High
Cross-customer reputation	No	High
Real-time botnet feeds	No	High

click fraud protection

Common false-positive patterns and how to tune

Every bot detector has a false-positive rate. The honest ones publish it. The dangerous ones do not. The categories of real users that get flagged most often, and what to do about each:

Corporate VPNs and proxies

Traffic from corporate egress IPs often looks like data-center traffic to ASN classifiers. The IP belongs to a hosting provider; the User-Agent is real; the user is real. The mitigation:

• Maintain an allowlist of known corporate VPN providers (Cisco AnyConnect ranges, Palo Alto Prisma, Zscaler)
• Weight ASN signal lower when other signals (canvas, WebGL, mouse entropy) are clean
• For B2B advertisers, expect 5-15% of legitimate traffic to come from these ranges

Privacy browsers and extensions

Brave, Tor, Librewolf, and aggressive uBlock Origin configurations all strip fingerprintable surface area. Canvas hashes return blanks, WebGL renderer queries fail, navigator.plugins is empty. These users look indistinguishable from headless Chrome on some signals.

The fix: do not over-weight fingerprint absence. If a user has clean mouse entropy, plausible Accept-Language, and a residential ASN, the missing canvas is privacy, not automation.

Accessibility tools and screen readers

JAWS, NVDA, VoiceOver, and similar screen readers drive the page without mouse events. Mouse-entropy detection flags these users as bot-like because they genuinely don’t move the mouse. They also produce keyboard navigation patterns that differ from sighted users.

The fix: when mouseentropy === 0 but keyboard interaction is rich and timing is human-realistic, score as accessibility user, not bot.

CGNAT and shared mobile IPs

Mobile carriers and some ISPs route many users behind a single egress IP via Carrier-Grade NAT. Twenty users from one IP within a minute looks like a bot burst on rate-based detection. It is not.

The fix: treat mobile-carrier ASNs as expected high-density. Apply rate limits per fingerprint, not per IP, on mobile.

Tuning loop

The practical tuning loop we run for new deployments:

1. Run scoring in observe-only mode for 14 days. Log scores, do not block.
2. Sample 100 high-score sessions manually. Categorize as true positive (bot) or false positive (real user behind something).
3. Adjust signal weights to reduce false positives without losing true positives.
4. Move to soft-action for two weeks: serve harder CAPTCHAs to high-score traffic, do not hard-block.
5. Move to hard block only after confirming false-positive rate below your tolerance (typically 0.5-2%).

The single biggest mistake we see teams make is launching with hard-block on day one. Two weeks of observation saves months of customer-support pain.

Where Adsafee fits

Adsafee provides multi-signal bot traffic detection across server, client, and behavioral layers, returning a verdict in under 100 ms via JavaScript tag, S2S postback, or REST API. We maintain the JA3/JA4 corpus, residential-proxy maps, and known-bot canvas hash database that single-tenant detection cannot reasonably build. For developer teams that want to keep their in-house baseline detection and layer commercial signals on top, our scoring API accepts your existing signal payload and returns the merged verdict plus per-signal explanation for audit and refund disputes.

If you want to compare your current bot detection coverage against a multi-signal baseline, start a free trial. Setup takes about 10 minutes and the first audit returns same-day.

FAQ

The frontmatter faq block contains the structured Q&A this page exposes to search engines. The questions above cover what GA4 misses, navigator.webdriver, JA3, false positives, normal bot traffic baselines, and when commercial tools become worth the spend.

---

Sources

1. HUMAN Security, 2024-2025 Quadrennial Bot Defense Reports and Satori Threat Intelligence briefings, documenting sophisticated bot evasion rates and residential-proxy abuse. Visit: humansecurity.com. ↩

2. IAB Tech Lab and ABC International Federation of Audit Bureaux of Circulations, “International Spiders & Bots List” — the declared-bots maintained list GA4 and most platform filters consume. Visit: iabtechlab.com. ↩

3. Media Rating Council, “Invalid Traffic Detection and Filtration Guidelines Addendum” — definitions of General Invalid Traffic (GIVT) vs Sophisticated Invalid Traffic (SIVT). ↩

4. Cloudflare, “What is bot management” and bot-detection learning center, covering JA3/JA4, navigator.webdriver, WebGL renderer, and multi-signal correlation. Visit: cloudflare.com/learning/bots/. ↩