Traffic from search engines and other bots
Does bot traffic show up in your RUM data? Short answer: No. Here's why.
We ignore any RUM data that appears to come from bots or automation software. We do this because the browsing behavior and performance characteristics of bots does not reflect real users, and we do not want bots to skew your RUM data.
We detect bots by parsing the user-agent header for every RUM beacon we receive. We then match the user-agent against a list of known bots. The list of bot user-agents that we check against is below (last updated 22 November 2023).
- AdsBot-Google
- AmazonAdBot
- APIs-Google
- Applebot
- Baiduspider
- bingbot
- BingPreview
- bot.html
- Bytespider
- Catchpoint
- Chrome-Lighthouse
- Cypress
- FeedFetcher-Google
- Fundamental Monitor
- Ghost Inspector
- Google Page Speed Insights
- Google Web Preview
- Google-Read-Aloud
- Googlebot
- googleweblight
- HeadlessChrome
- https://deepcrawl.com/bot
- HubSpot Crawler
- JobBot
- Mediapartners-Google
- moatbot
- PetalBot
- PhantomJS
- Pingdom
- PTST
- quantcast
- RuxitSynthetic
- Webcrawler
- Yahoo Ad monitoring
- YandexBot
- YandexMobileBot
- Any user-agent identified as "wget" by ua-parser
- Any user-agent identified as "bot" by ua-parser
Updated 5 months ago