GatewayCompareBot

We run a crawler that reads payment providers' public pricing and documentation pages, and downloads bulk files published by financial regulators. If you have found this page in your logs, here is everything about it.

Identifying it

GatewayCompareBot/1.0 (+https://gatewaycompare.com/bot)

How it behaves

• One request per second per host, maximum. Never parallel against the same host.
• Obeys robots.txt, including Allow and longest-match precedence.
• Fetches a small number of specific pages — pricing, documentation, status — not a full crawl.
• Typically visits a given provider weekly.
• Does not execute JavaScript, submit forms, or follow links behind authentication.

Blocking it

User-agent: GatewayCompareBot
Disallow: /

We will honour that. Your listing stays, but its figures will go stale and eventually be marked unverified, which costs you points in our matching. Claiming your listing is the better route — you then maintain the data yourself and we stop fetching.

Problems

If the crawler is causing load or behaving badly, email [email protected] and we will fix it. Include a log excerpt if you can.