Publisher information

Magdox crawler

How our public-page collector identifies itself, respects access rules, and responds to publishers.

Magdox monitors permitted public business pages for changes such as pricing and product updates. This page identifies our public-page collector for website operators.

Identification and access

Our public-page collector identifies itself as Magdox-Archive/1.0. The crawler token for robots.txt is Magdox-Archive.

Collection checks robots.txt and pauses when permission cannot be established or the publisher refuses access. We do not use alternate transports to get around a refusal, solve access challenges, sign in to private accounts, or submit forms through this collector.

Robots rules are a collection constraint, not a content license. Publisher terms and applicable permissions also govern the sources we use.

Contact us about a source

For a collection concern, correction or removal request, email [email protected] with the affected URL and your relationship to the website. Please do not send account passwords or other secrets.