BentAxleBot

User agent: BentAxleBot/0.1 (+https://bent-axle.co.za/bot)

Who we are

Bent Axle is a South African home for the radio-control (RC) hobby: a product comparison that shows shoppers which local shops stock a product and at what price and sends them to the shop to buy, an events calendar, a directory of places to drive and fly, tutorials, and a marketplace where owners of pre-owned RC gear find buyers. BentAxleBot is the automated program that collects the product information for the comparison. Operator: Bent Axle, South Africa. Contact: info@bent-axle.co.za.

What it reads

Public product pages and the sitemap and robots.txt of a shop, and only these facts: the product name, the price, whether it is in stock, and the product images. It does not log in, use a cart or checkout, open account or search pages, fill in forms, or buy anything, and it stores no personal data about shoppers.

How it behaves

One request at a time per shop, at least 7.5 seconds apart: at most 8 requests a minute and 3,000 a day per shop.
It reads your robots.txt before each run and obeys it, including Crawl-delay and disallowed paths.
It uses your sitemap's last-modified dates and conditional requests, so a page that has not changed is not downloaded again.
It honours Retry-After and slows down when a shop is slow or answers with errors.
It stops visiting a shop after repeated refusals (429, 403, 401 or a bot challenge) and does not try to get around them. It does not start again until a person at Bent Axle has looked at it.

Signed requests

Some platforms, Shopify among them, ask automated visitors to identify themselves with Web Bot Auth. For shops hosted on such platforms, BentAxleBot signs each request with an HTTP Message Signature (RFC 9421, Ed25519, tag web-bot-auth) and sends Signature-Agent: "https://bent-axle.co.za". The public key that checks the signature is published at:

https://bent-axle.co.za/.well-known/http-message-signatures-directory

Other shops receive the plain user agent only.

If you would like BentAxleBot to stop

Any one of these is enough:

Disallow it in your robots.txt:
```
User-agent: BentAxleBot
Disallow: /
```
Email info@bent-axle.co.za with your shop's address. We stop visiting it and take its products off Bent Axle.
Block the user agent at your firewall or CDN. We treat that as a refusal and stop.

If it is causing a problem

Email info@bent-axle.co.za with the time and what your server saw. We stop the crawler for your shop straight away and look into it.

If you would like to be listed accurately

We are glad to correct how your products appear or to receive a product feed from you instead of reading your pages.