Commit graph

13 commits

Author SHA1 Message Date
Glyn Normington
d50615d394 Improve formatting
This clarifies the scope of the tip is Apache httpd.
2024-11-10 01:06:13 +00:00
Glyn Normington
2c88909be3 Fix formatting 2024-11-10 01:02:18 +00:00
Glyn Normington
9295b6a963 Clarify our rationale
I deleted the point about excessive load on
crawled sites as any other crawler could potentially
be guilty of this and I wouldn't want our scope to
creep to all crawlers.

Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/53#issuecomment-2466042550
2024-11-09 04:45:47 +00:00
Glyn Normington
e6bb7cae9e Augment the "why" FAQ
Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/40#issuecomment-2419078796
2024-10-17 12:27:05 +01:00
Glyn Normington
b229f5b936 Re-order the FAQ
The "why" question should come first.
2024-10-17 12:25:54 +01:00
Malte Ubl
809851ae88
Add instructions for AI bot blocking on Vercel 2024-09-07 15:59:25 -07:00
bc66d10afd
chore: update faq 2024-08-14 09:21:26 -07:00
Glyn Normington
b4d25bf0cb Add FAQ 2024-08-06 17:20:26 +01:00
Joshua Sheard
8dbbdbf44c
Add Cloudflares first-party scraper blocking to FAQ 2024-08-04 21:38:02 +01:00
nisbet-hubbard
2b56c72bac
Update FAQ.md 2024-08-03 14:27:25 +08:00
nisbet-hubbard
b24e5cb3bb
Update FAQ.md 2024-08-03 14:12:50 +08:00
nisbet-hubbard
74b1502839
Update FAQ.md 2024-08-03 14:04:58 +08:00
6c596a50ea
chore: move FAQ into repo 2024-08-01 07:53:43 -07:00