cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-19 16:35:03 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
380 commits 2 branches 29 tags 668 KiB
Commit graph

14 commits

Author SHA1 Message Date
Glyn Normington
d05ede8fe1 Clarify our position on sponsorship 
Some firms, including those with .ai domains, have
offered to sponsor this project. So make our position
clear.
 2025-04-18 17:46:56 +01:00
Glyn Normington
d50615d394 Improve formatting 
This clarifies the scope of the tip is Apache httpd.
 2024-11-10 01:06:13 +00:00
Glyn Normington
2c88909be3 Fix formatting 2024-11-10 01:02:18 +00:00
Glyn Normington
9295b6a963 Clarify our rationale 
I deleted the point about excessive load on
crawled sites as any other crawler could potentially
be guilty of this and I wouldn't want our scope to
creep to all crawlers.

Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/53#issuecomment-2466042550
 2024-11-09 04:45:47 +00:00
Glyn Normington
e6bb7cae9e Augment the "why" FAQ 
Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/40#issuecomment-2419078796
 2024-10-17 12:27:05 +01:00
Glyn Normington
b229f5b936 Re-order the FAQ 
The "why" question should come first.
 2024-10-17 12:25:54 +01:00
Malte Ubl
809851ae88
Add instructions for AI bot blocking on Vercel 2024-09-07 15:59:25 -07:00
Cory Dransfeldt
bc66d10afd
chore: update faq 2024-08-14 09:21:26 -07:00
Glyn Normington
b4d25bf0cb Add FAQ 2024-08-06 17:20:26 +01:00
Joshua Sheard
8dbbdbf44c
Add Cloudflares first-party scraper blocking to FAQ 2024-08-04 21:38:02 +01:00
nisbet-hubbard
2b56c72bac
Update FAQ.md 2024-08-03 14:27:25 +08:00
nisbet-hubbard
b24e5cb3bb
Update FAQ.md 2024-08-03 14:12:50 +08:00
nisbet-hubbard
74b1502839
Update FAQ.md 2024-08-03 14:04:58 +08:00
Cory Dransfeldt
6c596a50ea
chore: move FAQ into repo 2024-08-01 07:53:43 -07:00