cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-04 19:13:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Update robots.json

Browse source 
Co-authored-by: Glyn Normington <work@underlap.org>
This commit is contained in:
Joshua Sheard 2025-01-20 10:59:02 +00:00 committed by GitHub
parent 5aa08bc002
commit 7427d96bac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
robots.json
View file

@ -95,7 +95,7 @@
"respect": "[Yes](https://news.ycombinator.com/item?id=42756654)", "respect": "[Yes](https://news.ycombinator.com/item?id=42756654)",
"function": "Scrapes data", "function": "Scrapes data",
"frequency": "Unclear at this time.", "frequency": "Unclear at this time.",
"description": "Provides crawling services for any purpose, but most likely to be used for AI model training." "description": "Provides crawling services for any purpose, probably including AI model training."
}, },
"Diffbot": { "Diffbot": {
"operator": "[Diffbot](https://www.diffbot.com/)", "operator": "[Diffbot](https://www.diffbot.com/)",