cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-19 08:26:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge pull request #102 from ai-robots-txt/imgproxy-bot
Some checks are pending
/ run-tests (push) Waiting to run
Details

Browse source 
chore(robots.json): adds imgproxy crawler
This commit is contained in:
ai.robots.txt 2025-04-10 19:22:34 +00:00
parent a891ad7213
commit 4a764bba18
4 changed files with 4 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

1
robots.txt
View file

@ -26,6 +26,7 @@ User-agent: iaskspider/2.0
User-agent: ICC-Crawler
User-agent: ImagesiftBot
User-agent: img2dataset
User-agent: imgproxy
User-agent: ISSCyberRiskCrawler
User-agent: Kangaroo Bot
User-agent: Meta-ExternalAgent