mirror of
https://github.com/ai-robots-txt/ai.robots.txt.git
synced 2025-04-07 04:17:46 +00:00
Add Kagibot to robots.txt
This commit is contained in:
parent
5e7c3c432f
commit
9f506ac0aa
1 changed files with 1 additions and 0 deletions
|
@ -27,6 +27,7 @@ User-agent: ICC-Crawler
|
|||
User-agent: ImagesiftBot
|
||||
User-agent: img2dataset
|
||||
User-agent: ISSCyberRiskCrawler
|
||||
User-agent: Kagibot
|
||||
User-agent: Kangaroo Bot
|
||||
User-agent: Meta-ExternalAgent
|
||||
User-agent: Meta-ExternalFetcher
|
||||
|
|
Loading…
Reference in a new issue