mirror of
https://github.com/ai-robots-txt/ai.robots.txt.git
synced 2025-05-17 16:03:10 +00:00
parent
84d63916d2
commit
1c470babbe
6 changed files with 6 additions and 3 deletions
|
@ -46,6 +46,7 @@ User-agent: PanguBot
|
|||
User-agent: Perplexity-User
|
||||
User-agent: PerplexityBot
|
||||
User-agent: PetalBot
|
||||
User-agent: QualifiedBot
|
||||
User-agent: Scrapy
|
||||
User-agent: SemrushBot-OCOB
|
||||
User-agent: SemrushBot-SWA
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue