chore: block more ai bots
This commit is contained in:
parent
b455172fc2
commit
9d85f1b834
1 changed files with 16 additions and 1 deletions
|
@ -11,6 +11,12 @@ Disallow:
|
|||
User-agent: AdsBot-Google
|
||||
Disallow: /
|
||||
|
||||
User-agent: Amazonbot
|
||||
Disallow: /
|
||||
|
||||
User-agent: anthropic-ai
|
||||
Disallow: /
|
||||
|
||||
User-agent: Bytespider
|
||||
Disallow: /
|
||||
|
||||
|
@ -20,6 +26,9 @@ Disallow: /
|
|||
User-agent: ChatGPT-User
|
||||
Disallow: /
|
||||
|
||||
User-agent: cohere-ai
|
||||
Disallow: /
|
||||
|
||||
User-agent: FacebookBot
|
||||
Disallow: /
|
||||
|
||||
|
@ -29,8 +38,14 @@ Disallow: /
|
|||
User-agent: GPTBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: Omgili
|
||||
User-agent: omgili
|
||||
Disallow: /
|
||||
|
||||
User-agent: Omgilibot
|
||||
Disallow: /
|
||||
|
||||
User-agent: PerplexityBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: YouBot
|
||||
Disallow: /
|
Reference in a new issue