chore: update robots
This commit is contained in:
parent
d7b608709f
commit
e8be3e8d25
2 changed files with 12 additions and 9 deletions
|
@ -8,26 +8,29 @@ Sitemap: https://coryd.dev/sitemap.xml
|
|||
User-agent: *
|
||||
Disallow:
|
||||
|
||||
User-agent: CCBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: AdsBot-Google
|
||||
Disallow: /
|
||||
|
||||
User-agent: Google-Extended
|
||||
Disallow: /
|
||||
|
||||
User-agent: Bytespider
|
||||
Disallow: /
|
||||
|
||||
User-agent: ChatGPT-User
|
||||
User-agent: CCBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: GPTBot
|
||||
User-agent: ChatGPT-User
|
||||
Disallow: /
|
||||
|
||||
User-agent: FacebookBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: Google-Extended
|
||||
Disallow: /
|
||||
|
||||
User-agent: GPTBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: Omgili
|
||||
Disallow: /
|
||||
|
||||
User-agent: Omgilibot
|
||||
Disallow: /
|
Reference in a new issue