cdransf/coryd.dev-eleventy
Archived
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: dry up robots.txt templating

Browse source
This commit is contained in:
Cory Dransfeldt 2024-02-01 12:31:43 -08:00
parent 29915a2e2c
commit 3edecfe47c
No known key found for this signature in database
4 changed files with 26 additions and 42 deletions
Download patch file Download diff file Expand all files Collapse all files

44
src/meta/robots.liquid
View file

@ -7,45 +7,7 @@ Sitemap: https://coryd.dev/sitemap.xml
User-agent: *
Disallow:
User-agent: AdsBot-Google
{% for robot in robots %}
User-agent: {{ robot }}
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: omgili
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: YouBot
Disallow: /
{% endfor %}