cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-19 00:26:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Fix html-mangled hyphen in Perplexity-Users

Browse source 
Fixes: #99
This commit is contained in:
Frederic Barthelemy 2025-04-04 17:34:14 -07:00
parent 6b0349f37d
commit 5f5a89c38c
No known key found for this signature in database
GPG key ID: 6FF43C49A5D473EF
12 changed files with 42 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

7
code/test_files/robots.json
View file

@ -223,6 +223,13 @@
"operator": "[Webz.io](https://webz.io/)",
"respect": "[Yes](https://web.archive.org/web/20170704003301/http://omgili.com/Crawler.html)"
},
"Perplexity-User": {
"operator": "[Perplexity](https://www.perplexity.ai/)",
"respect": "[No](https://docs.perplexity.ai/guides/bots)",
"function": "Used to answer queries at the request of users.",
"frequency": "Only when prompted by a user.",
"description": "Visit web pages to help provide an accurate answer and include links to the page in Perplexity response."
},
"PerplexityBot": {
"operator": "[Perplexity](https://www.perplexity.ai/)",
"respect": "[No](https://www.macstories.net/stories/wired-confirms-perplexity-is-bypassing-efforts-by-websites-to-block-its-web-crawler/)",