cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-04 19:13:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Update Perplexity information

Browse source
This commit is contained in:
Corbin Davenport 2024-06-21 22:17:45 -04:00 committed by GitHub
parent 20dc4e16ad
commit fe98e41546
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
table-of-bot-metrics.md
View file

@ -20,5 +20,5 @@
|omgilibot | [Webz.io](https://webz.io/) | [Yes](https://web.archive.org/web/20170704003301/http://omgili.com/Crawler.html) | Data is sold. | No information | Legacy user agent initially used for Omgili search engine. Unknown if still used, `omgili` agent still used by Webz.io. |
|peer39_crawler| [Peer39](https://www.peer39.com/) | [Yes](https://www.peer39.com/crawler-notice) | Targeted advertising. | No information | Web crawler used to "enhance the visibility of your site to advertisers who value and seek out such quality content." |
|peer39_crawler| [Peer39](https://www.peer39.com/) | [Yes](https://www.peer39.com/crawler-notice) | Targeted advertising. | No information | Web crawler used to "enhance the visibility of your site to advertisers who value and seek out such quality content." |
|PerplexityBot | [Perplexity](https://www.perplexity.ai/) | [Yes](https://docs.perplexity.ai/docs/perplexitybot) | Used to answer queries at the request of users. | Takes action based on user prompts. | Operated by Perplexity to obtain results in response to user queries. |
|PerplexityBot | [Perplexity](https://www.perplexity.ai/) | [No](https://www.macstories.net/stories/wired-confirms-perplexity-is-bypassing-efforts-by-websites-to-block-its-web-crawler/) | Used to answer queries at the request of users. | Takes action based on user prompts. | Operated by Perplexity to obtain results in response to user queries. |
|YouBot | [You](https://about.you.com/youchat/) | [Yes](https://about.you.com/youbot/) | Scrapes data for search engine and LLMs. | No information | Retrieves data used for You.com web search engine and LLMs. |