cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-05-17 07:53:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Merge pull request #129 from ai-robots-txt/google-cloudvertexbot

Browse source 
chore(robots.json): adds Google-CloudVertexBot
This commit is contained in:
Glyn Normington 2025-05-16 12:35:04 +01:00 • committed by GitHub
commit 89c0fbaf86
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 8 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

7
robots.json
View file

@ -160,6 +160,13 @@
"operator": "Unknown",
"respect": "[Yes](https://imho.alex-kunz.com/2024/01/25/an-update-on-friendly-crawler)"
},
"Google-CloudVertexBot": {
"operator": "Google",
"respect": "[Yes](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers)",
"function": "Build and manage AI models for businesses employing Vertex AI",
"frequency": "No information.",
"description": "Google-CloudVertexBot crawls sites on the site owners' request when building Vertex AI Agents."
},
"Google-Extended": {
"operator": "Google",
"respect": "[Yes](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers)",