From 5fe40a1397e162d18b86a75c3305001eea87ee4e Mon Sep 17 00:00:00 2001 From: MSK <73292171+msk-nightly@users.noreply.github.com> Date: Mon, 12 May 2025 18:45:54 +0600 Subject: [PATCH] Update robots.txt (Added GoogleBot and PiplBot) Update robots.txt to add GoogleBot and PiplBot. --- robots.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/robots.txt b/robots.txt index 232e119..3f6e996 100644 --- a/robots.txt +++ b/robots.txt @@ -21,6 +21,7 @@ User-agent: FacebookBot User-agent: Factset_spyderbot User-agent: FirecrawlAgent User-agent: FriendlyCrawler +User-agent: Googlebot User-agent: Google-Extended User-agent: GoogleOther User-agent: GoogleOther-Image @@ -46,6 +47,7 @@ User-agent: PanguBot User-agent: Perplexity-User User-agent: PerplexityBot User-agent: PetalBot +User-agent: PiplBot User-agent: Scrapy User-agent: SemrushBot-OCOB User-agent: SemrushBot-SWA