cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-03 19:03:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
367 commits 2 branches 27 tags 521 KiB
Commit graph

9 commits

Author SHA1 Message Date
ai.robots.txt
5b8650b99b Update from Dark Visitors
Some checks failed
/ run-tests (push) Has been cancelled
Details
2025-03-29 00:54:10 +00:00
Cory Dransfeldt
ec18af7624
Revert "Merge pull request #91 from deyigifts/perplexity-user" 
This reverts commit 68d1d93714.
 2025-03-27 12:51:22 -07:00
ai.robots.txt
68d1d93714 Merge pull request #91 from deyigifts/perplexity-user 
Update perplexity bots
 2025-03-27 19:29:30 +00:00
Dennis Camera
a884a2afb9 .htaccess: Make regex in RewriteCond safe 
Improve the regular expression by removing unneeded anchors and
escaping special characters (not just space) to prevent false positives
or a misbehaving rewrite rule.
 2025-02-18 12:53:22 +01:00
Dennis Camera
c0d418cd87 .htaccess: Allow robots access to /robots.txt 2025-02-18 12:49:29 +01:00
ai.robots.txt
693289bb29 chore: add Brightbot 1.0 2025-02-16 21:37:52 +00:00
ai.robots.txt
89d4c6e5ca Merge pull request #73 from nisbet-hubbard/patch-8 
Actually block Semrush’s AI tools
 2025-02-01 10:51:01 +00:00
ai.robots.txt
6c552a3daa Merge pull request #71 from jsheard/patch-1 
Add Crawlspace
 2025-01-20 17:45:42 +00:00
Massimo Gismondi
933aa6159d Implementing htaccess generation 2025-01-07 11:02:29 +01:00