ai.robots.txt

cdransf/ai.robots.txt

Fork 0

mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-04 11:03:59 +00:00

Commit graph

Select branches

Hide pull requests

main

missing-dependency

#1

#10

#14

#15

#16

#17

#18

#2

#21

#22

#24

#25

#27

#28

#29

#31

#32

#33

#35

#36

#37

#38

#39

#41

#42

#43

#45

#46

#47

#49

#50

#51

#54

#55

#56

#57

#58

#6

#61

#62

#63

#64

#65

#66

#67

#68

#71

#73

#79

#8

#80

#83

#84

#86

#91

#92

#94

#95

#95

#97

#98

#98

v1

v1.1

v1.10

v1.11

v1.12

v1.13

v1.14

v1.15

v1.16

v1.17

v1.18

v1.19

v1.2

v1.20

v1.21

v1.22

v1.23

v1.24

v1.25

v1.26

v1.3

v1.4

v1.5

v1.6

v1.7

v1.8

v1.9

1c9810d8c5

Add facebookexternalhit nisbet-hubbard 2024-07-29 19:37:58 +0800
2972926532

chore: add OAI-SearchBot v1.6 Cory Dransfeldt 2024-07-26 09:06:10 -0700
c17cae6e9d

link to bot metrics table Glyn Normington 2024-07-17 02:28:32 +0100
3692d66918

chore: update bots table v1.5 Cory Dransfeldt 2024-07-16 12:07:01 -0700
af52578965

chore: drop google adbot; add GoogleOther bots Cory Dransfeldt 2024-07-16 12:05:34 -0700
570fd36ea2

chore: update bots table Cory Dransfeldt 2024-07-10 19:47:23 -0700
0ca6bce87e

chore: add ImagesiftBot v1.4 Cory Dransfeldt 2024-07-09 17:41:32 -0700
0971af19b6

chore: peer39 unrelated to ai Cory Dransfeldt 2024-07-09 17:39:51 -0700
74fa789985

Merge pull request #18 from glyn/add-ref Cory Dransfeldt 2024-06-29 12:22:23 -0700
4a8ce0b51d Add reference Glyn Normington 2024-06-29 12:20:38 +0100
70e8985622

chore: remove unused image v1.3 Cory Dransfeldt 2024-06-22 12:44:02 -0700
fd4ade555c

Merge pull request #17 from corbindavenport/main Cory Dransfeldt 2024-06-22 12:42:52 -0700
fe98e41546

Update Perplexity information Corbin Davenport 2024-06-21 22:17:45 -0400
20dc4e16ad

Update releases feed links Corbin Davenport 2024-06-21 22:02:54 -0400
149a72be0c

Add bot info for omgili, peer39, and youbot Corbin Davenport 2024-06-21 21:43:41 -0400
ec4610a118

Add information for Google, Meta, and img2dataset bots Corbin Davenport 2024-06-21 20:49:25 -0400
4163ca92a5

chore: clean up unnecessary ai.txt Cory Dransfeldt 2024-06-21 08:58:44 -0700
89de2d2d91

chore: resolve conflict v1.2 Cory Dransfeldt 2024-06-20 08:12:24 -0700
a980ace544

chore: update readme Cory Dransfeldt 2024-06-20 08:11:38 -0700
a90ee5e9f0

chore: clean up bots and narrow scope Cory Dransfeldt 2024-06-20 08:09:21 -0700
4b8499d718

Merge pull request #16 from nisbet-hubbard/patch-1 Cory Dransfeldt 2024-06-20 07:57:16 -0700
56c2285462

Update robots.txt nisbet-hubbard 2024-06-20 11:31:08 +0800
429336d725

fix: resolve conflict v1.1 Cory Dransfeldt 2024-06-15 09:26:14 -0700
3f65a93891

chore: keeps Applebot-Extended in favor of Applebot as the latter is simply for search Cory Dransfeldt 2024-06-15 09:25:17 -0700
ca730a81ca

Merge pull request #15 from christiansievers/main Cory Dransfeldt 2024-06-13 10:50:10 -0700
ce0b99124b

Update table-of-bot-metrics.md Christian Sievers 2024-06-13 14:17:31 +0200
1a4868f1a2

chore: update readme Cory Dransfeldt 2024-06-12 19:05:15 -0700
b5d4d5bc74

chore: update readme v1 Cory Dransfeldt 2024-06-12 19:01:23 -0700
548a6686ab

chore: add rss badge Cory Dransfeldt 2024-06-12 18:56:09 -0700
1ac25b9135

chore: propagate applebot-extended out Cory Dransfeldt 2024-06-12 18:53:26 -0700
c546dc048c

Merge pull request #14 from chriskirknielsen/patch-1 Cory Dransfeldt 2024-06-12 18:51:09 -0700
39363fc813

Block Applebot-Extended Christopher Kirk-Nielsen 2024-06-12 16:43:12 -0400
d464f260e0

Merge branch 'main' of github.com:ai-robots-txt/ai.robots.txt Cory Dransfeldt 2024-06-10 13:23:55 -0700
fd10fbda94

chore: add cloudflare waf expression Cory Dransfeldt 2024-06-10 13:23:47 -0700
7a5a21fdad

chore: update bots table Cory Dransfeldt 2024-05-29 09:21:31 -0700
2ca4154d4e

chore: update perplexity entry Cory Dransfeldt 2024-05-29 09:14:35 -0700
e152c110c5

Merge pull request #10 from kg583/main Cory Dransfeldt 2024-05-20 19:43:29 -0700
04688e56fb

Update table-of-bot-metrics.md KG 2024-05-16 18:25:46 -0500
696d2acef8

chore: add cohere-ai Cory Dransfeldt 2024-05-06 10:39:07 -0700
6940db2412

chore: add anthropic bots to table Cory Dransfeldt 2024-05-06 10:36:16 -0700
ec8e10a56c

chore: add bytespider to table Cory Dransfeldt 2024-05-06 10:34:43 -0700
262fe71ad9

chore: add ccbot data to table Cory Dransfeldt 2024-05-06 10:32:05 -0700
dea035365f

chore: add Diffbot and scoopit Cory Dransfeldt 2024-05-05 14:50:04 -0700
29e0799332

chore: add img2dataset to table-of-bot-metrics.md Cory Dransfeldt 2024-04-22 09:30:00 -0700
5025939e5d

chore: add img2dataset to edge func Cory Dransfeldt 2024-04-22 09:28:39 -0700
8ddd2a0bb3

chore: add img2dataset to nginx.conf Cory Dransfeldt 2024-04-22 09:28:21 -0700
2b77b0b0ad

chore: add img2dataset to apache.conf Cory Dransfeldt 2024-04-22 09:27:41 -0700
118ec00126

chore: add img2dataset to robots.txt Cory Dransfeldt 2024-04-22 09:26:59 -0700
6f1a28e17a

chore: update nginx Cory Dransfeldt 2024-04-16 08:20:32 -0700
1664980617

chore: comment Cory Dransfeldt 2024-04-14 10:04:25 -0700
1e91744ed2

feat: add edge function Cory Dransfeldt 2024-04-14 10:03:40 -0700
9a92568266

Merge branch 'main' of github.com:ai-robots-txt/ai.robots.txt Cory Dransfeldt 2024-04-14 09:56:30 -0700
9d56001747

chore: move server configs Cory Dransfeldt 2024-04-14 09:56:23 -0700
d05a6956b8

Create FUNDING.yml Cory Dransfeldt 2024-04-12 11:40:44 -0700
0d9b75b227

chore: move additional resources to wiki Cory Dransfeldt 2024-04-12 09:33:19 -0700
9be338094b

chore: populate gptbot Cory Dransfeldt 2024-04-10 10:53:16 -0700
ef5d847e65

chore: add gitignore; move logo file Cory Dransfeldt 2024-04-10 10:44:07 -0700
cc5966af5f

chore: populate ChatGPT-User Cory Dransfeldt 2024-04-10 10:39:34 -0700
0a8952b34f

chore: add applebot Cory Dransfeldt 2024-04-10 10:18:51 -0700
b402ec681c

chore: add anthropic + amazonbot Cory Dransfeldt 2024-04-10 10:15:22 -0700
46b0532daf

Merge pull request #8 from mattlinares/feature/add-table-of-crawler-detail Cory Dransfeldt 2024-04-10 09:08:22 -0700
db06035714

Merge branch 'main' into feature/add-table-of-crawler-detail Cory Dransfeldt 2024-04-10 09:07:45 -0700
aa0edf05b3 Add Google Gemini response on AbsBot-Google for bot metrics table. Matthew Linares 2024-04-10 15:40:33 +0100
b6be895994

fix: clean up syntax Cory Dransfeldt 2024-04-09 16:08:57 -0700
ed190d4803

fix: syntax Cory Dransfeldt 2024-04-09 16:08:13 -0700
0518b3e777

chore: add FriendlyCrawler to the table Cory Dransfeldt 2024-04-08 12:50:36 -0700
964b29d330

Merge pull request #6 from mattlinares/main Cory Dransfeldt 2024-04-08 12:48:35 -0700
e03d5f4ec2

chore: add FriendlyCrawler to nginx.conf.txt Cory Dransfeldt 2024-04-08 12:41:40 -0700
2d4efec184

chore: add FriendlyCrawler to apache.conf.txt Cory Dransfeldt 2024-04-08 12:41:19 -0700
d6d40989f4

chore: add FriendlyCrawler to robots.txt Cory Dransfeldt 2024-04-08 12:40:59 -0700
048045ac6d Adds table of bot details and metrics. Matthew Linares 2024-04-08 20:29:45 +0100
74e4f64d89

chore: create nginx.conf.txt Cory Dransfeldt 2024-04-08 12:27:59 -0700
609de2af12

chore: create apache.conf.txt Cory Dransfeldt 2024-04-08 12:26:04 -0700
47fc45f2f9

chore: add PiplBot Cory Dransfeldt 2024-04-06 20:25:28 -0700
46c8c9adb3

chore: add Meltwater Cory Dransfeldt 2024-04-03 08:56:30 -0700
c8a6d7f02d

chore: add Seekr Cory Dransfeldt 2024-04-03 08:56:12 -0700
bdf10968fd

chore: add ai.txt Cory Dransfeldt 2024-04-01 09:56:10 -0700
d7064d23fe

chore: additional resources Cory Dransfeldt 2024-04-01 09:55:46 -0700
5e02ebc168

chore: update README Cory Dransfeldt 2024-03-29 12:14:48 -0700
2d1c4707aa

Merge pull request #2 from Explosion-Scratch/patch-1 Cory Dransfeldt 2024-03-28 09:15:51 -0700
3e57b5ab5d

Add GoogleOther --Explosion-- 2024-03-28 10:00:58 -0500
543dfc8452

Merge pull request #1 from glyn/logo Cory Dransfeldt 2024-03-28 07:26:14 -0700
780d772d53 Scale down logo Glyn Normington 2024-03-28 09:11:49 +0000
290d603b06 Reduce logo size Glyn Normington 2024-03-28 09:09:29 +0000
a71da3ecc4 Add logo Glyn Normington 2024-03-28 09:06:46 +0000
297071a664

Update robots.txt Cory Dransfeldt 2024-03-27 11:41:12 -0700
8aeddbdce8

Create robots.txt Cory Dransfeldt 2024-03-27 10:59:01 -0700
be9d22abed

Create README.md Cory Dransfeldt 2024-03-27 10:58:49 -0700
400e0141db

Initial commit Cory Dransfeldt 2024-03-27 10:48:29 -0700