cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-04-04 11:03:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Commit graph

  • 6f58ddc623
         Merge pull request #54 from glyn/rationale Glyn Normington 2024-11-10 00:58:29 +0000
  • 9295b6a963 Clarify our rationale Glyn Normington 2024-11-09 04:45:47 +0000
  • 9e06cf3bc9 Updated from new robots.json v1.19 dark-visitors 2024-10-29 00:52:12 +0000
  • bc0a0ad0e9 Update from Dark Visitors dark-visitors 2024-10-29 00:52:12 +0000
  • fe5f407673 Update from Dark Visitors dark-visitors 2024-10-27 00:54:47 +0000
  • a66b16827d
         Merge pull request #51 from fabianegli/php-to-python-plus-tests Adam Newbold 2024-10-22 21:32:58 -0400
  • 3ab22bc498 make conversions and updates separately triggerable fabianegli 2024-10-19 19:56:41 +0200
  • 6ab8fb2d37 no more failure when run without network fabianegli 2024-10-19 19:11:01 +0200
  • 7e2b3ab037 rename action fabianegli 2024-10-19 19:09:34 +0200
  • 0c05461f84 simplify repo and added some tests fabianegli 2024-10-19 13:06:34 +0200
  • 6bb598820e ignore venv fabianegli 2024-10-18 23:24:13 +0200
  • d62cab66c5
         Merge pull request #50 from glyn/fix-typo Glyn Normington 2024-10-19 04:43:09 +0100
  • 6a359e7fd7 Fix typo and trigger rerun of main job ai.robots.txt 2024-10-19 03:43:00 +0000
  • 38a388097c Fix typo and trigger rerun of main job Glyn Normington 2024-10-19 04:42:27 +0100
  • 83c8603071
         Merge pull request #49 from glyn/php-diagnostics Glyn Normington 2024-10-19 04:34:53 +0100
  • a80bd18fb8 Dump out file contents in PHP script ai.robots.txt 2024-10-19 03:34:29 +0000
  • bdf30be7dc Dump out file contents in PHP script Glyn Normington 2024-10-19 04:33:46 +0100
  • 4d47b17c45
         Merge pull request #47 from fabianegli/fabianegli-patch-1 Glyn Normington 2024-10-19 02:58:05 +0100
  • faf81efb12 Daily update from Dark Visitors dark-visitors 2024-10-19 01:17:15 +0000
  • 25adc6b802
         log git repository status Fabian Egli 2024-10-19 00:28:41 +0200
  • b584f613cd
         add some signposts to the log Fabian Egli 2024-10-19 00:13:09 +0200
  • b3068a8d90
         add some signposts Fabian Egli 2024-10-19 00:12:25 +0200
  • a46d06d436
         log changes made by the action in main.yml Fabian Egli 2024-10-19 00:04:15 +0200
  • cfaade6e2f
         log the diff in the update action daily_update.yml Fabian Egli 2024-10-19 00:01:15 +0200
  • 04f630f7f8
         Merge pull request #45 from glyn/faq-update Cory Dransfeldt 2024-10-18 06:35:47 -0700
  • 898c8ab82d
         Merge pull request #46 from isagalaev/case-insensitive-sorting Glyn Normington 2024-10-18 07:57:56 +0100
  • 7bb5efd462
         Sort the content case-insensitively before dumping to JSON Ivan Sagalaev 2024-10-17 21:08:43 -0400
  • e6bb7cae9e Augment the "why" FAQ Glyn Normington 2024-10-17 12:27:05 +0100
  • b229f5b936 Re-order the FAQ Glyn Normington 2024-10-17 12:25:54 +0100
  • b1491d2694 Daily update from Dark Visitors dark-visitors 2024-10-09 01:17:37 +0000
  • 9be286626d Merge pull request #43 from lxjv/main ai.robots.txt 2024-10-08 02:30:17 +0000
  • 01993b98c3
         Merge pull request #43 from lxjv/main Glyn Normington 2024-10-08 03:30:07 +0100
  • dc15afe847
         Update robots.json with Claude respect link Laker Turner 2024-10-07 17:38:01 +0100
  • 6da804e826 chore: add ISSCyberRiskCrawler v1.18 ai.robots.txt 2024-09-30 23:50:18 +0000
  • 9c2394f23b
         chore: add ISSCyberRiskCrawler Cory Dransfeldt 2024-09-30 16:25:20 -0700
  • 6d9ce1d62a chore: add sidetrade bot v1.17 ai.robots.txt 2024-09-28 20:58:18 +0000
  • 6a988be27f
         chore: add sidetrade bot Cory Dransfeldt 2024-09-28 13:58:00 -0700
  • 632e9d6510 Daily update from Dark Visitors ai.robots.txt 2024-09-28 01:17:19 +0000
  • 7851cea4fd Daily update from Dark Visitors dark-visitors 2024-09-27 01:18:04 +0000
  • 75343c790e
         Merge pull request #38 from urvish-p80/main Glyn Normington 2024-09-27 01:26:04 +0100
  • 44d975c799 Merge pull request #42 from commoncrawl/main ai.robots.txt 2024-09-27 00:21:49 +0000
  • 2f67e77ddb
         Merge pull request #42 from commoncrawl/main Glyn Normington 2024-09-27 01:21:37 +0100
  • a6de89e6bd feat: make CCBot entry more accurate Greg Lindahl 2024-09-26 21:41:28 +0000
  • 60bdfa7eb3
         Merge pull request #41 from cityrolr/patch-1 Cory Dransfeldt 2024-09-24 12:53:52 -0700
  • af05890b07
         Update README.md Julian Mair 2024-09-23 23:27:27 +0200
  • d2cd37442c
         Update table-of-bot-metrics.md Michael Davey 2024-09-23 22:26:05 +0100
  • c1e6265ef4
         Update robots.json Michael Davey 2024-09-23 18:03:19 +0100
  • 2c68b9a88b
         Update robots.txt Michael Davey 2024-09-23 17:59:44 +0100
  • 0106d4b15a
         Add additional resource - README.md Urvish Patel 2024-09-23 08:19:27 -0400
  • 1abf68b107
         Adding dataprovider.com's spider Gregory Hammond 2024-09-14 23:24:43 +0000
  • 6b8d7f5890 Daily update from Dark Visitors ai.robots.txt 2024-09-09 01:16:21 +0000
  • 5963cbf9f7 Daily update from Dark Visitors dark-visitors 2024-09-08 01:19:31 +0000
  • b15b8062ce
         Merge pull request #36 from cramforce/patch-1 Glyn Normington 2024-09-08 01:26:07 +0100
  • 809851ae88
         Add instructions for AI bot blocking on Vercel Malte Ubl 2024-09-07 15:59:25 -0700
  • 1c1b423684 chore: add iaskspider/2.0 v1.16 ai.robots.txt 2024-09-07 02:05:43 +0000
  • 8373294404
         chore: add iaskspider/2.0 Cory Dransfeldt 2024-09-06 19:05:26 -0700
  • b30ca5f193
         Merge pull request #35 from nisbet-hubbard/patch-7 Cory Dransfeldt 2024-09-02 18:40:57 -0700
  • fb5c995243 Daily update from Dark Visitors ai.robots.txt 2024-09-03 01:12:57 +0000
  • 7151f6c569 Removing previously generated files ai.robots.txt 2024-09-03 01:12:56 +0000
  • cc18b8617c
         Update main.yml nisbet-hubbard 2024-09-03 07:48:48 +0800
  • c9325c9e18 Daily update from Dark Visitors ai.robots.txt 2024-09-02 01:15:07 +0000
  • 567bd00aec Removing previously generated files ai.robots.txt 2024-09-02 01:15:07 +0000
  • 543e993b08 Daily update from Dark Visitors ai.robots.txt 2024-09-01 01:24:53 +0000
  • 01589718df Removing previously generated files ai.robots.txt 2024-09-01 01:24:52 +0000
  • 9a7f556d87 Daily update from Dark Visitors ai.robots.txt 2024-08-31 01:13:04 +0000
  • 9a4ebb57ee Removing previously generated files ai.robots.txt 2024-08-31 01:13:04 +0000
  • 054c97ad4f Daily update from Dark Visitors ai.robots.txt 2024-08-30 01:13:29 +0000
  • b2970316d8 Removing previously generated files ai.robots.txt 2024-08-30 01:13:29 +0000
  • 008a34ceb4 chore: add ai2bot v1.15 ai.robots.txt 2024-08-29 03:07:52 +0000
  • 3bce634e4a Removing previously generated files ai.robots.txt 2024-08-29 03:07:51 +0000
  • 0f8723558f
         chore: add ai2bot Cory Dransfeldt 2024-08-28 20:07:32 -0700
  • 6dc900b582 Daily update from Dark Visitors ai.robots.txt 2024-08-29 01:13:19 +0000
  • 71eefcdb05 Removing previously generated files ai.robots.txt 2024-08-29 01:13:19 +0000
  • 1d417ffab9 Daily update from Dark Visitors ai.robots.txt 2024-08-28 01:12:35 +0000
  • 00ef18f93c Removing previously generated files ai.robots.txt 2024-08-28 01:12:35 +0000
  • 84a2376f65 Daily update from Dark Visitors v1.14 ai.robots.txt 2024-08-27 01:12:20 +0000
  • 699862f4bd Removing previously generated files ai.robots.txt 2024-08-27 01:12:19 +0000
  • ccec3eef15 Daily update from Dark Visitors ai.robots.txt 2024-08-26 01:11:41 +0000
  • 6cb9bc8ebf Removing previously generated files ai.robots.txt 2024-08-26 01:11:40 +0000
  • 42a7ca7eda Daily update from Dark Visitors ai.robots.txt 2024-08-25 01:16:28 +0000
  • 907866301f Removing previously generated files ai.robots.txt 2024-08-25 01:16:27 +0000
  • b202b9e1e3 Daily update from Dark Visitors ai.robots.txt 2024-08-24 01:09:29 +0000
  • ac1250cfa5 Removing previously generated files ai.robots.txt 2024-08-24 01:09:29 +0000
  • d95f2e8072 Daily update from Dark Visitors ai.robots.txt 2024-08-23 01:10:54 +0000
  • 61d851baf5 Removing previously generated files ai.robots.txt 2024-08-23 01:10:53 +0000
  • 7bfc1647a8 Daily update from Dark Visitors dark-visitors 2024-08-22 01:11:43 +0000
  • 3580a7096f Daily update from Dark Visitors ai.robots.txt 2024-08-21 01:10:11 +0000
  • fad335178f Removing previously generated files ai.robots.txt 2024-08-21 01:10:10 +0000
  • 358df0833e Daily update from Dark Visitors ai.robots.txt 2024-08-20 01:10:11 +0000
  • 7e0dd921db Removing previously generated files ai.robots.txt 2024-08-20 01:10:11 +0000
  • 591a99c320 Daily update from Dark Visitors ai.robots.txt 2024-08-19 01:11:49 +0000
  • 394e447c78 Removing previously generated files ai.robots.txt 2024-08-19 01:11:49 +0000
  • ab4a6547f6
         Merge branch 'main' of github.com:ai-robots-txt/ai.robots.txt Cory Dransfeldt 2024-08-18 11:34:47 -0700
  • 1d3194f75d
         chore: update readme Cory Dransfeldt 2024-08-18 11:34:43 -0700
  • 2363e57608
         chore: minor update Cory Dransfeldt 2024-08-18 11:34:08 -0700
  • b8e68c12f3 Daily update from Dark Visitors ai.robots.txt 2024-08-18 01:14:50 +0000
  • 60ff792ba9 Removing previously generated files ai.robots.txt 2024-08-18 01:14:49 +0000
  • 3afcefdff5 Daily update from Dark Visitors ai.robots.txt 2024-08-17 01:08:17 +0000
  • 558d5871b2 Removing previously generated files ai.robots.txt 2024-08-17 01:08:17 +0000
  • 2a075cb2f1 Daily update from Dark Visitors ai.robots.txt 2024-08-16 01:10:14 +0000