Commit graph

  • 3ef9cb7ce4 Removing previously generated files ai.robots.txt 2024-08-16 01:10:13 +0000
  • 5937434aff Daily update from Dark Visitors dark-visitors 2024-08-15 01:07:15 +0000
  • 407b9e12e6
    chore: sort output Cory Dransfeldt 2024-08-14 17:10:29 -0700
  • bc66d10afd
    chore: update faq Cory Dransfeldt 2024-08-14 09:21:26 -0700
  • df5b6ef647 Daily update from Dark Visitors ai.robots.txt 2024-08-14 01:11:03 +0000
  • 2c8ed062b9 Removing previously generated files ai.robots.txt 2024-08-14 01:11:02 +0000
  • 2e8e8af8e4 Daily update from Dark Visitors ai.robots.txt 2024-08-13 01:12:03 +0000
  • f1d0c5b1fe Removing previously generated files ai.robots.txt 2024-08-13 01:12:02 +0000
  • 53a39b2f71 Daily update from Dark Visitors ai.robots.txt 2024-08-12 01:12:23 +0000
  • 274d48b8f0 Removing previously generated files ai.robots.txt 2024-08-12 01:12:23 +0000
  • 6472e07f09 Daily update from Dark Visitors ai.robots.txt 2024-08-11 01:16:04 +0000
  • cb98669cc2 Removing previously generated files ai.robots.txt 2024-08-11 01:16:03 +0000
  • 7662d06eb3
    Merge pull request #33 from nisbet-hubbard/patch-6 Cory Dransfeldt 2024-08-09 19:42:36 -0700
  • 53449ad1bd Daily update from Dark Visitors ai.robots.txt 2024-08-10 01:10:53 +0000
  • 4242f8cc7b Removing previously generated files ai.robots.txt 2024-08-10 01:10:53 +0000
  • 46540633ba
    Update README.md nisbet-hubbard 2024-08-10 08:22:28 +0800
  • 21e5cd96a9 Daily update from Dark Visitors ai.robots.txt 2024-08-09 01:11:12 +0000
  • ed7d7d3fdf Removing previously generated files ai.robots.txt 2024-08-09 01:11:11 +0000
  • 57f006150b Daily update from Dark Visitors ai.robots.txt 2024-08-08 01:10:13 +0000
  • 40f9325a4f Removing previously generated files ai.robots.txt 2024-08-08 01:10:12 +0000
  • 0122dea1e9 Merge pull request #32 from ChenghaoMou/main ai.robots.txt 2024-08-07 22:40:24 +0000
  • 663b85cc07 Removing previously generated files ai.robots.txt 2024-08-07 22:40:24 +0000
  • 5c8b4593f4
    Merge pull request #32 from ChenghaoMou/main Adam Newbold 2024-08-07 18:40:13 -0400
  • 6f96795edc restore cron Chenghao Mou 2024-08-07 12:43:44 +0100
  • ab17662f96 Daily update from Dark Visitors ai.robots.txt 2024-08-07 11:41:00 +0000
  • 8738c66c65 Removing previously generated files ai.robots.txt 2024-08-07 11:40:59 +0000
  • b00067bc86 restore files deleted by failed workflow and fix main commit message Chenghao Mou 2024-08-07 12:36:21 +0100
  • 4a63c482c4 Removing previously generated files ai.robots.txt 2024-08-07 11:31:02 +0000
  • 366e49dc6d restore files deleted by failed workflow and fix main commit message Chenghao Mou 2024-08-07 12:21:40 +0100
  • aaa55594e1 Removing previously generated files ai.robots.txt 2024-08-07 11:13:16 +0000
  • fbebbbfefb restore files deleted by failed workflow Chenghao Mou 2024-08-07 12:02:46 +0100
  • 6a275366be Daily update from Dark Visitors dark-visitors 2024-08-07 10:50:45 +0000
  • 09c6b78b46 fix job dependency Chenghao Mou 2024-08-07 11:45:37 +0100
  • d4f34363ec Removing previously generated files ai.robots.txt 2024-08-07 10:40:50 +0000
  • 30eaff1447 call main after update ai.robots.txt 2024-08-07 10:32:13 +0000
  • bd3eee7a30 Removing previously generated files ai.robots.txt 2024-08-07 10:32:12 +0000
  • 944bee0f56 call main after update Chenghao Mou 2024-08-07 11:31:58 +0100
  • cebf809391 Daily update from Dark Visitors dark-visitors 2024-08-07 00:14:26 +0000
  • 3d4bf2c3db restore original robots.json ai.robots.txt 2024-08-06 18:50:54 +0000
  • d6a5e8cd81 Removing previously generated files ai.robots.txt 2024-08-06 18:50:53 +0000
  • 4cf82b703f restore original robots.json Chenghao Mou 2024-08-06 19:50:38 +0100
  • 0b6eba8dd5 skip push if no change Chenghao Mou 2024-08-06 19:40:43 +0100
  • 379c339f97 skip push if no change Chenghao Mou 2024-08-06 19:40:01 +0100
  • 01edb6c78c
    Merge branch 'ai-robots-txt:main' into main Chenghao Mou 2024-08-06 19:35:03 +0100
  • 2a3685385c restrict scope Chenghao Mou 2024-08-06 19:33:49 +0100
  • 85275e55b8
    Merge pull request #31 from glyn/addfaq Cory Dransfeldt 2024-08-06 11:15:29 -0700
  • 8c6482fb45 restore the cron Chenghao Mou 2024-08-06 18:12:41 +0100
  • 63c7e742c3 Daily update from Dark Visitors dark-visitors 2024-08-06 16:54:29 +0000
  • 55e92f4324 update existing ones Chenghao Mou 2024-08-06 17:48:06 +0100
  • 52d54cf127 restore the cron Chenghao Mou 2024-08-06 17:28:00 +0100
  • fdd261dad4 Daily update from Dark Visitors dark-visitors 2024-08-06 16:27:02 +0000
  • 6d2285f5e0 Add FAQ ai.robots.txt 2024-08-06 16:21:01 +0000
  • 83d9397f17 Removing previously generated files ai.robots.txt 2024-08-06 16:21:00 +0000
  • b4d25bf0cb Add FAQ Glyn Normington 2024-08-06 17:20:26 +0100
  • 8ab1e30a6c test workflow Chenghao Mou 2024-08-06 17:12:26 +0100
  • 192bf67631 add dark visitor workflow Chenghao Mou 2024-08-06 17:01:56 +0100
  • e12ddc0f42 Merge pull request #29 from jbowdre/dev ai.robots.txt 2024-08-06 15:44:54 +0000
  • b54e274bbc Removing previously generated files ai.robots.txt 2024-08-06 15:44:53 +0000
  • 3e91a84d11
    Merge pull request #29 from jbowdre/dev Cory Dransfeldt 2024-08-04 16:04:59 -0700
  • b0a93aeb70 only build on changes to robots.json John Bowdre 2024-08-04 17:45:18 -0500
  • eb924b9856 Merge pull request #28 from jsheard/patch-2 v1.13 ai.robots.txt 2024-08-04 21:54:17 +0000
  • 1cfc071498 Removing previously generated files ai.robots.txt 2024-08-04 21:54:16 +0000
  • 24c3509a6e
    Merge pull request #28 from jsheard/patch-2 Cory Dransfeldt 2024-08-04 14:54:06 -0700
  • c2f177870f Merge pull request #27 from jsheard/patch-1 ai.robots.txt 2024-08-04 21:53:48 +0000
  • 0072b8f5f0 Removing previously generated files ai.robots.txt 2024-08-04 21:53:47 +0000
  • 9c7257e7cf
    Merge pull request #27 from jsheard/patch-1 Cory Dransfeldt 2024-08-04 14:53:36 -0700
  • 8dbbdbf44c
    Add Cloudflares first-party scraper blocking to FAQ Joshua Sheard 2024-08-04 21:38:02 +0100
  • 146fd4ffba
    Fix Imagesift user agent Joshua Sheard 2024-08-04 21:33:04 +0100
  • c7b781034e chore: restore FriendlyCrawler + ImageSift v1.12 ai.robots.txt 2024-08-04 19:29:01 +0000
  • 9a8fa66772 Removing previously generated files ai.robots.txt 2024-08-04 19:29:00 +0000
  • 1ca936ce11
    chore: restore FriendlyCrawler + ImageSift Cory Dransfeldt 2024-08-04 12:28:48 -0700
  • 8de5bc8e01 Merge pull request #25 from mirium999/add_icc_crawler ai.robots.txt 2024-08-04 01:21:56 +0000
  • 8c632e1ba4 Removing previously generated files ai.robots.txt 2024-08-04 01:21:55 +0000
  • 8d4d52cdab
    Merge pull request #25 from mirium999/add_icc_crawler Adam Newbold 2024-08-03 21:21:45 -0400
  • 5826c18909 Add ICC-Crawler Mirium999 2024-08-04 10:11:25 +0900
  • ffbad453f3 Merge pull request #24 from nisbet-hubbard/patch-5 ai.robots.txt 2024-08-03 14:27:47 +0000
  • b1907d86be Removing previously generated files ai.robots.txt 2024-08-03 14:27:46 +0000
  • 55c585e9e3
    Merge pull request #24 from nisbet-hubbard/patch-5 Cory Dransfeldt 2024-08-03 07:27:37 -0700
  • 2b56c72bac
    Update FAQ.md nisbet-hubbard 2024-08-03 14:27:25 +0800
  • b24e5cb3bb
    Update FAQ.md nisbet-hubbard 2024-08-03 14:12:50 +0800
  • 74b1502839
    Update FAQ.md nisbet-hubbard 2024-08-03 14:04:58 +0800
  • d8de1ebdd5 chore: contribution note ai.robots.txt 2024-08-02 16:32:00 +0000
  • 9d8d3de8ed Removing previously generated files ai.robots.txt 2024-08-02 16:31:59 +0000
  • 349c35eed6
    chore: contribution note Cory Dransfeldt 2024-08-02 09:31:48 -0700
  • b144225ece chore: drop in additional data ai.robots.txt 2024-08-01 22:33:23 +0000
  • 06b950bce9 Removing previously generated files ai.robots.txt 2024-08-01 22:33:23 +0000
  • b20dfec1e4
    chore: drop in additional data Cory Dransfeldt 2024-08-01 15:33:07 -0700
  • f18f0d99b9 chore: remove test data ai.robots.txt 2024-08-01 22:29:02 +0000
  • 747cc834c4 Removing previously generated files ai.robots.txt 2024-08-01 22:29:01 +0000
  • efabf3e721
    chore: remove test data Cory Dransfeldt 2024-08-01 15:25:55 -0700
  • 1fdc79dacb Adding GitHub Action Adam Newbold 2024-08-01 18:17:19 -0400
  • 17a84f2c2d
    chore: update robots table Cory Dransfeldt 2024-08-01 15:06:49 -0700
  • 6c596a50ea
    chore: move FAQ into repo v1.11 Cory Dransfeldt 2024-08-01 07:53:43 -0700
  • 6a8e7a8eb0
    Merge pull request #22 from nisbet-hubbard/patch-4 Cory Dransfeldt 2024-08-01 07:49:30 -0700
  • df89722038
    Add PetalBot (and facebookexternalhit?) nisbet-hubbard 2024-07-31 18:27:29 +0800
  • fa7b64ae4b
    chore: add Scrapy v1.10 Cory Dransfeldt 2024-07-30 10:28:46 -0700
  • 55b4505e30
    chore: add Timpibot v1.9 Cory Dransfeldt 2024-07-29 12:38:22 -0700
  • d49e860b74
    chore: add VelenPublicWebCrawler v1.8 Cory Dransfeldt 2024-07-29 12:12:42 -0700
  • 6e323554c6
    chore: add Meta-ExternalAgent v1.7 Cory Dransfeldt 2024-07-29 08:27:31 -0700
  • 1df6d3bec1
    Add meta-externalagent nisbet-hubbard 2024-07-29 20:04:59 +0800