source: trunk/autoquest-plugin-php/data/robotfilter.txt @ 1447

Last change on this file since 1447 was 584, checked in by sherbold, 12 years ago
  • moved PHP plugin implementation from quest-ui-core to quest-php-guitar
  • Property svn:mime-type set to text/plain
File size: 885 bytes
Line 
1findlinks
2discobot
3Googlebot
4Slurp
5YandexBot
6Spider
7ScholarUniverse
8Baiduspider
9Exabot
10Robot
11MetaGer-Bot
12YandexImages
13Gigabot
14SiteBot
15bingbot
16Ezooms
17Jeeves/Teoma
18msnbot
19DotBot
20changedetection.com/bot.html
21FAST Enterprise Crawler 6
22psbot
23http://ws.daum.net/aboutWebSearch.html
24NerdByNature.Bot
25Sogou web spider
26ssearch_bot
27Purebot
28http://www.icjobs.de
29scoutjet
30Netcraft Web Server Survey
31TurnitinBot
32ia_archiver
33MJ12bot
34Domnutch-Bot
35Eurobot
36GarlikCrawler
37CMS Crawler
38MSIECrawler
39NaverBot
4080legs
41AhrefsBot
42SISTRIX Crawler
43NetcraftSurveyAgent
44Search17Bot
45Semager
46YandexFavicons
47heritrix
48suggybot
49Netluchs
50Ocelli
51PHPCrawl
52SolomonoBot
53Sosospider
54Xerka WebBot
55YahooCacheSystem
56Xenu Link Sleuth
57cmsworldmap
58suchen.de
59amaredo.com/de/suche.html
60ibot
61w3af.sourceforge.net
62w3af.sf.net
63yacybot
64larbin2
65t-h-u-n-d-e-r-s-t-o-n-e
66sqlmap
Note: See TracBrowser for help on using the repository browser.