站长资源网站运营
搜索引擎爬虫蜘蛛的USERAGENT大全
简介百度爬虫 * Baiduspider+(+http://www.baidu.com/search/spider.htm”) google爬虫 * Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html
百度爬虫
* Baiduspider+(+http://www.baidu.com/search/spider.htm”)
google爬虫
* Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
* Googlebot/2.1 (+http://www.googlebot.com/bot.html)
* Googlebot/2.1 (+http://www.google.com/bot.html)
雅虎爬虫(分别是雅虎中国和美国总部的爬虫)
*Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html”)
*Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp”)
新浪爱问爬虫
*iaskspider/2.0(+http://iask.com/help/help_index.html”)
*Mozilla/5.0 (compatible; iaskspider/1.0; MSIE 6.0)
搜狗爬虫
*Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07″)
*Sogou Push Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07″)
网易爬虫
*Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/”; )
MSN爬虫
*msnbot/1.0 (+http://search.msn.com/msnbot.htm”)
* Baiduspider+(+http://www.baidu.com/search/spider.htm”)
google爬虫
* Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
* Googlebot/2.1 (+http://www.googlebot.com/bot.html)
* Googlebot/2.1 (+http://www.google.com/bot.html)
雅虎爬虫(分别是雅虎中国和美国总部的爬虫)
*Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html”)
*Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp”)
新浪爱问爬虫
*iaskspider/2.0(+http://iask.com/help/help_index.html”)
*Mozilla/5.0 (compatible; iaskspider/1.0; MSIE 6.0)
搜狗爬虫
*Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07″)
*Sogou Push Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07″)
网易爬虫
*Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/”; )
MSN爬虫
*msnbot/1.0 (+http://search.msn.com/msnbot.htm”)
最新资源
- [ABC唱片]-《雪莉—蓝》[SACD-008][SACD][WAV+C
- 黎瑞恩.2002-理想的日子精选2CD【环球】【WAV+C
- 【驰放沙发(J)】VA-2024-WinterTime,Vol.12(FLA
- 雷婷《梦里蓝天HQⅡ》 2023头版限量编号低速原抓
- 韩红《青藏高原+红+韩红精选集》3CD[WAV分轨][1
- 高胜美《经典金选》1991-1994 4CD[WAV整轨][1.
- 【迷幻电音】AstralWaves-2023-Singles,Remixes
- 中唱唱片群星《好歌珍藏-游子情深》2CDWAV
- 张国荣.2011-极品天碟LPCD45II(雨果版)【环球】
- 张学友《醒着做梦》[DFF][1.1G]