Search Engine Spiders and Robots problem

Joined
Dec 15, 2010
Messages
1
Reaction score
0
Points
1
I have a problem from spider and robots that keep visit my website every second. Any webmaster here have face this problem like me?

I have search from the ip address (211.43.152.54) that these robot from who is domain which the result is from Korea, Republic. Moreover I have a question about these robot that could it harm my website?
robot+problem.jpg



Thank you in advance for any suggestion.
 

Raz0rEdge

Well-known member
Staff member
Moderator
Joined
Jul 17, 2009
Messages
15,771
Reaction score
2,111
Points
113
Location
MA
Your Mac's Specs
2022 Mac Studio M1 Max, 2023 M2 MBA
How do you know it's a Spider for a search engine that's doing that? Did you actually track down the IP address to belong to any particular search engine in Korea?

Any sane search engine spider would come by every few hours on a site that doesn't change often and more frequently if it's a fast changing site.

Hitting your site every second sounds more like a DOS (Denial of Service) attack than a benign spider trying to catalog your site's information..


As suggested by @webguy, block that particular IP in your .htaccess file to avoid causing problems with your hosting service who could suspend your account for this sort of activity..

Regards
 

Shop Amazon


Shop for your Apple, Mac, iPhone and other computer products on Amazon.
We are a participant in the Amazon Services LLC Associates Program, an affiliate program designed to provide a means for us to earn fees by linking to Amazon and affiliated sites.
Top