Hello, I'd like to blocks all bad bots from crawling my website such as Ahrefs, SEOPROFILER...
I see some articles guiding me to search bots ips then block them, but very hard to know all bots
Do you know how to block all of them?
Yes, you just simple add these lines to top of your htaccess. I will block all bad bot as we know well. You can add more if you like
SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot SetEnvIfNoCase User-Agent .*exabot.* bad_bot SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot SetEnvIfNoCase User-Agent .*dotbot.* bad_bot SetEnvIfNoCase User-Agent .*gigabot.* bad_bot SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot SetEnvIfNoCase User-Agent .*sitebot.* bad_bot SetEnvIfNoCase User-Agent .*sitebot.* bad_bot SetEnvIfNoCase User-Agent .*BLEXbot.* bad_bot SetEnvIfNoCase User-Agent .*Blekkobot.* bad_bot SetEnvIfNoCase User-Agent .*SEOkicks-Robot.* bad_bot <Limit GET POST HEAD> Order Allow,Deny Allow from all Deny from env=bad_bot </Limit>