Unknown Botnet Using Mozilla/5.0 (X11; Linux x86_ User Agent Ignoring Crawl Delay on WordPress Sites

A mysterious botnet, identified by the user-agent string Mozilla/5.0 (X11; Linux x86_64), is aggressively scraping WordPress websites at an alarming rate, flagrantly disregarding the crawl delay directives set in the robots.txt file. The botnet has been found to be causing significant server strain, bypassing web administrators' efforts to control access and protect their site resources.

Despite the bot's user-agent string mimicking a legitimate Linux browser, its bad behavior is anything but lawful. Not only does it ignore robots.txt instructions, but it also carries out non-stop scraping activities that severely impact the performance and security of affected websites. The unregulated scraping can lead to slowed website loading times and increased bandwidth consumption.

IP Addresses and ASN Sources Involved

The IP addresses behind this botnet appear to originate from multiple autonomous system numbers (ASNs), primarily hosted by less-reputable service providers. The IPs exhibit coordinated behavior, systematically scraping WordPress websites with no regard for the implemented crawl-delay directives.

The ASNs and their affiliated IP addresses are:

• ASN 13886 - Cloud-South

  
  

• ASN 21769 - AS-COLOAM

  
  

• ASN 263735 - Sociedad Buena Hosting, S.A.

  
  

• ASN 52393 - Corporacion Dana S.A.

185.199.117.126 216.10.7.2 104.233.51.70 104.249.4.61 186.179.1.64 181.177.71.218 185.195.215.202 185.193.73.239 104.239.116.255 185.199.118.137 185.193.72.244 216.10.3.36 186.179.11.6 181.177.70.128 104.239.115.2 186.179.10.14 104.249.5.146 185.199.116.245 104.233.54.62 185.207.97.112 185.199.116.218 181.177.71.59 104.249.0.172 104.233.49.164 185.188.78.129 216.10.2.44 104.233.48.205 216.10.6.143 104.249.1.226 185.196.191.240 67.227.122.211 185.195.221.166 181.177.79.165 186.179.25.11 185.199.117.248 185.195.220.198 104.239.114.195 181.177.66.111 67.227.127.113 185.205.196.57 199.168.122.42 186.179.27.9 185.193.75.105 216.10.3.10 216.10.2.161 185.188.77.24 104.233.48.70 185.188.79.15 186.179.2.186 181.177.72.76 216.10.6.205 186.179.13.13 181.177.78.227 181.177.72.12 181.177.79.141 186.179.25.210 104.233.49.36 104.249.3.157 104.239.117.199 104.233.48.52 104.233.51.106 216.10.3.78 216.10.0.49 185.207.99.161 67.227.120.127 67.227.121.26 104.233.55.225 104.249.3.194 185.195.220.41 181.177.71.84 104.233.48.111 104.249.2.46 181.177.67.170 104.249.2.195 186.179.13.80 67.227.124.74 104.239.116.151 104.239.119.180 185.195.221.180 104.249.2.99 104.239.114.190 104.239.117.112 181.177.71.49 67.227.121.109 199.168.121.93 185.195.223.14 181.177.67.30 181.177.76.63 181.177.77.79 181.177.66.29 181.177.77.197 186.179.24.252 185.196.188.94 181.177.76.62 216.10.7.75 181.177.68.33 186.179.11.165 181.177.71.188 185.195.213.87 185.193.74.97 67.227.122.57 185.196.189.63 216.10.1.142 199.168.122.193 186.179.2.117 181.177.72.134 181.177.66.212 185.188.77.122 185.207.96.121 199.168.121.246 104.249.0.31 185.195.222.160

Potential Mitigation Strategies

Web administrators are urged to monitor their logs for suspicious activity involving the Mozilla/5.0 (X11; Linux x86_64) user-agent string and IPs associated with the above ASNs. Immediate actions to consider include:

• Blocking or rate-limiting the offending IPs through firewall rules.

  
  

• Implementing CAPTCHA systems for suspicious traffic.

  
  

• Using bot protection plugins or services such as Cloudflare to prevent excessive scraping.

  
  

The persistence of this botnet highlights the importance of continually refining web security measures to protect digital assets from unauthorized data scraping and potential attacks.

This unknown botnet's activity is a pressing issue for WordPress site owners, and if left unchecked, its presence could lead to server overloads, data leaks, or even downtime for many websites.

Interestingly, despite the aggressive nature of this botnet, none of the associated IPs have been flagged or reported in popular databases like AbuseIPDB. Even more surprising, these IPs are not natively blocked by Cloudflare’s managed rules, which typically catch such malicious behavior early on. This suggests that the botnet is currently flying under the radar, operating in a gray zone of anonymity.

The lack of detection and reporting raises concerns about how sophisticated and stealthy this botnet may be. For now, it seems to be an entirely unknown entity in the cybersecurity landscape. However, it does seem look like it is scraping WordPress content.