Baidu Spider Goes Wild! A DDoS-like Cyberattack from Chongqing, China

Starting at some unknown point yesterday, the Vedfolnir website suddenly became very unstable, with connections frequently delayed and error code 500 (similar to 404) constantly appearing. After the network administrator checked the system, it was found that CPU usage had surged to 100% full speed and had maintained that high load for several hours.

To resolve this abnormal situation, we first restarted the website service, hoping this would fix the problem and restore normal operation. However, this fix was only a flash in the pan; a few seconds later, the website crashed again. This made me begin to suspect that some external force was targeting our website.

After some inspection of the system, it turned out that several groups of IPs from the Chongqing region of mainland China (listed in the table below) were frantically scanning the ports of our server. This was what caused the system’s CPU usage to surge and made the website extremely unstable.

• 183.69.137.80
• 183.69.137.71
• 183.69.137.65
• 183.69.137.91
• 183.69.137.94
• And many others too numerous to list

Using a Firewall to Solve 100% Server CPU

After going through a series of symptoms, the website administrator first took the crudest and simplest countermeasure against this DDoS[1]-like attack: blocking the source IPs at the firewall level.

But upon inspection, it was found that after blocking one group of IPs, countless others kept frantically scanning the server. For this reason, the only option was to pull out the big gun and block the entire network segment, using CIDR notation[2] written as “183.69.137.0/24”.

After blocking with the firewall, I had originally just wanted to observe how much malicious IP traffic there was. It was fine not to look, but once I looked, it was astonishing: at peak times it reached as many as 2,000 blocks every 5 minutes, and even during the trough it averaged as many as 1,000 visits every five minutes. That means that every minute, 200 to 400 IPs in succession were trying to scan the server.

Number of crawls by Baidu search engine’s web spider

Although this volume cannot yet be called a malicious DDoS denial-of-service attack, it has indeed achieved the same effect for a small server like ours. Even such a high volume of visits, and the bandwidth it consumes, adds up to no small cost.

Later, taking advantage of the firewall doing its job, once the server CPU finally dropped to a normal level, the author also tried to look up exactly which IPs were messing with us like this.

The result revealed an awkward fact: these IPs were actually not any malicious cyberattack at all, and could at most be called a “well-intentioned” cyberattack event (?).

These IPs all came from the spider (also known as a web crawler) of the well-known mainland Chinese search engine “Baidu”. Its original purpose was to scan website content to provide indexing for the search engine, but for some unknown reason, this group of spiders got excited and went wild as if they had been injected with chicken blood.

Moreover, we were not the only victims.

A quick search online reveals that website administrators across the strait often complain about this problem. Worse still, if you block that network segment’s IPs, you will be “punished” by the Baidu search engine, that is, your ranking will be lowered, which for a content website that relies on search engines for traffic is almost a catastrophe.

So, if you are a friend from mainland China, we suggest you hurry up and subscribe to our Vedfolnir website! Because we may very well be about to “disappear” from the Baidu engine too (tears).

Appendix

1. DDoS (Distributed Denial of Service) refers to a distributed denial-of-service attack, a common form of network attack. In a DDoS attack, the attacker uses multiple infected computers or devices (called “zombies”) to send a large number of requests to a target website or server, exceeding its processing capacity, thereby causing the website or server to fail to operate normally and preventing legitimate users from accessing the site or service. This type of attack usually uses distributed resources and sources, making it difficult to defend against and trace, and causing serious impact on networks and online businesses.
2. CIDR (Classless Inter-Domain Routing) notation is a concise and flexible method for representing IP addresses and network prefixes. It uses a slash symbol (/) followed by a number to indicate the number of valid bits in the network prefix, thereby specifying the size of the network.