Sustaining WordPress XML-RPC Attack Traffic

I’ve been experiencing the same increased frequency of attacks against WordPress’ integrated XML-RPC service in recent months as reported by many other site operators. The attacks have been covered well elsewhere, but I wanted to chronicle what I’m seeing and share some remarks.

Background

In late 2012, starting with version 3.5, WordPress core developers enabled the previously experimental XML-RPC server by default. This opened up a window of opportunity for hackers, who I can only imagine must have been delighted. The ability to direct attacks against xmlrpc.php conferred many benefits compared to the prior attack surface presented by WordPress consisting primarily of wp-login.php and wp-admin. The attacks became faster, since the XML-RPC requests and responses involved were more compact and drew less server overhead. They became stealthier, since fewer administrators were concerned with patrolling xmlrpc.php traffic, particularly in the feature’s infancy and prior to the advent of the waves of attacks to come.

Most importantly, attacks became nontrivial for administrators to counter without throwing off unwanted side effects. The exposure of wp-login.php and wp-admin can be controlled trivially via .htaccess. On this site for example, I restrict access to a select whitelist of allowed IP addresses corresponding to my own systems, and all other (e.g. rogue) requesters are efficiently denied with HTTP 403 Forbidden. WordPress’ xmlrpc.php cannot be blocked off in such blunt fashion. The XML-RPC service offers up dozens of infrastructure methods, some of which are quite desirable to maintain, including pingbacks and trackbacks. Denying requests to xmlrpc.php from the network at large or entirely disabling the XML-RPC subsystem, while it works, has the effect of blocking legitimate procedure calls. For these reasons, initially, xmlrpc.php would have landed widely unguarded.

Threats soon evolved to target and exploit this opportunity. A variety, rather than a single type, of modes of attack on WordPress’ xmlrpc.php have been described. These include run-of-the-mill spam, direct denial of service attacks (said to be fixed in WordPress 3.9.2) designed to overwhelm individual sites by exhausting resources, distributed denial of service attacks designed to take down individual sites by reflecting requests through thousands of others, and password dictionary attacks attempting to gain unauthorized access by automatically throwing thousands of commonly used login credentials at authenticating request methods. The latter form of attack follows essentially the same modus operandi, and in fact probably employs the exact same password dictionary, as hackers would use against remote shell services like SSH (a threat I’ve written plenty about in the past).

Evidence of being targeted

It is that latter form of attack, the dictionary attack, that I believe is washing up here in my logs in the past few months.

I haven’t captured the XML-RPC request content for inspection, but just scanning my access_log‘s and analyzer reports I can make a heuristic call based on the shape of the traffic. It comes from really just a handful of source IP addresses in the usual suspect countries – so few that I could block them manually if I felt like babysitting them – thousands of requests from each, sustained over the course of weeks and months. The total volume of traffic is on the same order of magnitude as what I trap out 403 as mentioned above. The requests and responses are small and always of virtually the same byte size. These observations suggest attackers looking to brute force their way in rather than deny availability.

Here’s a chart of my xmlrpc.php traffic by month, to date, back to the start of 2013:

Yeah, that looks natural. Before three months ago, the XML-RPC server saw but an insignificant trickle of activity – presumably valid traffic, or a negligible amount of traffic at any rate. Now it’s racking up close to half of my page views. (The analyzer I’m using, awstats, counts each request as viewed content.) In absolute terms I’m seeing several tens of thousands of attempts per month. These intruders will never gain entry because the credentials I use are not dictionary-able. But that doesn’t mean I’m thrilled with giving them unlimited tries.

Countering this attack

The good news is this attack can be met with an effective response without infringing upon the legitimate functionality of xmlrpc.php.

All the authenticating methods served by the XML-RPC server call the core function wp_authenticate to do login credential verification. (See /wp-includes/class-wp-xmlrpc-server.php.) Which means on catching invalid credentials they should trigger the hookable wp_login_failed action. (See /wp-includes/pluggable.php.) Which means we can hook the wp_login_failed action with a plugin like WP fail2ban. Which means we can sit back and watch while the offending IP addresses get themselves blacklisted by fail2ban at iptables. Meanwhile, xmlrpc.php would remain in service for valid procedure calls.

I’m reading reports from some administrators and certain security plugin developers that wp_login_failed is not being called on XML-RPC authentication failures like it’s supposed to be. As a fallback, some are scripting bans against straight up too-frequent invocations of xmlrpc.php, which would be sufficient if imperfect. The source code is pretty black and white about it, but it may require further testing.