WordPress get a lot of, somewhat deserved, bad press in regards to being able to stand up to large amounts of traffic. Especially on your run of the mill, everyday shared hosting.
Even on some VPS or dedicated hardware, people have had trouble keeping WordPress from falling over. There are some very simple fixes you can do to your server to make it almost bulletproof.
The below benchmarks were all run on a base 256mb Hudson Valley Host unmanaged VPS running Ubuntu 11.10 which is currently only $9/mo. This VPS has “burstable” memory to 512mb and 8 CPU cores.
I have used the following setup for all tests except where noted:
- Base install of apache2 / MySQL (apt-get update && apt-get upgrade && apt-get install phpmyadmin mysql-server)
- Apache and MySQL both running on the same server
- Apache mod-rewrite enabled for fancy permalinks
- Apache configured (by default) in /etc/apache2/apache2.conf:
<IfModule mpm_prefork_module>
StartServers 1
MinSpareServers 1
MaxSpareServers 5
MaxClients 10
MaxRequestsPerChild 0
</IfModule>
- /etc/apache2/sites-available/default edited to allow .htaccess overrides in /var/www
- WordPress 3.4.1 installed into /var/www with the default Twenty Eleven theme and no plugins enabled, except when noted
- Running siege on second server to benchmark (X is concurrency level of requests): siege -c X -t 1M http://wordpress.test/
The above command will send X number of simultaneous requests to the test URL for 1 minute.
Let’s get a simple baseline with 50 concurrent requests.
Transactions: 1778 hits
Availability: 100.00 %
Elapsed time: 59.25 secs
Data transferred: 3.91 MB
Response time: 1.16 secs
Transaction rate: 30.01 trans/sec
Throughput: 0.07 MB/sec
Concurrency: 34.77
Successful transactions: 1778
Failed transactions: 0
Longest transaction: 6.72
Shortest transaction: 0.26
As you can see from the “Transaction Rate” line, we were able to serve up 30.01 pages per second [ref]These are just therotical WordPress pages, and each WordPress page, of course, has references to images, CSS and JS files. While this number does not exactly correlate to active visitors your site can serve, by offloading as much of your static content as possible by using CloudFlare, Amazon S3, or other CDNs, you can get closer to these numbers in practice.[/ref], which by itself is pretty nice. But we can probably crank up the concurrency more. Let’s try 100 requests per second.
Transactions: 1835 hits
Availability: 100.00 %
Elapsed time: 60.10 secs
Data transferred: 4.04 MB
Response time: 2.66 secs
Transaction rate: 30.53 trans/sec
Throughput: 0.07 MB/sec
Concurrency: 81.10
Successful transactions: 1835
Failed transactions: 0
Longest transaction: 7.46
Shortest transaction: 0.27
Ok, this is statistically the same as the previous result. We’re going to say that 30 req/sec is our “all motor” limit.
Apache Pie Factory
This is where most sites that fall over are stuck. Basically Apache can only output 30 WordPress pages per second, if the server starts getting more than that, things get backed up.
And what happens when Apache gets backed up? It starts spinning up more processes to deal with the load. If you take a look at the snippet from the /etc/apache2/apache2.conf above, that is not the Ubuntu default. Hudson Valley Host was nice enough to put some sane defaults in there. This is what comes as default on most Ubuntu installs:
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>
The big issue is the MaxClients value. MaxClients is basically the number of processes Apache will spin up to deal with the incoming traffic. Each one of those individual processes uses memory. On my default install, each process was using about 16mb of memory, or a maximum total of 160mb. The above configuration will use up to 2400mb or over 2gb of RAM. As you can see, that’s a whole lot larger than the 256mb we have available on this server. It’s bigger than pretty much any affordable VPS and that’s just for Apache processes! If left like this, any decent amount of traffic will come along and kill your server. This is usually the case when people blame WordPress for not being able to handle traffic[ref]Also, be aware that each plugin you use adds memory overhead. The theme you’re using may have additional memory overhead as well. Then there’s MySQL which will want to start using more memory as you add more posts and get more comments. But we’re not going to worry about that right now, you’ll see why.[/ref].
How can we fix this?
Apart from changing your mpm_prefork_module settings so your server doesn’t kill itself (and you should do that as step 1), there are a few quick changes you can make to your server. We’ll start with the easiest (and least effective).
Install APC
Basically when Apache gets a request for a page on your blog, it fires up a PHP process, PHP takes the human-readable WordPress code and compiles it to something the server can then run and then that talks to MySQL, etc etc. This happens every time a page request is made, and this is horribly inefficient. What APC does is save the compiled version of the WordPress PHP code, removing that step from the process. Thankfully, that’s one of the slower parts of the whole chain. It’s enabled just by running this command (either via sudo or as root):
apt-get install php-apc && service apache2 restart
That’s it! The results from just this tweak, with the same 100 requests per second?
Transactions: 4461 hits
Availability: 100.00 %
Elapsed time: 59.23 secs
Data transferred: 9.81 MB
Response time: 0.81 secs
Transaction rate: 75.32 trans/sec
Throughput: 0.17 MB/sec
Concurrency: 61.19
Successful transactions: 4461
Failed transactions: 0
Longest transaction: 1.39
Shortest transaction: 0.18
Holy crap! That’s more than twice as fast! It’s like getting a second server for free! There is more speed available in there though…
W3 Total Cache
This is a free plugin available from the WordPress codex. Basically this plugin acts similar to APC except instead of saving the compiled version of the PHP script, it will save the HTML output of the page, bypassing the PHP to MySQL to PHP exchange. With Disk caching Apache will barely even talk to PHP and just serve up the cached HTML by itself.
There’s tons of options in here, but we’re only going to worry about the first page. I tend to disable the preview mode just so I can make my changes “live” right away for testing.
Note: Not all VPSs are created the same. Some have horrible disk I/O, some have over subscribed RAM that’s partially host swap, etc. Test the various settings to see what works best for you.
First off we’re going to look at caching everything to disk (Page cache: Disk Enhanced, DB: Disk, Object: Disk).
Transactions: 6825 hits
Availability: 100.00 %
Elapsed time: 59.83 secs
Data transferred: 15.97 MB
Response time: 0.36 secs
Transaction rate: 114.07 trans/sec
Throughput: 0.27 MB/sec
Concurrency: 41.03
Successful transactions: 6825
Failed transactions: 0
Longest transaction: 9.71
No bad! We can also stuff the cached files directly into APC (Page cache: APC, DB: APC, Object: APC). This is a bit more CPU intensive, but if your host has horrible disk I/O, this maybe your faster choice.
Transactions: 7174 hits
Availability: 100.00 %
Elapsed time: 59.43 secs
Data transferred: 16.69 MB
Response time: 0.31 secs
Transaction rate: 120.71 trans/sec
Throughput: 0.28 MB/sec
Concurrency: 37.64
Successful transactions: 7174
Failed transactions: 0
Longest transaction: 1.66
Shortest transaction: 0.14
Wow! 120.71 requests per second is about 10.5 million requests per day. That’s a lot of visitors.
Ok, what if we crank our concurrency up more? Let’s say 250.
Disk:
Transactions: 6332 hits
Availability: 99.95 %
Elapsed time: 59.78 secs
Data transferred: 14.79 MB
Response time: 1.78 secs
Transaction rate: 105.92 trans/sec
Throughput: 0.25 MB/sec
Concurrency: 188.71
Successful transactions: 6332
Failed transactions: 3
Longest transaction: 24.75
Shortest transaction: 0.30
APC:
Transactions: 5844 hits
Availability: 99.98 %
Elapsed time: 59.96 secs
Data transferred: 13.60 MB
Response time: 1.97 secs
Transaction rate: 97.46 trans/sec
Throughput: 0.23 MB/sec
Concurrency: 192.04
Successful transactions: 5844
Failed transactions: 1
Longest transaction: 28.02
Shortest transaction: 0.14
What the what? They’re LESS! Yeah, well Apache is starting to fall over here, and you can see that there’s some failed requests as well. That’s not good. Can we help poor Apache?
Nginx
Nginx is pretty awesome. It’s a web server just like Apache, but very very fast. WordPress.com uses it, so it should be good enough for us. Now, if you’re just starting out with a fresh VPS, you’re probably better off setting up Nginx + PHP-FPM. We’re not going to do that here, instead we’re going to use Nginx as a caching proxy for Apache. There’s a bunch of tutorials about setting Nginx + PHP-FPM, we’re just looking to get some more performance from our Apache install with the least amount of work with the ability to roll back easily if necessary.
Basically Nginx will sit in front of Apache. If it can serve up a file it will, if not, it will hand the request to Apache and then save the result for future requests.
Warning: We’re going to be editing some important files, make backups just in case!
/etc/apache2/ports.conf: Make these changes:
NameVirtualHost *:8080
Listen 8080
/etc/apache2/apache.conf:
/etc/apache2/sites-available/default: Make these changes:
Restart Apache:
service apache2 restart
Install Nginx and Apache mod_rpaf
apt-get install nginx libapache2-mod-rpaf && a2enmod rpaf
Replace /etc/nginx.conf with the following, changing your worker_processes to no more than the number of CPU cores you have:
user www-data;
worker_processes 8;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
client_body_temp_path /var/lib/nginx/body 1 2;
gzip_buffers 32 8k;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_comp_level 6;
gzip_http_version 1.0;
gzip_min_length 0;
gzip_types text/html text/css image/x-icon application/x-javascript application/javascript text/javascript application/atom+xml application/xml ;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
/etc/nginx/sites-available/default (replace yourserver.com with your actual server name)
server {
listen 80;
server_name yourserver.com;
access_log /var/log/nginx.access.log;
error_log /var/log/nginx_error.log debug;
location / {
root /var/www;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
if ($http_cookie ~* “comment_author_|wordpress_(?!test_cookie)|wp-postpass_”) {
set $do_not_cache 1;
}
proxy_cache_key “$scheme://$host$request_uri $do_not_cache”;
proxy_cache cache;
proxy_cache_valid 200 302 60m;
proxy_cache_valid 404 1m;
proxy_pass http://127.0.0.1:8080;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /var/www/nginx-default;
}
}
Create /etc/nginx/conf.d/proxy.conf with the following:
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
client_header_buffer_size 64k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 16k;
proxy_buffers 32 16k;
proxy_busy_buffers_size 64k;
proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=cache:8m max_size=1000m inactive=600m;
proxy_temp_path /var/lib/nginx/temp;
Create the following directories if they don’t exist and chown them for www-data:www-data:
/var/lib/nginx/cache
/var/lib/nginx/temp
mkdir /var/lib/nginx/cache
chown -R www-data:www-data /var/lib/nginx/cache
mkdir /var/lib/nginx/temp
chown -R www-data:www-data /var/lib/nginx/temp
Restart Apache and Nginx
service apache2 restart && service nginx restart
Check your site, you should see your WordPress homepage!
So what did all this work get us? The following is at the same 250 concurrency setting that choked Apache above, but with W3 Total Cache not enabled yet:
Transactions: 18447 hits
Availability: 100.00 %
Elapsed time: 59.37 secs
Data transferred: 40.60 MB
Response time: 0.24 secs
Transaction rate: 310.71 trans/sec
Throughput: 0.68 MB/sec
Concurrency: 75.54
Successful transactions: 18447
Failed transactions: 0
Longest transaction: 13.74
Shortest transaction: 0.14
Wow! 310.71 req/sec! That’s more than 2.5x our best Apache only result (@100 concurrency)! And it’s at almost 0 CPU usage as well.
How about 500 concurrency?
Transactions: 33119 hits
Availability: 100.00 %
Elapsed time: 60.23 secs
Data transferred: 72.87 MB
Response time: 0.35 secs
Transaction rate: 549.88 trans/sec
Throughput: 1.21 MB/sec
Concurrency: 193.97
Successful transactions: 33119
Failed transactions: 0
Longest transaction: 21.53
Shortest transaction: 0.14
Just shy of 550 requests per second! That works out to 1,425,288,960 requests per 30 day month. Again at almost 0 CPU. Granted, we’ll run out of our allocated bandwidth long before the month is over, but we’ve tackled the issue of WordPress falling over.
