nginx + Apache + W3 Total Cache – a bad combination

WordPress is notorious for buckling if a site sees a sudden spike in traffic (also known as the Digg Effect, slashdotted, fireballed etc. etc.). As a result, a number of plugins have emerged to replace the default “render-on-every-access” WordPress model with serving static files from disk (ala Drupal or Joomla). The other option is to use nginx as a front-end proxy for Apache, letting nginx handle serving static files while Apache takes care of dynamic content. This is the setup I have in place for the wiki and my Gitweb installation as well.

One of the most popular plugins for content caching is W3 Total Cache. W3 Total Cache handles page caching (aka serving content from static HTML) but also does DB caching, browser caching and CDN redirection (if you’re so inclined). With a stock Apache or nginx (with PHP-CGI) install, getting W3 Total Cache working is fairly straightforward.

Unfortunately, once you decide to use the combination of nginx + Apache, the story gets a lot murkier.

Let’s look at the stock nginx configuration for WordPress when proxying to Apache:

In this configuration, page caching will simply not work:

W3 Total Cache - rejected URI

The reason for this is the URL that gets passed to Apache:

In contrast, look at the actual URL that’s being used to access the page:

The inclusion of the index.php in the URL is breaking the caching as W3 Total Cache is expecting the URL to match what nginx is seeing. The nginx configuration is the problem and needs to be fixed.

Let’s try a different nginx configuration:

Success! W3 Total Cache is now seeing the URL the way nginx sees it and page caching is starting to work.

W3 Total Cache enabled

Except, the home page seems to be missing :(. The reason for this is that the default / URL looks for the directory file, i.e. index.php which isn’t currently handled by the nginx configuration.
So let’s add an handler for .php files in nginx:

Let’s refresh the browser and check.. yup, the homepage is back. Looks it’s time to celebrate except..

W3 Total Cache - homepage

This happens because index.php is not normally cached by W3 Total Cache. Only the output from index.php needs to be cached by W3 Total Cache.

W3 Total Cache - page cache settings

Finally, remember the try_files directive that we used to get permalink caching working? That directive tells nginx to look for the file on disk first and only if not found, pass the request to Apache. This means that any static files in the wordpress directory will be served by nginx and W3 Total Cache doesn’t come into effect at all. This is mostly fine, since nginx can also handle the browser caching and content gzipping. Where this breaks down is if you intend to use a CDN as the files will not be redirected by nginx.

So that’s where things stand – You can either have caching on articles but a broken homepage or caching on articles, no caching on the homepage and no CDN capabilities. If anyone has suggestions on how to get W3 Total Cache working with nginx+Apache, let me know in the comments.