Developers can use live Varnish config locally

Developers can use live Varnish config locally

I'm copying our live Varnish config from w4 to our Varnish config in Vagrant (which until now has been relatively untouched). Our w4 Varnish config has had quite a bit of work done on it, so is very different from the out-of-the-box config.

As part of this work, I'll look to refactor, clean up and add comments to the live Varnish config.

I'd like to get the dev and live versions of the config to be as similar as possible.

The live config is customised in various ways to achieve the following:

  • General
    • Debug statements throughout the various functions reveal the processing workflow used to serve each request (very useful in troubleshooting)
  • vcl_recv
    • Blacklist of paths which should never be cached (cart, checkout, admin, etc - we have about 50 patterns currently)
    • We have created a way of purging specific URLs from the cache by appending ?PURGE to the end of the URL.
    • A set of Drupal-related customisations have been added, including:
      • Prevent caching of install.php, update.php, cron.php.
      • Remove cookie from requests for static files (images, etc), to ensue these file types are always cached.
      • Remove cookie from requests for sites/default/files*, sites/all/modules/*, sites/all/themes*, to ensure they're always cached.
      • Remove various cookies that Drupal doesn't care about
    • Set the custom header X-AFFILIATE on DSD. This value of this header is taken from the cookie and then removed from the cookie.
    • Requests to the TPB home page are redirected to /login if no php session is found in the request cookie.
    • Delete all cookies from requests to almost all front-end pages, to ensure they are cached.
  • vcl_hash
    • Normalise various URLs by removing extraneous suffixes (eg. "\?quicktabs=[^&]*").
    • Normalise DC calendar URLs to be the same across all brands.
  • vcl_fetch
    • Increase default cache time, based on instructions https://www.varnish-cache.org/trac/wiki/VCLExampleLongerCaching
    • Tell browsers to always make conditional GET requests for TPB pricing, before using browser cached pricing data.
    • Cache Qubit remaining quantity feed for two minutes.
    • Cache DC calendar ajax calls for four weeks.
    • Don't allow static files to set cookies.
  • vcl_deliver
    • Set the custom http header X-Varnish-Cache to either HIT or MISS (useful for understanding if a request is being cached or not).
  • vcl_error
    • Provide a mechanism for other functions to specify 302 redirects, by specifying error 750 (eg. error 750 "/login")
    • Display brand-specific, generic "high traffic" error pages in case anything goes wrong (not just high traffic!). These pages are stored in /var/www/errors/hightraffic/ on w4 and reference images stores on Rackspace Cloud (since the web server back end might be down when this error page is displayed). Currently these pages are missing for TPB, CO and DSD - I'll add a task for us to create these.

Changes to make to the live config:

  • General
    • Replace all the identical per-brand backends with one backend called "d6" (at some point we'll be adding one or more "d7" backends too)
    • Add a JIRA task for us to create "high traffic" pages for TPB, CO and DSD (Done - see GEN-1625)
  • vcl_recv
    • Remove all the old Bonza header stuff and Bonza theme stuff (it's already commented out)
    • Remove all the old ATD cart experiment stuff
    • Tidy up comments around TPB login redirect
    • Refactor the redundant "If cookies remain or authorization..." bit
  • vcl_hash
    • Remove commented out theme bit at the bottom
  • vcl_fetch
    • Fix the DC comment (currently says one week but the setting is four)
  • vcl_hit
    • Remove the commented out bit
  • vcl_miss
    • Remove the commented out bit

Here is the live config before the changes above and after the changes above. The config file before the changes is also backed up on w4 in /etc/varnish/default.vcl.bak.

Copy live config to Vagrant config, section by section:

  • vcl_recv
  • vcl_hash
  • vcl_fetch
  • vcl_hit
  • vcl_miss

 

blog tag: