Varnish information

×

Error message

Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in _menu_load_objects() (line 569 of /homepages/46/d762693627/htdocs/dc/includes/menu.inc).
Varnish information

Overview

Varnish is an HTTP accelerator and caching reverse proxy. It sits in front of Apache and used to serve cached content to anonymous users. This provides a significantly faster page response takes the load of the back-end servers.

See also: Varnish information

Drupal integration

See also Drupal / Varnish integration.

Drupal integrates with Varnish via the Purge and Expire modules. 

Configuration

Service

This is where you define the Varnish settings such as which port it listens on ( VARNISH_LISTEN_PORT ), how much memory to use for caching, etc.

sudo vim /etc/sysconfig/varnish  	# Red Hat

sudo vim /etc/default/varnish    	# Ubuntu

Varnish should be listening on port 80. In order for the Varnish service to start, Apache must be set to use a different port. I've been using port 81.

sudo vim /etc/httpd/ports.conf   	# Red Hat

sudo vim /etc/apache2/ports.conf 	# Ubuntu

If you are setting up this up on a fresh machine, remember to update your vhosts to use the correct port. For example, here is the vhost set-up for the staging.atdtravel.com URLs I'm using for the Pressflow testing.

<VirtualHost *:81>

  DocumentRoot "/var/www/versions/staging"
  
  ServerName staging.atdtravel.com
  ServerAlias *-staging.atdtravel.com

  <Directory "/var/www/versions/staging">
    allow from all
    Options +Indexes
  </Directory>

 </VirtualHost>

Varnish Configuration Language (VCL)

The specific behaviour of Varnish is defined in a VCL file. We are using one that is optimised for Drupal.

 sudo vim /etc/varnish/default.vcl	# Both

The backend section at the top defines where Varnish will pass requests through to. It should match the port number that Apache listens on (81).

Example VCL configurations

Tracking cookies

Some advertising services will liberally dose the browser with cookies. This isn't so good as Varnish will automatically pass any request containing cookies straight through to the back-end. The VCL we're using has exceptions in place for Google Analytics but as we discover more kinds of tracking cookie, we'll need to add exceptions into the VCL file. The first example of this is to block cookies prefixed with _#. For example, _#sess, _#tsa, _#uid, etc. Here is the snippet in the VCL file that allows Varnish to ignore them.

// remove cookies starting with _#
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_#[a-z]+)=[^;]*", "");

Regular expression cheat sheets

Testing VCL syntax

If you're making changes to the live site VCL, don't just restart the service. If you have a syntax error, you will have just taken down all the sites! You can check syntax with the command below. If the syntax is correct, you will see a lot of output (the VCL complied to C language). If there is a syntax error, you will see a short message describing the error.

sudo /usr/sbin/varnishd -C -f /etc/varnish/default.vcl

Service control

Apache

sudo /sbin/service httpd start | stop | restart | reload	# Red Hat

sudo /sbin/service apache2 start | stop | restart | reload	# Ubuntu

Varnish

sudo /sbin/service varnish start | stop | restart | reload	# Both

How can I tell if it's working?

Apart from the obvious noticeable speed increase, you can see if Varnish is enabled by examining the response headers. If Varnish is enabled and working, you should see Varnish and Varnish-related headers. Here is an example of typical response headers when the cache is missed. i.e. bypassed. Notice how the Age value is zero and X-Varnish-Cache is MISS. You would expect to see this if visiting a page for the first time.

Accept-Ranges: bytes
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Connection: keep-alive
Content-Length: 71265
Content-Type: text/html; charset=utf-8
Date: Thu, 07 Feb 2013 13:14:07 GMT
ETag: "1360242842"
Last-Modified: Thu, 07 Feb 2013 13:14:02 +0000
Server: Apache
Via: 1.1 varnish
Age: 0
X-Varnish: 1050161467
X-Varnish-Cache: MISS

Compare that with the following headers, representative of a successful cache hit; clearly indicated by the X-Varnish-Cache value of HIT and an Age of 6825 seconds.

Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Connection: keep-alive
Content-Length: 32689
Content-Type: text/html; charset=utf-8
Date: Thu, 07 Feb 2013 13:21:53 GMT
ETag: "1360236463"
Expires: Sun, 11 Mar 1984 12:00:00 GMT
Last-Modified: Thu, 07 Feb 2013 11:27:43 +0000
Server: Apache
Vary: Cookie
Via: 1.1 varnish
Age: 6825
X-Varnish: 1050161543 1050159627
X-Varnish-Cache: HIT

How can I tell if it's working... properly!

The section above will help you identify that Varnish is being used and is functional. However, the main issue I ran into was pages disappearing too early from the cache. If you ssh into Live and run varnishstat -1, you'll see all the statistics for Varnish. Look for n_lru_nuked. This will indicate if objects are being thrown out of the cache because it's run out of space.

The problem I encountered was that this value was high and, according to New Relic, Varnish wasn't using up much memory, which was surprising as the configuration file was set to use up to 20G! Using htop, I checked the command used to run varnishd and it seemed to be missing chunks of the configuration parameters I'd set in /etc/sysconfig/varnish.

It turned out that the problem was actually the configuration file. I'd used a standard template but the fact that I was including variables made of variables in the DAEMON_OPTS variable itself was causing it not to be parsed correctly. If it sounds confusing, don't worry, just go easy on the variables!

After a service restart, all parameters were showing in htop and now the cache is showing zero nuked objects and taking up a more reasonable 1.5Gb of RAM. Also, checking HTTP headers yielded ages of up to 18hrs and counting so everything is working properly.

How can I purge a single object from the cache?

The easiest way is to edit a node and then save it. This will causing it to expire from the cache.

If that doesn't fit your requirement, use this command line method.

More information on this can be found here although be aware that it is referencing the Varnish 2.0 command purge. This article explains that you use ban for Varnish 3.0, as shown in the example above.

Further reading

blog tag: