Varnish is an HTTP accelerator and caching reverse proxy. It sits in front of Apache and used to serve cached content to anonymous users. This provides a significantly faster page response takes the load of the back-end servers.
See also: Varnish information
See also Drupal / Varnish integration.
This is where you define the Varnish settings such as which port it listens on ( VARNISH_LISTEN_PORT ), how much memory to use for caching, etc.
sudo vim /etc/sysconfig/varnish # Red Hat sudo vim /etc/default/varnish # Ubuntu
Varnish should be listening on port 80. In order for the Varnish service to start, Apache must be set to use a different port. I've been using port 81.
sudo vim /etc/httpd/ports.conf # Red Hat sudo vim /etc/apache2/ports.conf # Ubuntu
If you are setting up this up on a fresh machine, remember to update your vhosts to use the correct port. For example, here is the vhost set-up for the staging.atdtravel.com URLs I'm using for the Pressflow testing.
<VirtualHost *:81> DocumentRoot "/var/www/versions/staging" ServerName staging.atdtravel.com ServerAlias *-staging.atdtravel.com <Directory "/var/www/versions/staging"> allow from all Options +Indexes </Directory> </VirtualHost>
Varnish Configuration Language (VCL)
The specific behaviour of Varnish is defined in a VCL file. We are using one that is optimised for Drupal.
sudo vim /etc/varnish/default.vcl # Both
The backend section at the top defines where Varnish will pass requests through to. It should match the port number that Apache listens on (81).
Example VCL configurations
Some advertising services will liberally dose the browser with cookies. This isn't so good as Varnish will automatically pass any request containing cookies straight through to the back-end. The VCL we're using has exceptions in place for Google Analytics but as we discover more kinds of tracking cookie, we'll need to add exceptions into the VCL file. The first example of this is to block cookies prefixed with _#. For example, _#sess, _#tsa, _#uid, etc. Here is the snippet in the VCL file that allows Varnish to ignore them.
// remove cookies starting with _# set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_#[a-z]+)=[^;]*", "");
Regular expression cheat sheets
Testing VCL syntax
If you're making changes to the live site VCL, don't just restart the service. If you have a syntax error, you will have just taken down all the sites! You can check syntax with the command below. If the syntax is correct, you will see a lot of output (the VCL complied to C language). If there is a syntax error, you will see a short message describing the error.
sudo /usr/sbin/varnishd -C -f /etc/varnish/default.vcl
sudo /sbin/service httpd start | stop | restart | reload # Red Hat sudo /sbin/service apache2 start | stop | restart | reload # Ubuntu
sudo /sbin/service varnish start | stop | restart | reload # Both
How can I tell if it's working?
Apart from the obvious noticeable speed increase, you can see if Varnish is enabled by examining the response headers. If Varnish is enabled and working, you should see Varnish and Varnish-related headers. Here is an example of typical response headers when the cache is missed. i.e. bypassed. Notice how the Age value is zero and X-Varnish-Cache is MISS. You would expect to see this if visiting a page for the first time.
Accept-Ranges: bytes Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0 Connection: keep-alive Content-Length: 71265 Content-Type: text/html; charset=utf-8 Date: Thu, 07 Feb 2013 13:14:07 GMT ETag: "1360242842" Last-Modified: Thu, 07 Feb 2013 13:14:02 +0000 Server: Apache Via: 1.1 varnish Age: 0 X-Varnish: 1050161467 X-Varnish-Cache: MISS
Compare that with the following headers, representative of a successful cache hit; clearly indicated by the X-Varnish-Cache value of HIT and an Age of 6825 seconds.
Accept-Ranges: bytes Cache-Control: public, max-age=86400 Connection: keep-alive Content-Length: 32689 Content-Type: text/html; charset=utf-8 Date: Thu, 07 Feb 2013 13:21:53 GMT ETag: "1360236463" Expires: Sun, 11 Mar 1984 12:00:00 GMT Last-Modified: Thu, 07 Feb 2013 11:27:43 +0000 Server: Apache Vary: Cookie Via: 1.1 varnish Age: 6825 X-Varnish: 1050161543 1050159627 X-Varnish-Cache: HIT
How can I tell if it's working... properly!
The section above will help you identify that Varnish is being used and is functional. However, the main issue I ran into was pages disappearing too early from the cache. If you ssh into Live and run
varnishstat -1, you'll see all the statistics for Varnish. Look for
n_lru_nuked. This will indicate if objects are being thrown out of the cache because it's run out of space.
The problem I encountered was that this value was high and, according to New Relic, Varnish wasn't using up much memory, which was surprising as the configuration file was set to use up to 20G! Using
htop, I checked the command used to run
varnishd and it seemed to be missing chunks of the configuration parameters I'd set in
It turned out that the problem was actually the configuration file. I'd used a standard template but the fact that I was including variables made of variables in the
DAEMON_OPTS variable itself was causing it not to be parsed correctly. If it sounds confusing, don't worry, just go easy on the variables!
After a service restart, all parameters were showing in
htop and now the cache is showing zero nuked objects and taking up a more reasonable 1.5Gb of RAM. Also, checking HTTP headers yielded ages of up to 18hrs and counting so everything is working properly.
How can I purge a single object from the cache?
The easiest way is to edit a node and then save it. This will causing it to expire from the cache.
If that doesn't fit your requirement, use this command line method.
More information on this can be found here although be aware that it is referencing the Varnish 2.0 command purge. This article explains that you use ban for Varnish 3.0, as shown in the example above.
- http://www.lullabot.com/blog/article/installing-varnish-mac-os-x-mamp-or-any-other-server (set up Varnish on local dev)