Setting up Varnish and Nginx test cloud servers for API test

Setting up Varnish and Nginx test cloud servers for API test

Server setup

Created two new Ubuntu 14 servers at https://mycloud.rackspace.co.uk

Changed root passwords to phoenix10

Rackspace name: api-test-nginx
ip address: 162.13.86.171
root: phoenix10

Rackspace name: apt-test-varnish
ip address: 162.13.86.181
root: phoenix10

Domain setup

Created three new A records on the atdtravel.com domain, all pointing at the Nginx server:

  • api-test1.atdtravel.com
  • api-test2.atdtravel.com
  • api-test3.atdtravel.com

These were set up on the main Rackspace portal https://my.rackspace.com

The A records have a ttl of 3600, so we can have changes reflected more frequently.

Varnish setup

Installed Varnish with "apt-get install varnish". Also installed git, not sure if it will be needed through.

Backed up /etc/varnish/default.vcl to /etc/varnish/default.vcl.bak

Adding new backends d6 and d7 to /etc/varnish/default.vcl. At first it didn't work because it turns out that every backend created in the vcl must be referenced at least once in the config. So I uncommented the default vcl_recv() routine and added "set req.backend = d7; set req.backend = d6;" there, so they're at least referenced. Will tweak these later once we know where the traffic needs sending. For the proof-of-concept we can probably send all /api/ requests to d6 and everything else to d7.

Changed /etc/default/varnish to have Varnish listen on port 80 rather than the default 6081.

After restarting Varnish (reload didn't work), I can see it successfully connecting to the d6 backend (as a bunch of Drupal html is coming out), when I do "curl localhost:80".

However port 80 appears to be closed on the firewall, as the page hangs when I connect to the ip address in my browser.

Running this to open the port worked: "sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT".

Nginx setup

Installed Nginx with "apt-get install nginx". Also installed git, not sure if it will be needed through.

Opening port 80 on firewall with "sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT".

I deleted the symlink /etc/nginx/sites-enabled/default and created this new server config /etc/nginx/conf.d/default.conf, pointing all traffic at Varnish for the moment and passing along the hostname:

upstream varnish {
    server 162.13.86.181:80;
}
# DEFAULT SERVER
server {
    listen 80 default_server;
    proxy_read_timeout 12000;
    location / {
        proxy_pass   http://varnish;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;
    }
}

At first Nginx wasn't loading this config so I did a stop (/etc/init.d/stop) and a start after putting some gibberish into the new config file and the start failed. Looking in the error log (/var/log/nginx/error.log) confirmed it was due to the gibberish. Removing the gibberish and starting worked.

I can now get to the d6 site on the Nginx server's IP address.

The setup looks like this: