Jenkins - Continuous Integration (CI)

×

Error message

Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in _menu_load_objects() (line 569 of /homepages/46/d762693627/htdocs/dc/includes/menu.inc).
Jenkins - Continuous Integration (CI)

Set up

Item

Location

Details
Temp Dir

/var/lib/jenkins/tmp
The standard /tmp directory had noexec set so Jenkins cannot execute the files it stores in there. A new Jenkins directory gets around the issue.Config File

/etc/sysconfig/jenkins
The main config file for Jenkins (port changes etc)Git Repository

/var/lib/jenkins/scm-sync-configuration/checkoutConfiguration/
The location of the files that are backed up in Git so that configuration changes can be trackedSCM Sync Configuration plugingit@bitbucket.org:atddev/jenkins.gitThe Bitbucket branch for the Jenkins Jobs, see below for more details

Note: The configuration is source controlled via a branch in Bitbucket.

Access

Jenkins can be accessed on the following path, http://ci.atdtravel.com:8080/. It cannot be accessed outside of the office. If it becomes inaccessible, it may be down to a Firewall rule change.

In order to log into Jenkins, use one of the following set of credentials:

Username

Password

Description
4dm1nn1nj4123The admin account for setting up jobs, user and configurationd4pl0yn1nj4321The user used for executing jobs for deployments and drush requests etc

Sudo access:

su -s /bin/bash jenkins

The ssh details can be found in /var/lib/jenkins/.ssh/.

Add New User

In order to add a new user for Jenkins, go to the Add User page.

Setting up a Jenkins user on a new VM example:

adduser jenkins-example

passwd jenkins-example

usermod -g atd -G jenkins-example jenkins-example

ssh-keygen -t rsa

touch /home/jenkins-example/.ssh/authorized_keys

chmod 600 /home/jenkins-example/.ssh/authorized_keys

On the ci1 VM, you'll need to sudo as Jenkins to the new server in order to get the details added to the known_hosts file. See above for how to sudo as the Jenkins user. In addition, for things like a code pull, you'll need to run it manually on the VM first.

New remote hosts are added into Jenkins the configure system page.

Server User

Jenkins has corresponding users on our VMs that it can use to ssh in without the need for a password. The ssh is an authorized key. These include:

Server

Username

Password

Purpose
web1jenkins-web1n1nj4123Used for running drush commands and shell scriptsweb6jenkins-web6n1nj4123Used for running drush commands and shell scriptsnfs1jenkins-nfs1n1nj4123Used to deploy codecache2jenkins-cache2n1nj4123Used to clear varnish in primeweb10jenkins-web10n1nj4123Used to clear the D8 cachedsd1jenkins-dsd1n1nj4123Used for running drush commands and shell scripts for DSDdsd2jenkins-dsd1n1nj4123Clone of dsd1. Access required to clear redis.dsd3jenkins-dsd1n1nj4123Clone of dsd1. Access required to clear redis.stage-02jenkins-stage02n1nj4123D6 Stagingstage-03jenkins-stage03n1nj4123D7 Stagingstage-04jenkins-stage04n1nj4123D7 DSD Stagingstage-05jenkins-stage05n1nj4123D8stage-06jenkins-stage06n1nj4123Gateway Stagingstage-dbjenkins-databasen1nj4123Databasesweb12jenkins-web12n1nj4123Gateway

 

Local Setup

A Vagrant VM is available for local testing. This VM is called 'ci', short for Continuous Integration. This can be used for testing potential jobs before being added to the live environment. It can also be used to test plugins.

Url

User

Password
http://jenkins-ci.dev:8080/Needs to be set up on start upNeeds to be set up on start up

Bitbucket

User Account

Access to Bitbucket for the Jenkins users is:

Email

Username

Password
dev+admin+jenkins@atdtravelservices.co.ukatd-jenkins4tdjenk1ns

Configuration & Source Control

The configuration git branch for the Jenkins configuration can be found on Bitbucket. All the configuration should source controlled via the SCM Sync configuration plugin. This includes the configuration of Jenkins and all the jobs that have been created.

Automatic Code deployments

DSD live job build url: http://ci.atdtravel.com:8080/buildByToken/build?job=Staging%20-%20Server%20Config%20Code%20Deployment&token=DcJrt_tBeF9gN

It is possible to trigger a build in Jenkins via a Bitbucket Webhook.

The initial attempt to get automatic code deployments working involved setting up a Jenkins job that could be triggered externally. Using a token set in the job, a url to access it is made available. In Bitbucket, this url can then be called when the repository is updated in some way. In Bitbucket, this is known as a webhook. Using ssh, the job itself then accesses the server(s) hosting the code and runs git pull origin.

There are various plugins that may offer better solutions than this. They have been included in the setup of Jenkins and can be investigated when the functionality is required.

Whilst looking into auto code deployments, I encountered timeout issues on the pushes from Bitbucket. However, this was down to it not being possible to access Jenkins externally. After amending the Firewall rules so that Bitbucket could access Jenkins, the job began to fire.

Jenkins is currently locked down as you need to login to access it. In order for the job url to be accessible, the Build Token Root Plugin is used.

Creating a new job

The best way to decide how to set up a job is to look at the ones that already exist. 

The initial process involves the following steps:

  1. Log in as the admin user
  2. Click 'New Item' on the left hand side
  3. Select 'Freestyle project'
  4. Click 'OK'

The majority of the jobs involve executing a shell script on one of the servers. You are not limited to one build option per job. You can add as many as required. Within the build section, you do the following:

  1. Click 'Add build step'
  2. Select 'Execute shell script on remote host using ssh'
  3. Select the server you want to execute the script from
  4. Add command into textarea.

The 'Build Triggers' section allows to specify a cron job style run rule. You can also specify that it runs straight after another Jenkins job.There are also options to throttle the build such as 'Throttle Concurrent Builds' to make sure that the job is only running once at any given time. You can specify jobs to run after the build has completed such as clearing the cache. Note: Activating Check Norris is optional in the after build step.

If you want to achieve something but the functionality does not seem to be available, there could be a plugin that could help you. To add a plugin do the following:

  1. Log in
  2. Click 'Manage Jenkins' on the left hand menu
  3. Click 'Manage Plugins'

Support

  1. I cannot access the login screen
    Check whether the jenkins.war file was recently updated / if a new version has been released, /usr/lib/jenkins/jenkins.war (http://mirrors.jenkins.io/war/). If a new version has been added, it may stop things from working and you may need to roll things back.

Useful Links

blog tag: