May 11, 2014

Multi-feature staging environment(s)

It is hard to work on several versions of a web app and demo them to other people on the Internet (or in an organization).

In other words: how do you elegantly deploy all of your git branches and make them accessible on the Internet?

The Objective is to access your project version via a subdomain like http://<feature_name>.alpha.cryptojobslist.com, while all developer has to do is to push code to a feature/feature_name git branch.

Note: this tutorial is for SPAs (frontend / static files / js apps / whichever way you call them). Set-up for backend apps (node/ror/python/java) is different, I might write about it later. The approach described below is relevant to decoupled frontends.

Solution #

Summary: every branch you push to, should be deployed. Every push should go through same CI process → branch should be built → artifacts are moved to the same server(s) → every built branch should result in a directory with the name of the branch. HTTP request is fired → DNS resolves *.alpha.cryptojobslist.com to your server/load balancer → depending on <feature_name>.alpha.cryptojobslist.com NGINX points to the right directory in that server and returns relevant branch back to the browser.

Lets cover that in detail:

Git workflow #

I recommend git-flow. TLDR: Everything pushed to master goes to production, develop is staging and/or general development, feature/feature_name branches are for massive features. Upon completion feature branches are merged into develop. After testing develop is merged into master.

CI #

During CI process you should determine the name of the branch that is being built and place resulting artifacts in a folder with the name of your branch.
I do that with a bash script that is being called during Jenkins build process:

Example of jenkins.sh:

#!/usr/bin/env bash

set -e

IFS='/';
BRANCHES=$(git describe --contains --all HEAD);
BRANCHES=($BRANCHES);
BRANCH=${BRANCHES[3]};
if [[ "$BRANCH" == "" ]]
then
 BRANCH=${BRANCHES[2]};
fi;

if [[ -n "$GIT_TAG" ]]
then
    BRANCH="MASTER";
fi;

BRANCH=$(echo $BRANCH | tr '[:upper:]' '[:lower:]')
# at this moment BRANCH will be equal to master, develop or some kind of feature_name. We want to make sure that BRANCH does not contain feature/ part, as you would not be able to do mkdir with it the right way.

cd www
npm install

# actual build process
grunt stage
grunt test
grunt publish
# at this stage compiled frontend code will be in a `publish` directory


rm -rf tmp;
mkdir tmp;
mv publish/** tmp
mkdir publish/$BRANCH
cp -rf tmp/* publish/$BRANCH

# at this stage all files from `publish` directory are moved to a directory named after your feature e.g. `feature_name` and ready to be sent to the server.

As build phase succeeds you want to transfer your directory to the server. E.g. to /var/www/branches/ directory. After a while, this directory should contain directories like:

.
..
master
develop
my_cool_feature
feature231
some_other_feaure
many_more_omg_features

Note: I’d recommend using Travis instead of Jenkins. But if you are not using CI at all, you can run a similar script as a post-receive git hook. Thanks for noting, Chinmay!

DNS #

In your DNS (I used Route 53) create an A record *.alpha.cryptojobslist.com and point it to your server, with your build artifacts. Simple.

NGINX config #

Within server scope, before specifying location and root of your static files, you want to set a variable, that will define end route to branch’s files.

# Serve features on *.alpha subdomain
if ($host ~* ^([^.]+)\.alpha\.cryptojobslist\.com$) {
  set $branch $1;
}

# All branches are hosted from a subdirectory
root /var/www/branches/$branch;

The whole config will look something like:

server {
  listen 80;
  server_name *.alpha.cryptojobslist.com;

  set $branch "master";   #default branch to serve
  if ($host ~* ^([^.]+)\.alpha\.cryptojobslist\.com$) {
    set $branch $1;
  }

  root /var/www/branches/$branch;

  index index.html;

  # Serve the HTML
  location * /path/ {
    # this part should be more complex, obviously
    try_files $uri $uri/ /index.html;
  }
}

All this setup will lead to the following:

you request http://my_cool_feature.alpha.cryptojobslist.com
nginx serves frontend code from /var/www/branches/my_cool_feature

Because we did set $branch "master"; before regexing hostname, whenever you access http://alpha.cryptojobslist.com — master branch will be served automatically.

This approach is successfully used for more than a year at ever growing Redmart; other companies are adopting this. Got positive feedback from from JSConf.asia and GitHub folks.

Hope that was useful! Would be great to leave me your feedback and experience with this setup via @ramanshalupau.
You should subscribe to this blog.

Kudos