Deploying LinkedIn’s Iris to AWS as a Docker image

What is Iris? Iris is a highly configurable and flexible service for paging and messaging. And today we’ll be setting it up on AWS.

Iris

For this project I will be using Ubuntu Desktop 16.04 LTS inside a VirtualBox machine.

While Ubuntu installs we can also create our own AWS account, which although asks for your credit card, is free (or $0.01) as long as you are not using Enterprise level resources. For testing Free Tier will work.

AWS Free Tier

We will be using EC2 to manually deploy our image today, so go ahead and get that set up.

And once you are set up let’s go ahead and select “Launch a virtual machine”

We’ll go for the EC2 instance, go ahead and click on “Get started”

So here we chose Ubuntu 16.04 as our OS. This should make things pretty simple. Avoid using Red Hat based ones like Amazon Linux or Red Hat itself, as they have a package called “packer” and we need to use a different application with that name alter on.

We probably need the least amount of resources for testing

Make sure that when you download this file you keep it safe. In my case I stored it in my home folder to make it easier for me to find it, but in general production, be careful with this file. If you launch other instances you can use the same key, so that’s useful as well. Let’s go ahead and finish up and create the instance.

So, it’s being made, click on the blue button below and let’s head to the console.

You’ll see that our instance is still initializing, wait for it.

It’s ready!

Once the status is running and the status checks are all good, we’re set to connect.

Select our instance and click on Connect

Something along these lines should display. On the Terminal cd into the folder with your .pem file and run chmod 400 IrisMachine.pem and then run the ssh command under example as shown.

If everything went according to plan our command line should now be the command line of our AWS instance. We’ve succesfully ssh’ed into the instance.

Let’s quickly go and run a sudo apt-get update to refresh the package indexes.

Then let’s run sudo apt-get install vim unzip docker.io python-pip to install vim (it’s my text editor of choice, feel free to use nano or emacs), unzip to extract files, Docker, and easy_install. The latter is necessary to install pip, which we use to install the Python dependency pyyaml, which Iris uses when building its Docker image.

$ sudo pip install pyyaml

We will also need Packer

Download Packer - Packer by HashiCorp

Go ahead and download and unzip inside our instance

$ wget https://releases.hashicorp.com/packer/1.0.2/packer_1.0.2_linux_amd64.zip
$ unzip packer_1.0.2_linux_amd64.zip

Let’s move the unzipped packer folder to /usr/local/

$ sudo mv packer /usr/local/bin/

And check if packer works

$ packer

If the command is recognized, awesome! Otherwise try $ exit and ssh back in and see if that does it.

We now have most of our dependencies before we even get to try to install Iris. As a last step, let’s set up a MySQL database.

Let’s start with a simple one:

$ sudo apt-get install mysql-server

You’ll be prompted for a password, pick one for testing that you will remember.

If you want to set up security configurations run this command and you’ll be prompted for all the necessary information

sudo mysql_secure_installation

In my case I will not run it and will go with the test default settings.

This should have done all the work for us, let’s see how it’s going

$ systemctl status mysql.service

If you get anything like this, you are golden:

$ systemctl status mysql.service
● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: en
   Active: active (running) since Wed 2017-07-12 07:07:49 UTC; 4min 47s ago
 Main PID: 15607 (mysqld)
   CGroup: /system.slice/mysql.service
           └─15607 /usr/sbin/mysqld
Jul 12 07:07:48 ip-172-31-42-118 systemd[1]: Starting MySQL Community Server...
Jul 12 07:07:49 ip-172-31-42-118 systemd[1]: Started MySQL Community Server.
lines 1-9/9 (END)

If for some reason your service isn’t running, get it going with this command:

sudo systemctl start mysql

To further check that everything is alright try running this:

mysqladmin -p -u root version

For more info and configuration with MySQL on Ubuntu, check this page out:

How To Install MySQL on Ubuntu 16.04 | DigitalOcean

Alright, now it seems we’re all set to go and install Iris (finally!)

First, let’s clone their repoditory

$ git clone https://github.com/linkedin/iris.git

Once cloned, let’s go into the packer folder:

$ cd iris/ops/packer/

From here we will run their Python script that generates a Packer template. If you don’t know what Packer is, or what is used for, check their page. They are essentially making a template from which you can build all sorts of images, including Docker.

Introduction - Packer by HashiCorp

This is how we run their Python script:

$ python gen_packer_cfg.py ./iris.yaml

And then we can compile the Packer image:

$ packer build -only=amazon-ebs \
    -var "aws_access_key=AAAAAAAAAAAAAAAAAAAA" \
    -var "aws_secret_key=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB" \
    ./output/iris.json

If you don’t know where to get your access and secret key, go to the menu under your name:

Click on My Security Credentials

I have a few but you might have none, that’s fine, let’s go and Create New Access Key

I recommend you download the file and keep it somewhere safe, you can’t see the access key after this. Click on Show Access Key and fill them in the command above, then run it.

The builder will run for a while, but hopefully at the end you get something like this:

Now let’s build our Docker image with the command:

$ sudo packer build -only=docker ./output/iris.json

This should also take a bit to build, especially if it’s the first Ubuntu based Docker image you build on this instance as it will have to download ubuntu and other dependencies.

If everything went alright all the text running through your screen should be green and not red, and in the end you should get something like this:

We got our image! We can check if it’s there:

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
quay.io/iris/iris   0.0.14              f6f2e66e1b8b        About a minute ago   721.2 MB
quay.io/iris/iris   latest              f6f2e66e1b8b        About a minute ago   721.2 MB
ubuntu              16.04               d355ed3537e9        3 weeks ago          119.2 MB

Perfect! Now let’s spin up our Iris instance and connect it to existing MySQL database (using your values where necessary):

$ sudo docker run -e DOCKER_DB_BOOTSTRAP=1 \
	-e IRIS_CFG_DB_USER=root -e IRIS_CFG_DB_PASSWORD=admin -e IRIS_CFG_DB_HOST=IP_ADDRESS \
	quay.io/iris/iris:latest

Make sure the user has remote access, and that port 3306 is exposed to remote connection. This is because Iris will be running through Docker, so the local IP address will be different.

You can also examine the Docker image using run

$ sudo docker run --rm=true -i -t quay.io/iris/iris:latest /bin/bash