Deploying LinkedIn’s Iris to AWS as a Docker image
What is Iris? Iris is a highly configurable and flexible service for paging and messaging. And today we’ll be setting it up on AWS.
For this project I will be using Ubuntu Desktop 16.04 LTS inside a VirtualBox machine.
While Ubuntu installs we can also create our own AWS account, which although asks for your credit card, is free (or $0.01) as long as you are not using Enterprise level resources. For testing Free Tier will work.
AWS Free Tier
AWS Marketplace offers free and paid software products that run on the AWS Free Tier. If you qualify for the AWS Free…
We will be using EC2 to manually deploy our image today, so go ahead and get that set up.
And once you are set up let’s go ahead and select “Launch a virtual machine”
We’ll go for the EC2 instance, go ahead and click on “Get started”
So here we chose Ubuntu 16.04 as our OS. This should make things pretty simple. Avoid using Red Hat based ones like Amazon Linux or Red Hat itself, as they have a package called “packer” and we need to use a different application with that name alter on.
Make sure that when you download this file you keep it safe. In my case I stored it in my home folder to make it easier for me to find it, but in general production, be careful with this file. If you launch other instances you can use the same key, so that’s useful as well. Let’s go ahead and finish up and create the instance.
So, it’s being made, click on the blue button below and let’s head to the console.
You’ll see that our instance is still initializing, wait for it.
Once the status is running and the status checks are all good, we’re set to connect.
Select our instance and click on Connect
Something along these lines should display. On the Terminal
cd into the folder with your
.pem file and run
chmod 400 IrisMachine.pem and then run the
ssh command under example as shown.
If everything went according to plan our command line should now be the command line of our AWS instance. We’ve succesfully ssh’ed into the instance.
Let’s quickly go and run a
sudo apt-get update to refresh the package indexes.
Then let’s run
sudo apt-get install vim unzip docker.io python-pip to install vim (it’s my text editor of choice, feel free to use nano or emacs), unzip to extract files, Docker, and easy_install. The latter is necessary to install pip, which we use to install the Python dependency pyyaml, which Iris uses when building its Docker image.
$ sudo pip install pyyaml
We will also need Packer
Download Packer - Packer by HashiCorp
- Packer by HashiCorp Download Packerwww.packer.io
Go ahead and download and unzip inside our instance
$ wget https://releases.hashicorp.com/packer/1.0.2/packer_1.0.2_linux_amd64.zip
$ unzip packer_1.0.2_linux_amd64.zip
Let’s move the unzipped packer folder to /usr/local/
$ sudo mv packer /usr/local/bin/
And check if packer works
If the command is recognized, awesome! Otherwise try
$ exit and ssh back in and see if that does it.
We now have most of our dependencies before we even get to try to install Iris. As a last step, let’s set up a MySQL database.
Let’s start with a simple one:
$ sudo apt-get install mysql-server
You’ll be prompted for a password, pick one for testing that you will remember.
If you want to set up security configurations run this command and you’ll be prompted for all the necessary information
In my case I will not run it and will go with the test default settings.
This should have done all the work for us, let’s see how it’s going
$ systemctl status mysql.service
If you get anything like this, you are golden:
$ systemctl status mysql.service
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: en
Active: active (running) since Wed 2017-07-12 07:07:49 UTC; 4min 47s ago
Main PID: 15607 (mysqld)
└─15607 /usr/sbin/mysqldJul 12 07:07:48 ip-172-31-42-118 systemd: Starting MySQL Community Server...
Jul 12 07:07:49 ip-172-31-42-118 systemd: Started MySQL Community Server.
lines 1-9/9 (END)
If for some reason your service isn’t running, get it going with this command:
sudo systemctl start mysql
To further check that everything is alright try running this:
mysqladmin -p -u root version
For more info and configuration with MySQL on Ubuntu, check this page out:
Alright, now it seems we’re all set to go and install Iris (finally!)
First, let’s clone their repoditory
$ git clone https://github.com/linkedin/iris.git
Once cloned, let’s go into the packer folder:
$ cd iris/ops/packer/
From here we will run their Python script that generates a Packer template. If you don’t know what Packer is, or what is used for, check their page. They are essentially making a template from which you can build all sorts of images, including Docker.
Introduction - Packer by HashiCorp
Welcome to the world of Packer! This introduction guide will show you what Packer is, explain why it exists, the…
This is how we run their Python script:
$ python gen_packer_cfg.py ./iris.yaml
And then we can compile the Packer image:
$ packer build -only=amazon-ebs \
-var "aws_access_key=AAAAAAAAAAAAAAAAAAAA" \
-var "aws_secret_key=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB" \
If you don’t know where to get your access and secret key, go to the menu under your name:
Click on My Security Credentials
I have a few but you might have none, that’s fine, let’s go and Create New Access Key
I recommend you download the file and keep it somewhere safe, you can’t see the access key after this. Click on Show Access Key and fill them in the command above, then run it.
The builder will run for a while, but hopefully at the end you get something like this:
Now let’s build our Docker image with the command:
$ sudo packer build -only=docker ./output/iris.json
This should also take a bit to build, especially if it’s the first Ubuntu based Docker image you build on this instance as it will have to download ubuntu and other dependencies.
If everything went alright all the text running through your screen should be green and not red, and in the end you should get something like this:
We got our image! We can check if it’s there:
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
quay.io/iris/iris 0.0.14 f6f2e66e1b8b About a minute ago 721.2 MB
quay.io/iris/iris latest f6f2e66e1b8b About a minute ago 721.2 MB
ubuntu 16.04 d355ed3537e9 3 weeks ago 119.2 MB
Perfect! Now let’s spin up our Iris instance and connect it to existing MySQL database (using your values where necessary):
$ sudo docker run -e DOCKER_DB_BOOTSTRAP=1 \
-e IRIS_CFG_DB_USER=root -e IRIS_CFG_DB_PASSWORD=admin -e IRIS_CFG_DB_HOST=IP_ADDRESS \
Make sure the user has remote access, and that port 3306 is exposed to remote connection. This is because Iris will be running through Docker, so the local IP address will be different.
You can also examine the Docker image using
$ sudo docker run --rm=true -i -t quay.io/iris/iris:latest /bin/bash