Bundling Amazon Linux Images (from a windows guy)

As you may have seen from previous posts, we started using Amazon EC2 for everything, and I mean everything. We sign all our clients up with Amazon EC2, all our development servers are on Amazon EC2 and even our flagship product runs on Amazon EC2. Like I said, everything.

Half of our servers run different versions on Windows Server 2003/2008 and the other half are various flavors of Linux. Mostly Ubuntu because someone said that was the easiest to work with, and coming from an extensive windows background I was all about easy.

When one of our clients gets their Amazon account setup and ready for us to deploy to a staging server, we simply bundle the development image we have been working on , assign the clients Amazon ID number to the image. This allows the client to launch the exact version I bundled on my server.

To bundle a windows instance:
This worked exactly as expected.
            a.     simply login to the AWS Control Panel,
            b.     right click your instance, and choose Bundle Image from the Menu.
Your AMI will be ready in a few minutes.

To bundle a Linux Instance or the WTF department:
I was BLOWN away when I right clicked my linux instance and there was no option to bundle an AMI image. How could that be? Should this not be as simple? A bit of searching on the internet and hours of learning how a bunch of obscure Linux commands worked I did what any intelligent windows tech would do, I went and grabbed my Linux guy. His name was Casey but you can use any Linux guy flavor you like â€“ although they all liked to be called Casey. It is like a compliment in the uber tech community.

Working with Casey we did everything the â€œTimes New Rohanâ€™sâ€ blog told us to do. This took me about two hours to complete. Once we completed the process a few times we thought we would make everyoneâ€™s lives easier by making shell script to save everyone else in the world a lot of time.

The script will gather all the information it needs to complete the process. You will need the following information handy.
            a.    Account ID Number
            b.    Access key
            c.    Secret Access Key (I know)
            d.    Bucket Name (This is where your instance will be stored, it can be named anything you like)
            e.    Your Cert-XXXXXXXXXXXXX.pem (upload this to the /mnt folder)
            f.    Your pk-XXXXXXXXXXXXXXXX.pem (Upload this to the /mnt folder)

Most of this information can be gathered on the Security Credentials page after you login to your Amazon AWS account. The cert and pk files were downloaded when you opened your account. If you donâ€™t have them then you will need to generate some new ones on the same security Credentials page. Once you upload the pem files and the script to the /mnt folder then you are ready to run the script.

            a.    Type chmod +x ec2-bundle.sh (to allow the script to run)
            b.    Type ./ec2-bundle.sh (to actually run the script)

Now simply answer the questions and it will do its thing. There will be a long pause (10 or 15 minutes easily) while the image is bundled.

Once completed you will see a new bucked listed in your Amazon EC2 Control panel under the S3 tab. You should also see a new AMI listed under Amazon EC2 Control Panel.

You now have an easy way to rebundle Amazon Linux Instances. Who knows, maybe someday Amazon will allow a simple Right Click to make the process even easier, but until then I think these two commands is as easy as it gets.

-Jack

ec2-bundle.sh

#!/bin/bash

#/**
# * Project: EC2-Bundle : Script to bundle and register AMI with Amazon EC2
# * File name: ec2-bundle.sh
# * Description: Bash Script to bundle and register an AMI instance bundle with Amazon EC2
# *
# * @author Casey Davis (cdavis@gravityjack.com), Gravityjack https://gravityjack.com, Copyright (C) 2010.
# * @version 0.1.4
# *
# * @see The GNU Public License (GPL)
# */

#/**
# * This program is free software; you can redistribute it and/or modify
# * it under the terms of the GNU General Public License as published by
# * the Free Software Foundation; either version 2 of the License, or
# * (at your option) any later version.
# *
# * This program is distributed in the hope that it will be useful, but
# * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# * for more details.
# *
# * You should have received a copy of the GNU General Public License along
# * with this program; if not, write to the Free Software Foundation, Inc.,
# * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# */

#Warn the user to put their key and cert in /mnt this is so that the bundle doesn’t grab the users key/cert

echo "Please make sure to copy your PEM files (both key starts with ‘pk-‘ and cert starts with ‘cert-‘) to the /mnt directory"
read -p "Press Enter when your PEM files have been moved to /mnt"

echo "Enter you Amazon Account Number: NOTE without dashes"
read AMAZONACCOUNT

echo "Enter your AWS Access Key"
read AWSKEY

echo "Enter AWS Secret Access Key"
read AWSSECRET

echo "Enter the Name of the Key File"
read KEYNAME

echo "Enter the Name of your Cert File"
read CERTNAME

echo "Enter the name of the Bucket you wish to use"
read BUCKETNAME

# download and install the necessary packages for running the tools
# MUST HAVE java6 and the AMI and API tools
apt-get install zip unzip ruby openssl libopenssl-ruby curl ec2-ami-tools ec2-api-tools sun-java6-jdk

#explicity set the JAVA_HOME path var (this might not be the best solution)
echo "JAVA_HOME=/usr/lib/jvm/java-6-sun" >> ~/.bashrc
echo "export JAVA_HOME" >> ~/.bashrc

echo "Bundling Image this make take a while now would be a good time to grab a cup of coffee"
ec2-bundle-vol -d /mnt -k /mnt/$KEYNAME -c /mnt/$CERTNAME -u $AMAZONACCOUNT

read -p "Press Enter to send AMI to S3"

echo "Uploading Bundle"
ec2-upload-bundle -b $BUCKETNAME -m /mnt/image.manifest.xml -a $AWSKEY -s $AWSSECRET

#Register the AMI
ec2-register $BUCKETNA
ME/image.manifest.xml