EC2: Install ATLAS Software to an EBS volume

In CernVM: local ATLAS Software — the clean solution I proposed to install an ATLAS Software release to an EBS volume. I did it to make it locally available in CernVM running on EC2. The approach allows to move an EBS volume from one EC2 instance to another without losing important components and functionality. Here are some hints (both, CernVM specific and general) to follow before installing the software via pacman..

Start off from a fresh CernVM — migrate to group-atlas

Start a fresh CernVM 1.2.0 instance on EC2 (I used ami-a50cebcc, aki-9b00e5f2). Bootstrap it to the following configuration (via web interface):

• create the new linux user atlasuser
• under Virtual Organization Configuration choose ATLAS — keep everything else as No

Log in via ssh, using the new atlasuser account. At first, the necessary system components have to be installed using Conary.

[atlasuser@bla ~]$ sudo conary migrate group-atlas --interactive

You will need the “admin password” you set at bootstrap. If this migration step throws errors for you, consider this blog post: CernVM on Nimbus: kernel problems.

With migration to group-atlas, gcc gets installed. We will need it soon!

Create and attach EBS volume, make file system

Create a new EBS volume within the same availability zone as the CernVM instance just startet up. I chose 15 GB size. This should be enough to hold one ATLAS Software release plus a bit of additional stuff.

Attach this new EBS volume as e.g. /dev/sdh to the CernVM instance.

Regarding CernVM, now e2fsprogs has to be installed to create a filesystem on the fresh block device /dev/sdh. Therefore we need the compiler.
Work as root, download&install:

[root@bla ~]# wget http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.41.6.tar.gz
[...]
`e2fsprogs-1.41.6.tar.gz' saved [4422395/4422395]
[root@bla ~]# tar xzf e2fsprogs-1.41.6.tar.gz
[root@bla ~]# cd e2fsprogs-1.41.6
[root@bla e2fsprogs-1.41.6]# ./configure > cfg.log
[root@bla e2fsprogs-1.41.6]# make install > makeinst.log
make[1]: texi2dvi: Command not found
make[1]: [libext2fs.dvi] Error 127 (ignored)
/usr/bin/install: cannot stat `libext2fs.info*': No such file or directory
make[1]: [install-doc-libs] Error 1 (ignored)
gzip: /usr/share/info/libext2fs.info*: No such file or directory
make[1]: [install-doc-libs] Error 1 (ignored)

The warnings are no problem. Now mkfs.ext3 can be used:

[root@bla e2fsprogs-1.41.6]# /sbin/mkfs.ext3 /dev/sdh
mke2fs 1.41.6 (30-May-2009)
/dev/sdh is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
[...]
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
 
This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Mount it

Now the mountpoint has to be chosen. The objective is to install everything needed by the ATLAS Software to this EBS volume so that moving around with this volume between different instances (which are running operating systems supporting ATLAS Software) becomes possible.

Keep in mind that an ATLAS Software — once installed to a specific path — should always work under this path. By moving it around, you will get problems, even with new set up scripts. Furthermore, keeping everything necessary under the mountpoint allows to e.g. move an EBS volume from one EC2 instance to another without losing important components and functionality.

I chose the mountpoint /opt/atlas-local. Now and in the future, I will always have to mount the EBS volume to this directory.

Work as root: create mountpoint, mount device and give all rights:

[root@bla opt]# mkdir /opt/atlas-local
[root@bla opt]# mount /dev/sdh /opt/atlas-local
[root@bla opt]# chmod 777 /opt/atlas-local

Now log in as atlasuser and check out the new space:

[atlasuser@bla atlas-local]$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1              9293008   1999624   6821316  23% /
none                    890952         0    890952   0% /dev/shm
/dev/sdh              15481840    169592  14525816   2% /opt/atlas-local

Install ATLAS Software to this EBS volume

To install a new ATLAS Software Release to a subdirectory of the mountpoint, you can use my following script which automatically checks out pacman and invokes the installing command:

PACMANDIR=/opt/atlas-local/pacman
ATLASINSTALLDIR=/opt/atlas-local/15.2.0
if [ -e ${PACMANDIR} ]; then
    echo --info: ${PACMANDIR} exists
    echo --info: cd ${PACMANDIR}/pacman-*
    cd ${PACMANDIR}/pacman-*
    echo --info: pwd: `pwd`
    echo --info: pacman setup
    source setup.sh
else
    echo --info: mkdir -p and cd to ${PACMANDIR}
    mkdir -p ${PACMANDIR}
    cd ${PACMANDIR}
    echo --info: download latest pacman
    wget http://atlas.bu.edu/~youssef/pacman/sample_cache/tarballs/pacman-latest.tar.gz
    echo --info: extract..
    tar xzf pacman-latest.tar.gz
    echo --info: cd pacman-*
    cd pacman-*
    echo --info: pwd: `pwd`
    echo --info: setup pacman..
    source setup.sh
fi
 
echo --info: mkdir -p, cd to ${ATLASINSTALLDIR}
mkdir -p ${ATLASINSTALLDIR}
cd ${ATLASINSTALLDIR}
echo --info: pwd: `pwd`
 
echo --info: start installing ATLAS:
echo --info: invoke: pacman -pretend-platform SLC-4 -allow trust-all-caches -get am-IU:15.2.0
pacman -pretend-platform SLC-4 -allow trust-all-caches -get am-IU:15.2.0

With this script, the ATLAS Software 15.2.0 will get installed to /opt/atlas-local/15.2.0 while pacman resides at /opt/atlas-local/pacman. Adjust the version numbers to your needs. I use Indiana University mirror (am-IU). You perhaps want to modify this, too. Additionally, one could use :15.2.0+KV to automatically perform the Kit Validation after installation. This is a good idea for a new platform. But for CernVM 1.2.0 + group-atlas this KV always succeeds.

Furthermore, in CernVM: local ATLAS Software — the clean solution you can read about:

• how to set up CMT
• how to create a cmthome directory and a valid requirements file
• how to set up the runtime environment for an ATLAS Software Release

Have fun and let me know what you think!