The Language Applications Grid

The LAPPS Grid on Jetstream

This page describes how to create a LAPPS Grid instance on Jetstream, an NSF funded cloud computing system used by the LAPPS Grid to provide compute resources. Many of these notes are specific to LAPPS developers at Vassar and Brandeis, but the general flavor is the same for everyone. There are more notes on issues touched upon in this page in the readme file of the repository in https://github.com/lappsgrid-incubator/jetstream-scripts/.

To get a working LAPPS Grid on Jetstream you need to do the following:

1. Create a Ubuntu instance ready for LAPPS Grid install.
2. Install the LAPPS Grid on that instance, either by installing the LAPPS Grid from scratch or by using Docker images.

Starting an Ubuntu instance

While not always necessary for installing a LAPPS Grid on a Jetstream instance, it is useful to get onto the JetStream dashboard at https://jblb.jetstream-cloud.org/dashboard/auth/login/?next=/dashboard/, login with TACC as the domain and use your TACC username and password (password is the same as on XCEDE but username is different). You need to have service units allocated. There is a quick application process to get 200K service units allocated, which should allow you to run a grid for about a year.

You can create an instance from a local terminal by running the jetstream script. To run the script you first need to put three things in place.

1. You need to install the OpenStack client, see the OpenStack installation notes for details, but it should suffice to do the following:

   $ pip install python-openstackclient
   $ pip install python-neutronclient

   
   
2. You need a shell script named openrc.sh to configure the OpenStack API. See https://github.com/lappsgrid-incubator/jetstream-scripts/ on how to get it. This script will set some environment variables in your shell and it needs to be sourced before your run jetstream, when sourced it will ask for your OpenStack password.

3. You need a key file created on the Jetstream dashboard (at Compute > Access & Security > Key Pairs). Let's say this file is named jetstream.pem. The jetstream script uses two variables, KEY and PEM, to control access to Jetstream. The PEM variable stores the location of the key file and the KEY variable has the base name of the file without the .pem extension. The jetstream script itself uses a default for KEY named lappsgrid-shared-key, but you can overrule this with an environment variable (see below). Put the key file either in the same directory as the jetstream script or put it in the ~/.ssh directory. If you insist in putting it elsewhere you will have to edit the jetstream script and set the PEM variable manually.

We can now run the jetstream script. In the example below we assume that openrc.sh has not been sourced yet and we assume you are not using the default key name but instead use the key file named jetstream.pem.

$ source openrc.sh
$ export OS_KEY=jetstream
$ wget http://downloads.lappsgrid.org/scripts/jetstream
$ ./jetstream.sh list
$ ./jetstream.sh ssh proxy

NOTE: the above may not work anymore, check.

The first two commands only need to be done only once after you have opened a new terminal and obviously you only need to run the third command if you had not already downloaded jetstream. The list command gives you a list of available instances and the ssh command gets you into an instance (you can either give an IP address or a name from the instance as printed by the list command, it can actually be a substring of the name, so proxy in our case will match lappsgrid-proxy). The proxy instance is a special instance used by LAPPS Grid developers that allows you to log on to all the other instances even if those instances do not have a public IP address, it is not available to other users..

You can start a new instance by doing one of the following:

$ ./jetstream launch test
$ ./jetstream launch --ip free test
$ ./jetstream launch --ip alloc test

All three will create a new instance named lappsgrid-test based on an existing image or snapshot, which by default is the ubuntu image (see below). In the first case the instance will not have a public IP address whereas in the second and third case either one of the avaiable floating IP addresses will be used or a new floating IP address will be allocated. Use the third only when the second invocation does not work (either because there are no free floating IPs or because your version of the OpenStack client does not include neutron, which is used by the second jetstream invocation and which is a known issue). For more details on available commands see https://github.com/lappsgrid-incubator/jetstream-scripts/ or run jetstream help.

Once you have a running instance you can get its local and public IP addresses with jetstream list.

The jetstream script refers to four LAPPS Grid images on Jetstream:

CENTOS_IMAGE=217a24b4-2338-4802-8221-fff3bf3fc260
UBUNTU_IMAGE=d7fe3289-943f-4417-90e8-8a6b171677ca
DOCKER_IMAGE=f3847d59-6a90-43da-8666-b58699398b9a
MASTER_IMAGE=f9e0fa21-6cad-49fc-97db-efd3812fc199

When on the Dashboard you can list all the images available, which includes the four above (although they are not listed by the identifier, you have to select the image to see those details; there is also a bug in the display page for the images in that you may have to click the public tab when you get to what seems to be the last page). The image names associated with the lines above are:

When you do a listing of the current instances the images are displayed with those names.

The download directory at http://downloads.lappsgrid.org/scripts is a copy of the contents of the repository https://github.com/lappsgrid-incubator/jetstream-scripts. Note that downloads.lappsgrid.org/scripts is hosted on the proxy used on Jetstream. Also note that the repository is called jetstream-scripts, but all scripts except for jetstream are generic ubuntu scripts.

Installing the LAPPS Grid

To install a Service Manager, login to your instance with jetstream ssh and then follow the instructions in Installing a Service Manager. The short version of the instructions is repeated here:

$ wget http://downloads.lappsgrid.org/service-manager/setup.sh
$ chmod +x setup.sh
$ sudo ./setup.sh

This should work if the image you are running this on is an ubuntu image or a CentOS image.