There is currently one cluster running on the Jetstream IU cloud. The cluster uses Rancher for setup and management and Docker Swarm as the Orchestration Engine (OE).
- Create Nodes
- Assign IP Addresses
- Install Docker
- Launch Rancher
- Secure Rancher Very important! Do not skip this step.
- Add Worker Nodes
Note Additional worker nodes can be added at any time by running the
sudo docker run command from Step 6 on the new nodes.
To set up a cluster you will need to create one node (m1.small) for the Rancher service and one or more nodes (m1.medium or larger) as worker nodes. The number and size of the worker nodes will depend on the anticipated number and size of the services to be deployed. Services that require a large amount of memory or a large number of CPUs should be deployed to a cluster with large worker nodes, while smaller services can be deployed to clusters with smaller worker nodes.
1. Create a master node
You can use the Jetstream OpenStack UI to create the nodes or the OpenStack CLI tools. In either case use the following parameters to create the instances:
master node something like
Later on we will add
worker nodes that are controlled by the
master node will be running
rancher/server docker container that manages the clusters and provides web console, while
workers will be running
rancher/agent containers along with a container orchestration engine (OE).
$> source ~/.secret/openrc.sh $> openstack server create \ --image lappsgrid-ubuntu-16.04 \ --flavor m1.small \ --network lappsgrid-network \ --security-group lappsgrid-secgroup \ --key-name <YOUR_KEY_PAIR_NAME> \ master
If told that m1.small is too small for ubuntu-16.04 image, try m1.medium.
2. Assign a floating ip to the node
You will need to assign a public floating IP address to the master node.
$> openstack floating ip create public $> openstack server add floating ip master <IP-ADDRESS-FROM-ABOVE>
openstack floating ip create public will list the IP address that was allocated and this is the
<ip-address> that is used in the subsequent command.
3. Install Docker
You will need to install Docker 17.06-ce on the master node. SSH into each node:
$> ssh -i ~/.ssh/<YOUR_KEY_PAIR_NAME>.pem root@<ip-address>
Run the following command:
$root> curl https://releases.rancher.com/install-docker/17.06.sh | sh
When ubuntu-16.04 image starts up for the first time
dpkg, the built-in package manager used for docker installation, performs system update automatically.
You might get an error as the update process prevents other
dpkg operations, such as the installation of docker.
If this happens, just for a couple minutes and try again.
4. Launch the Rancher server
SSH into the master node and run the following command to launcher the Rancher server:
$root> docker run -d -p 8000:8080 --name rancher rancher/server:stable
After a few minutes you should be able to access the Rancher server at http://
5. Secure the Rancher Server
By default the Rancher server does no user authentication which means anyone on the Internet can deploy services to our cluster. THIS MUST BE CHANGED ASAP.
5a. Register Rancher with GitHub
- Go to Github application OAuth settings and click the
Register an applicationbutton.
- Enter something meaningful as the
Authorization callback URL, where
ip-addressis the public IP we assigned to the master (Rancher) node.
- Click the
- Make a note of the
Client Secretas you will need them below.
5b. Enable GitHub authentication in Rancher
- Open http://ip-address:8000 in a browser.
Access Controlfrom the
- Scroll down to section 2 and paste in the
Client Secretyou obtained above.
- Click the
- Click the
Authenticate with GitHubbutton. This will attempt to log you into the Rancher application using your GitHub account.
- A new broswer window will open. This is GitHub asking your permission to allow Rancher application to authorize. Since this is what we want click the
- The new browser window will close and you should be returned to the Rancher UI. In the
Site Accessarea add the following GitHub users:
Our Rancher server is now secure and only the GitHub users listed above can login to the server and deploy containers to our cluster.
A short (and slightly outdated) video showing the GitHub setup is available here.
6. Add Worker Nodes to the cluster.
Finally we add one or more worker nodes, which are called
hosts in Rancher manager, that actually runs micro Lapps services.
Start with firing up some new openstack servers, assigning public IP addresses.
$> openstack server create \ --image lappsgrid-ubuntu-16.04 \ --flavor m1.medium \ --network lappsgrid-network \ --security-group lappsgrid-secgroup \ --key-name <YOUR_KEY_PAIR_NAME> \ worker-1 ... $> openstack floating ip create public $> openstack server add floating ip worker-1 <ip-address-from-above> ...
Here we name worker nodes
Next we need to create a new cluster that rancher has control over. We will do that by creating a new
enviroment. While creating a new environment, you can select a container orchestration engine of your preference (Rancher has its native OE
Cattle but also support docker swarm, Kubernetes and Mesos.
In this tutorial we will specify Docker Swarm as the OE (which comes with a
portainer instance as a web-based manager application). First worker nodes need docker installed.
$> ssh -i ~/.ssh/<YOUR_KEY_PAIR_NAME>.pem root@<ip-address> $root> curl https://releases.rancher.com/install-docker/17.06.sh | sh # or $root> curl https://releases.rancher.com/install-docker/1.12.sh | sh
Note The process to set up a cluster using Kubernetes as the OE is almost identical with one important difference; to use Kubernetes with Rancher you need to install an older version of Docker:
$> curl https://releases.rancher.com/install-docker/1.12.sh | sh
You can see all the supported versions of Docker here.
Now in the Rancher manager (web browser),
- Go to
Default -> Manage environments
- Click the
- Enter a
Swarmas the Environment template.
- Click the
Don’t worry if the Rancher UI claims our new environment is
Unhealthy. It is unhealthy because it does not have any worker nodes so let’s add some nodes.
- Click the
Add a hostlink at the top of the page.
Customif it is not already selected.
- Scroll down to Section 5 and use the clipboard button to copy the entire
sudo docker runcommand.
- SSH into each worker node and paste in the above command:
$> ssh -i ~/.ssh/lappsgrid-shared-key.pem root@<ip-address> ... $root> sudo docker run --rm --privileged \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /var/lib/rancher:/var/lib/rancher \ rancher/agent:v1.2.6 \ http://220.127.116.11:8000/v1/scripts/A171767AD1C34F1F46BA:1483142400000:YXu6RtzQSIzcfx5dpLENWwJRY
NOTE The above command if for example purposes only. The command you paste in will look similar, but the IP address and “join token” will be different.
It may take several minutes for Rancher/Docker to configure the worker nodes. You can watch the setup progress by going to the
Infrastructure -> Hosts page.