Configuration¶
During the installation of cloudmesh it will automatically generate a configuration file in the directory:
~/.cloudmesh/cloudmesh.yaml
If this file is missing, you can run the command:
cm help to automatically generate it from defaults.
The file will be a template and it can either be modified with your favorite editor, or if you are at Indiana University and want to use the kilo cloud you can use the command
cm remote register
This will add the appropriate information into the yaml file. The file will be looking as follows. You will have several options to modify the file as explained bellow
meta:
yaml_version: 3.0
kind: clouds
cloudmesh:
profile:
firstname: TBD
lastname: TBD
email: TBD
username: None
github:
username: TBD
portal:
location: TBD
browser: firefox
comet:
auth_provider: userpass
userpass:
username: TBD
password: TBD
apikey:
api_key: TBD
api_secret: TBD
hpc:
experiment:
name: gregor-00000
active:
- comet
- juliet
clusters:
india:
cm_heading: India HPC CLuster
cm_host: india
cm_label: indiahpc
cm_type: slurm
cm_type_version: 14.11.9
credentials:
username: TBD
project: None
default:
queue: delta
experiment_dir: /N/u/{username}/experiment
prefix: {username}
comet:
cm_heading: Comet HPC CLuster
cm_host: comet
cm_label: comethpc
cm_type: slurm
cm_type_version: 14.11.9
credentials:
username: TBD
project: None
default:
queue: debug
experiment_dir: /home/{username}/experiment
prefix: {username}
active:
- kilo
clouds:
kilo:
cm_heading: India OpenStack, Kilo
cm_host: india
cm_label: kilo
cm_type: openstack
cm_type_version: kilo
cm_openrc: ~/.cloudmesh/clouds/india/kilo/openrc.sh
credentials:
OS_AUTH_URL: https://kilo.futuresystems.org:5000/v3
OS_PASSWORD: TBD
OS_TENANT_NAME: TBD
OS_USERNAME: TBD
OS_PROJECT_DOMAIN_ID: default
OS_USER_DOMAIN_ID: default
OS_PROJECT_NAME: TBD
OS_IMAGE_API_VERSION: 2
OS_VOLUME_API_VERSION: 2
default:
flavor: m1.small
image: Ubuntu-14.04-64
chameleon:
cm_heading: Chameleon
cm_host: chameleoncloud.org
cm_label: chameleon
cm_type: openstack
cm_type_version: kilo
credentials:
OS_AUTH_URL: https://openstack.tacc.chameleoncloud.org:5000/v2.0
OS_PASSWORD: TBD
OS_TENANT_NAME: TBD
OS_TENANT_ID: TBD
OS_PROJECT_NAME: TBD
OS_USERNAME: TBD
OS_VERSION: kilo
OS_REGION_NAME: RegionOne
default:
flavor: m1.small
image: Ubuntu-Server-14.04-LTS
cybera-c:
cm_heading: Cybera Calgary OpenStack
cm_host: cybera
cm_label: cybera-c
cm_type: openstack
cm_type_version: kilo
credentials:
OS_AUTH_URL: TBD
OS_TENANT_ID: TBD
OS_TENANT_NAME: TBD
OS_PROJECT_NAME: TBD
OS_USERNAME: TBD
OS_PASSWORD: TBD
OS_REGION_NAME: Calgary
default:
flavor: m1.small
image: Ubuntu 14.04
cybera-e:
cm_heading: Cybera Edmonton OpenStack
cm_host: cybera
cm_label: kilo
cm_type: openstack
cm_type_version: kilo
credentials:
OS_AUTH_URL: https://keystone-yyc.cloud.cybera.ca:5000/v2.0
OS_TENANT_ID: TBD
OS_TENANT_NAME: TBD
OS_PROJECT_NAME: TBD
OS_USERNAME: TBD
OS_PASSWORD: TBD
OS_REGION_NAME: Edmonton
default:
flavor: m1.small
image: Ubuntu 14.04
aws:
cm_heading: Amazon Cloud, AWS
cm_host: aws.amazon.com
cm_label: aws
cm_type: ec2
cm_type_version: null
credentials:
EC2_ACCESS_KEY: TBD
EC2_SECRET_KEY: TBD
keyname: TBD
userid: TBD
default:
flavor: t1.micro
image: ami-d85e75b0
location: us-east-1
chameleon-ec2:
cm_heading: Chameleon, EC2
cm_host: chameleoncloud.org
cm_label: chameleon_ec2
cm_type: ec2
cm_type_version: ec2
credentials:
EC2_ACCESS_KEY: TBD
EC2_SECRET_KEY: TBD
keyname: TBD_not_used
userid: TBD_not_used
EC2_URL: https://openstack.tacc.chameleoncloud.org:8773/services/Cloud
EC2_USER_ID: TBD
EC2_PRIVATE_KEY: ~/.cloudmesh/clouds/chameleon/TBD/pk.pem
EC2_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cert.pem
NOVA_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cacert.pem
EUCALYPTUS_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cacert.pem
default:
flavor: m1.small
image: Ubuntu-Server-14.04-LTS
azure:
cm_heading: Microsoft Azure Virtual Machines
cm_host: windowsazure.com
cm_label: azure
cm_type: azure
cm_type_version: null
credentials:
managementcertfile: TBD
servicecertfile: TBD
subscriptionid: TBD
thumbprint: TBD
default:
flavor: ExtraSmall
image: b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_2-LTS-amd64-server-20150610-en-us-30GB
location: East US
keys:
default: id_rsa
keylist:
id_rsa: ~/.ssh/id_rsa.pub
system:
data: ~/.cloudmesh/cloudmesh_inventory.yaml
console_color: true
logging:
file: ~/.cloudmesh/cloudmesh.log
level: DEBUG
You can modify the file by hand and replace the TBD values
according to your information about your cloud. You can add new clouds
or delete the once that you do not want.
Warning
Just as private keys should be kept private so does the cloudmesh.yaml file. Please, make sure the file is protected as it contains
sensitive information.
Get Registration from Indiana University¶
In case you have an account on http::/portal.futuresystems.org the integration can be done automatically for you with the account information available to you as previously explained.
The best way is to configure first your ssh client to conveniently log into india the machine where you can find the configuration information. To do so, please edit the following file
~/./ssh/config
and add the following lines to it
Host india
User: albert
Hostname: india.futuresystems.org
please replace albert with your portal name that you have used for registration with futuresystems.org. Once you have done this please verify that you have access to india with a command such as:
ssh india uname -a
Next register the FutureSystems clouds into your cloudmesh yaml file with the command:
cm register remote
This will update your cloudmesh.yaml file with the information retrieved from india. While retrieving the information on india from the file:
~/.cloudmesh/clouds/india/kilo/openrc.sh
Make sure you add a valid tenant to the yaml file. More information about using india can be found at http://portal.futuresystems.org
Registration of other clouds¶
The register command is quite powerful and useful and we encourage you to take a closer look at the manual pages. This includes command such as
To find out more about the registration command:
cm register help
To edit the yaml file with the editor defined by the Shell variable $EDITOR:
register edit
To list the cloudmesh.yaml file:
register list
Registration of Cybera Cloud¶
Cybera an organization from Canada provides an easy accessible openstack cloud. This cloud should only be used while following their access policies documented at:
YOu may ask for permission, if you do not fit this category. Once you have created an account at:
YOu can access to Openstack portal at
Just as Chameleon Cloud the Cybera cloud allows openstack rc and ec2 rc files.
Registration of Cybera Openstack Cloud¶
When you have an account ana a project it is simple to configure cloudmesh to include chameleon cloud in its resource set. To do so, edit the file:
~/.cloudmesh/cloudmesh.yaml
Edit the follwoing lines:
OS_PASSWORD: TBD
OS_TENANT_NAME: TBD
OS_TENANT_ID: TBD
OS_PROJECT_NAME: TBD
OS_USERNAME: TBD
Let us assume you have the username albert and the project id FG-101, Than the lines need to be changed to:
OS_PASSWORD: <your user password>
OS_TENANT_NAME: FG-101
OS_TENANT_ID: FG-101
OS_PROJECT_NAME: FG-101
OS_USERNAME: albert
You can find this information also in the openrc.sh file which you can download via the Openstack Horizon interface by following this link:
Registration of Cybera EC2 Cloud¶
Cybera cloud also support the usage of the EC2 interface which is a pit more complex to set up than the openstack configuration. First, you have to download a configuration directory, that is packaged as a zip file. This file can be found at
Let us assume you have the username albert and the project FG-101. Than the zip file will be called:
FG-101-x509.zip
Let us set some environment variables to make the configuration description easier
export C_USERNAME=<your cybera username> export C_PROJECT=<your cybera project name>
Unpack the zip file and place the entire directory in the .cloudmesh directory with. (We assume that you are in the directory where your browser downloaded the zip file and you have uncompressed it)
mkdir ~/.cloudmesh/clouds/cybera/$C_PROJECT cp $C_PROJECT ~/.cloudmesh/clouds/cybera/$C_PROJECT ls ~/.cloudmesh/clouds/cybera/$C_PROJECT
The directory should include the files:
cacert.pem
cert.pem
ec2rc.sh
pk.pem
Take a look at the ec2rc.sh file
cat ~/.cloudmesh/clouds/cybera/$C_PROJECT/ec2rc.sh
Now you can edit the cloudmesh yaml file at:
~/.cloudmesh/cloudmesh.yaml
locate the cybera-ec2 entry and change the TBD values with the values you see in the ec2rc.sh file:
EC2_ACCESS_KEY: <find the value in the ec2rc.sh file>
EC2_SECRET_KEY: <find the value in the ec2rc.sh file>
EC2_USER_ID: <find the value in the ec2rc.sh file>
For the following lines in the cloudmesh file, please replace the TBD values with the cybera project ID that you use for this cloud:
EC2_PRIVATE_KEY: ~/.cloudmesh/clouds/cybera/TBD/pk.pem
EC2_CERT: ~/.cloudmesh/clouds/cybera/TBD/cert.pem
NOVA_CERT: ~/.cloudmesh/clouds/cybera/TBD/cacert.pem
EUCALYPTUS_CERT: ~/.cloudmesh/clouds/cybera/TBD/cacert.pem
Chameleon Cloud¶
Registration of Chameleon Openstack Cloud¶
NSF sponsors an experimental cloud environment called Chameleon at
It is a KVM based Openstack cloud of version kilo. The documentation can be found here:
When you have an account and a project it is simple to configure cloudmesh to include chameleon cloud in its resource set. To do so, edit the file:
~/.cloudmesh/cloudmesh.yaml
Edit the follwoing lines:
OS_PASSWORD: TBD
OS_TENANT_NAME: TBD
OS_TENANT_ID: TBD
OS_PROJECT_NAME: TBD
OS_USERNAME: TBD
Let us assume you have the username albert and the project id FG-101, Than the lines need to be changed to:
OS_PASSWORD: <your user password>
OS_TENANT_NAME: FG-101
OS_TENANT_ID: FG-101
OS_PROJECT_NAME: FG-101
OS_USERNAME: albert
You can find this information also in the openrc.sh file which you can download via the Openstack Horizon interface by following this link:
Registration of Chameleon EC2 Cloud¶
The chameleon cloud also support the usage of the EC2 interface which is a pit more complex to set up than the openstack configuration. First, you have to download a configuration directory, that is packaged as a zip file. This file can be found at
Let us assume you have the username albert and the project FG-101. Than the zip file will be called:
FG-101-x509.zip
Let us set some environment variables to make the configuration description easier
export C_USERNAME=<your chameleon username> export C_PROJECT=<your chameleon project name>
Unpack the zip file and place the entire directory in the .cloudmesh directory with. (We assume that you are in the directory where your browser downloaded the zip file and you have uncompressed it)
mkdir ~/.cloudmesh/clouds/chameleon/$C_PROJECT cp $C_PROJECT ~/.cloudmesh/clouds/chameleon/$C_PROJECT ls ~/.cloudmesh/clouds/chameleon/$C_PROJECT
The directory should include the files:
cacert.pem
cert.pem
ec2rc.sh
pk.pem
Take a look at the ec2rc.sh file
cat ~/.cloudmesh/clouds/chameleon/$C_PROJECT/ec2rc.sh
Now you can edit the cloudmesh yaml file at:
~/.cloudmesh/cloudmesh.yaml
locate the chameleon-ec2 entry and change the TBD values with the values you see in the ec2rc.sh file:
EC2_ACCESS_KEY: <find the value in the ec2rc.sh file>
EC2_SECRET_KEY: <find the value in the ec2rc.sh file>
EC2_USER_ID: <find the value in the ec2rc.sh file>
For the following lines in the cloudmesh file, please replace the TBD values with the chameleon project ID that you use for this cloud:
EC2_PRIVATE_KEY: ~/.cloudmesh/clouds/chameleon/TBD/pk.pem
EC2_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cert.pem
NOVA_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cacert.pem
EUCALYPTUS_CERT: ~/.cloudmesh/clouds/chameleon/TBD/cacert.pem
Registration of CloudLab Openstack Cloud¶
Todo
not yet tested but should work. add cloud registration here
Registration of AWS Cloud¶
Todo
not yet supported but used to be so we work on it ASAP. add cloud registration here
Registration of Azure Cloud¶
Todo
not yet supported but used to be so we work on it ASAP. add cloud registration here
Registration of devcloud¶
Todo
not tested, but should work as is regular openstack. add cloud registration here
Registration of a libcloud available cloud¶
Todo
not yet supported. add cloud registration here