Chapter 4. API Interfaces

4.1. Installing and Configuring EC2Stack

CloudStack features a native EC2 query interface called awsapi that can be run on the management server. EC2Stack is a new project by CloudStack committer Ian Duffy and his friend Darren Brogan from University College Dublin. They did this as part of their third-year school project. Building on their previous experience with gstack (see Recipe 4.7), a GCE interface to CloudStack, they wrote a brand new EC2 interface to CloudStack.

The interface uses Flask microframework and is written 100% in Python. It also features a Vagrant box for easy testing, lots of unit tests, and automatic build tests (pep8, pylint and coverage) via Travis CI. As part of Google Summer of Code 2014, Darren Brogan is enhancing EC2Stack with additional API and unit tests.

Problem

You want an AWS EC2 compliant interface to your CloudStack cloud in order to use AWS clients like the AWS CLI or Python Boto (Recipe 4.4).

Solution

Download EC2Stack from the Python package index or install it from source by cloning the GitHub repository.

Discussion

Install EC2Stack using pip in a single operation:

$ sudo pip install ec2stack

If you want to do it from source and check out the code, then clone the Git repository and install it by hand:

$ git clone https://github.com/BroganD1993/ec2stack.git
$ sudo python ./setup.py install

You will now have ec2stack and ec2stack-configure binaries in your path. Before running the application, you will need to configure it. As an example, to set it up with exoscale do the following:

$ ec2stack-configure
EC2Stack bind address [0.0.0.0]:
EC2Stack bind port [5000]:
Cloudstack host [localhost]: api.exoscale.ch
Cloudstack port [8080]: 443
Cloudstack protocol [http]: https
Cloudstack path [/client/api]: /compute
Cloudstack custom disk offering name [Custom]:
Cloudstack default zone name: CH-GV2
Do you wish to input instance type mappings? (Yes/No): Yes
Insert the AWS EC2 instance type you wish to map: m1.small
Insert the name of the instance type you wish to map this to: Tiny
Do you wish to add more mappings? (Yes/No): No
INFO  [alembic.migration] Context impl SQLiteImpl.
INFO  [alembic.migration] Will assume non-transactional DDL.

Note

Note that we created a mapping between the AWS m1.small instance type and the Tiny instance type in exoscale. You could add more mappings.

You are now ready to run ec2stack. This setup process will run the application on the foreground. Because it is a Flask application, you can deploy it as a service in several ways. The Flask Documentation has good tips to do this. For testing, you can run it in the foreground; EC2Stack will be listening for requests and be ready to forward them to your CloudStack cloud:

$ ec2stack
 * Running on http://0.0.0.0:5000/
 * Restarting with reloader

4.2. Using the AWS CLI with EC2Stack

Problem

With EC2Stack running, you want to use the AWS CLI to make calls to your CloudStack cloud.

Solution

Install the AWS CLI from PyPI and configure it. Register your API keys with EC2Stack and start making requests to your cloud.

Discussion

Now that you are running EC2Stack on your local machine, you can use the AWS CLI to make calls to it. Install the CLI with:

$ sudo pip install awscli

Warning

The Python AWS CLI available on PyPI may change often, which can cause EC2Stack to break. You can install a specific awscli package with:

$ sudo pip install awscli==1.3.10

Currently, EC2Stack stack has been tested with 1.3.10.

In addition, you need to register your API keys with the AWS CLI. If you have not used exoscale (as in the preceding example), then choose the keys of your own CloudStack deployment and use the appropriate region name:

$ aws configure
AWS Access Key ID [None]: PQogHs2sk_3uslfvrASjQFDlZbt0mEDd14iN
AWS Secret Access Key [None]: aHuDB2ewpgxVuQlvD9P1o313BioI1W4v
Default region name [None]: CH-GV2
Default output format [None]:

You can see these settings in the ~/.aws/config file. Check the AWS CLI reference for further customization. The output format can be json, text, or table.

With your AWS CLI installed and configured, the final configuration step is to register a user with EC2Stack. To be on the safe side, upgrade the requests module:

$ sudo pip install --upgrade requests

Register your API keys like so:

$ ec2stack-register http://localhost:5000 <API accesskey> <API secret key>

The command should return a Successfully Registered! message. At this stage, you are now ready to use the AWS CLI (or Boto) and send requests to the EC2Stack endpoint:

$ aws ec2 describe-images --endpoint=http://localhost:5000
$ aws ec2 describe-key-pairs --endpoint=http://localhost:5000
$ aws ec2 create-key-pair --endpoint=http://localhost:5000 --key-name=test

To start an instance, for example:

$ aws ec2 run-instances --image-id=20d4ebc3-8898-431c-939e-adbcf203acec
 --endpoint=http://localhost:5000

The image-id parameter is the CloudStack uuid corresponding to the template that you want to start. You find it by running the aws describe-images call.

4.3. Improving the EC2Stack API Coverage

Problem

The EC2Stack API coverage does not cover all the AWS EC2 API. You want to add an API.

Solution

Because EC2Stack is open source, you can easily contribute to it and add the API that you need. Fork the project on GitHub and submit a pull request.

Discussion

Looking at the code, only the following AWS APIs are covered (more are being added through a Google Summer of Code 2014 project):

def _get_action(action):
        actions = {
        'AttachVolume': volumes.attach_volume,
        'AuthorizeSecurityGroupEgress':
         security_groups.authenticate_security_group_egress,
        'AuthorizeSecurityGroupIngress':
         security_groups.authenticate_security_group_ingress,
        'CreateKeyPair': keypairs.create_keypair,
        'CreateSecurityGroup': security_groups.create_security_group,
        'CreateTags': tags.create_tags,
        'CreateVolume': volumes.create_volume,
        'DeleteKeyPair': keypairs.delete_keypair,
        'DeleteSecurityGroup': security_groups.delete_security_group,
        'DeleteTags': tags.delete_tags,
        'DeleteVolume': volumes.delete_volume,
        'DescribeAvailabilityZones': zones.describe_zones,
        'DescribeImageAttribute': images.describe_image_attribute,
        'DescribeImages': images.describe_images,
        'DescribeInstanceAttribute': instances.describe_instance_attribute,
        'DescribeInstances': instances.describe_instances,
        'DescribeKeyPairs': keypairs.describe_keypairs,
        'DescribeSecurityGroups': security_groups.describe_security_groups,
        'DescribeTags': tags.describe_tags,
        'DescribeVolumes': volumes.describe_volumes,
        'DetachVolume': volumes.detach_volume,
        'GetPasswordData': passwords.get_password_data,
        'ImportKeyPair': keypairs.import_keypair,
        'RebootInstances': instances.reboot_instance,
        'RegisterSecretKey': register_secret_key,
        'RemoveSecretKey': remove_secret_key,
        'RevokeSecurityGroupEgress':
         security_groups.revoke_security_group_egress,
        'RevokeSecurityGroupIngress':
         security_groups.revoke_security_group_ingress,
        'RunInstances': instances.run_instance,
        'StartInstances': instances.start_instance,
        'StopInstances': instances.stop_instance,
        'TerminateInstances': instances.terminate_instance,
            }

Warning

Currently, EC2Stack is geared toward CloudStack basic zones. It is aimed at clouds that resemble AWS EC2 and want to allow access via SSH key pairs and security groups. Virtual private clouds (VPCs) are not currently supported.

The code is quite clean and it will be easy to add more actions and provide a much better coverage really soon. Pull requests are welcome if you are interested to contribute.

4.4. Using Python Boto with EC2Stack

There are many tools available to interface with an AWS-compatible API. In the previous recipe, we saw how to use the AWS CLI, but now let’s briefly look at Boto. Boto is a Python package that provides client-side bindings to work with the AWS API. It interfaces with EC2 but also with S3, CF, EMR, and so on. Boto has extensive documentation for each AWS service it supports.

Problem

You are familiar with Boto and want to use it with your CloudStack cloud.

Solution

Install Boto from the Python Package Index, and install and run an AWS EC2 interface to CloudStack. Write a Python script that imports the Boto module and creates a connection object to your EC2 interface endpoint.

Discussion

Installation is as easy as:

$ sudo pip install boto

With Boto installed on your client machine and an AWS EC2 interface running in front of CloudStack (see Recipe 4.1), you can now use the following script to start instances. Just replace the access and secret keys with your own and update the endpoint:

#!/usr/bin/env python

import sys
import os
import boto
import boto.ec2

region = boto.ec2.regioninfo.RegionInfo(endpoint="localhost")
apikey='GwNnpUPrO6KgIdZu01z_ZhhZnKjtSdRwuYd4DvpzvFpyxGMvrzno2q05MB0ViBoFYtdqKd'
secretkey='t4eXLEYWw7chBhDlaKf38adCMSHx_wlds6JfSx3z9fSpSOm0AbP9Moj0oGIzy2LSC8iw'

def main():
    '''Establish connection to EC2 cloud'''
    conn =boto.connect_ec2(aws_access_key_id=apikey,
                           aws_secret_access_key=secretkey,
                           is_secure=False,
                           region=region,
                           port=5000,
                           path="/",
                           api_version="2014-02-01")

    '''Get list of images that I own'''
    images = conn.get_all_images()
    myimage = images[0]

        '''Pick an instance type'''
    vm_type='m1.small'
    reservation = myimage.run(instance_type=vm_type,security_groups=['default'])

if __name__ == '__main__':
    main()

Warning

With Boto, you can also interact with other AWS services like S3. CloudStack has an S3 tech preview but it is backed by a standard NFS server and therefore is not a true scalable distributed block store. It will be removed from the code in an upcoming release. To provide an S3 service in your cloud, I recommend using other software like RiakCS, Ceph radosgw, or Glusterfs S3 interface. These systems handle large-scale chunking and replication.

This script will start an instance of type m1.small in a zone with security groups enabled. You could pass additional parameters to the run method like a key pair. If you are like me, you might like to have an interactive shell to your clouds, which means you might want to use Boto in a slightly different way. I use IPython to get an interactive shell, with tab completion, history, and logging capabilities. My shell script is:

#!/usr/bin/env python

import boto
import boto.ec2
from IPython.terminal.embed import InteractiveShellEmbed

accesskey="my api key"
secretkey="my secret key"

region = boto.ec2.regioninfo.RegionInfo(endpoint="localhost")
conn = boto.connect_ec2(aws_access_key_id=accesskey,
                        aws_secret_access_key=secretkey,
                                                is_secure=False,
                                                region=region,
                                                port=5000,
                                                path="/",
                                                api_version="2014-02-01")

ipshell = InteractiveShellEmbed(banner1="Hello, Cloud Shell!")
ipshell()

Starting this interactive shell, you can discover all the methods available in the connection object by entering conn. and pressing the Tab key. The AWS interfaces to CloudStack do not yet have 100% fidelity with the AWS API, so keep in mind that not all Boto methods will work.

Note

In this example, I used the EC2Stack interface, but you could also use the interface that comes natively bundled with CloudStack. Using the packages, it can be started with service cloudstack-awsapi start. I personally prefer EC2Stack because I was involved in the development and it supports a newer api_version.

4.5. Installing Eutester to Test the AWS Compatibility of Your CloudStack Cloud

Eutester was created by the folks at Eucalyptus to provide a framework to create functional tests for AWS zones and Eucalyptus-based clouds. What is interesting with Eutester is that it could be used to compare the AWS compatibility of multiple clouds. Therefore, you might be wonder, “Can we use Eutester with CloudStack?” And the answer is Yes. Certainly it could use more work, but the basic functionality is there. It allows you to write test scenarios and compare the results between an AWS EC2 availability zone and a CloudStack cloud.

Problem

You want to install Eutester to write integration tests for your AWS EC2 compliant cloud endpoints.

Solution

Grab the binary from the Python package index with pip or build it from source from GitHub.

Discussion

Install eutester with:

$ sudo pip install eutester

Warning

The master branch of eutester may still cause problems to list images from a CloudStack cloud. I recently patched a fork of the testing branch and opened an issue on their GitHub page. You might want to check its status if you want to use Eutester heavily.

To use eutester with CloudStack, clone the testing branch of my Eutester fork. Then install it by hand:

$ git clone -b testing https://github.com/runseb/eutester.git
$ cd eutester
$ sudo python ./setup.py install

4.6. Using Eutester with EC2Stack to Write Functional tests

Problem

You have installed Eutester and want to write a Python script to issue requests to your CloudStack cloud.

Solution

Import the eucaops module in a Python script, and create a connection object using your endpoint information and your API keys. To explore your cloud interactively and create testing scenarios, use IPython.

Discussion

Start a Python/IPython interactive shell or write a script that will import ec2ops and create a connection object to your AWS EC2 compatible endpoint. For example, using EC2Stack from Recipe 4.1:

#!/usr/bin/env python

from eucaops import ec2ops
from IPython.terminal.embed import InteractiveShellEmbed

accesskey="my api key"
secretkey="my secret key"

conn.ec2ops.EC2ops(endpoint="localhost",
                   aws_access_key_id=apikey,
                   aws_secret_access_key=secretkey,
                   is_secure=False,
                   port=5000,
                   path="/",
                   APIVersion="2014-02-01")

ipshell = InteractiveShellEmbed(banner1="Hello, Cloud Shell!")
ipshell()

Eutester, at the time of this writing, has 145 methods. Only the methods available through the CloudStack AWS EC2 interface that you will be using will be available. For example, get_zones and get_instances would return:

In [3]: conn.get_zones()
Out[3]: [u'ch-gva-2']

In [4]: conn.get_instances()
[2014-05-21 05:39:45,094] [EUTESTER] [DEBUG]:
--->(ec2ops.py:3164)Starting method: get_instances(self, state=None,
     idstring=None, reservation=None, rootdevtype=None, zone=None,
     key=None, pubip=None, privip=None, ramdisk=None, kernel=None,
     image_id=None, filters=None)
Out[4]:
[Instance:5a426582-3aa3-49e0-be3f-d2f9f1591f1f,
 Instance:95ee8534-b171-4f79-9e23-be48bf1a5af6,
 Instance:f18275f1-222b-455d-b352-3e7b2d3ffe9d,
 Instance:0ea66049-9399-4763-8d2f-b96e9228e413,
 Instance:7b2f63d6-66ce-4e1b-a481-e5f347f7e559,
 Instance:46d01dfd-dc81-4459-a4a8-885f05a87d07,
 Instance:7158726e-e76c-4cd4-8207-1ed50cc4d77a,
 Instance:14a0ce40-0ec7-4cf0-b908-0434271369f6]

This example shows that I am running eight instances at the moment in a zone called ch-gva-2, one zone of the exoscale cloud. Selecting one of these instance objects will give you access to all the methods available for instances. You could also list, delete, and create key pairs; list, delete, and create security groups; and so on.

Note

Eutester is meant for building integration tests and easily creating test scenarios. If you are looking for a client to build an application with, use Boto from Recipe 4.4.

4.7. Installing and Configuring gstack: The CloudStack GCE Interface

Google Compute Engine (GCE) is the Google public cloud. In December 2013, Google announced the General Availability (GA) of GCE. With AWS and Microsoft Azure, it is one of the three leading public clouds in the market. CloudStack has a GCE compatible interface that lets users use the GCE clients (i.e., gcloud and gcutil) to access their CloudStack cloud. Like EC2Stack, gstack is a Python Flask application that provides a REST API compatible with the GCE API and forwards the requests to the corresponding CloudStack API. The source is available on GitHub and the binary is downloadable via PyPI.

Problem

You want to install gstack on your machine.

Solution

Grab the binary from the Python package index with pip or clone the source code from GitHub.

Discussion

You can grab the gstack binary package from PyPI using pip in one single command:

$ sudo pip install gstack

Or, if you plan to explore the source and work on it, you can Clone the repository and install it by hand:

$ git clone https://github.com/NOPping/gstack.git
$ sudo python ./setup.py install

Both of these installation methods will install a gstack and a gstack-configure binary in your path.

Before running gstack you must configure it. To do so, run the following:

$ gstack-configure

And enter your configuration information when prompted. You will need to specify the host and port where you want gstack to run on, as well as the CloudStack endpoint that you want gstack to forward the requests to. In the following example, we use the exoscale cloud:

$ gstack-configure
gstack bind address [0.0.0.0]: localhost
gstack bind port [5000]:
Cloudstack host [localhost]: api.exoscale.ch
Cloudstack port [8080]: 443
Cloudstack protocol [http]: https
Cloudstack path [/client/api]: /compute

The information will be stored in a configuration file available at ~/.gstack/gstack.conf:

$ cat ~/.gstack/gstack.conf
PATH = 'compute/v1/projects/'
GSTACK_BIND_ADDRESS = 'localhost'
GSTACK_PORT = '5000'
CLOUDSTACK_HOST = 'api.exoscale.ch'
CLOUDSTACK_PORT = '443'
CLOUDSTACK_PROTOCOL = 'https'
CLOUDSTACK_PATH = '/compute'

You can start gstack as easily as this:

$ gstack

Tip

Like EC2Stack, this will run gstack in the foreground. This is acceptable for testing purposes but if you want to run gstack as a service in production setup, look at some of the WSGI HTTP servers that can be used to serve gstack. In production, you will also need to create a properly signed certificate for gstack and replace the self-signed certificate.

4.8. Using gstack with the gcutil Tool

Problem

With gstack installed and running on your machine, you want to use the gcutil command-line tool to issue requests to your CloudStack cloud.

Solution

Install and configure the standalone gcutil tool and start issuing commands to CloudStack via your running gstack server.

Discussion

The current version of gstack only works with the standalone version of gcutil.

Warning

Do not use the version of gcutil bundled in the Google Cloud SDK. Instead, install the 0.14.2 version of gcutil.

gstack comes with a self-signed certificate for the local endpoint gstack/data/server.crt, copy the certificate to the gcutil certificates file gcutil/lib/httplib2/httplib2/cacerts.txt.

At this stage, your CloudStack API key and secret key need to be entered in the gcutil auth_helper.py file in the gcutil/lib/google_compute_engine/gcutil/ directory.

This is far from ideal. Thus, we opened a feature request with Google to pass the client_id and client_secret as options to gcutil. Hopefully a future release of gcutil will allow us to do so.

Create a cached parameters file for gcutil. Assuming you are running gstack on your local machine, use the defaults that were suggested during the configuration phase. Modify ~/.gcutil_params with the following:

--auth_local_webserver
--auth_host_port=9999
--dump_request_response
--authorization_uri_base=https://localhost:5000/oauth2
--ssh_user=root
--fetch_discovery
--auth_host_name=localhost
--api_host=https://localhost:5000/

Warning

Make sure to set the --auth_host_name variable to the same value as GSTACK_BIND_ADDRESS in your ~/.gstack/gstack.conf file. Otherwise you will see certificates errors.

With this setup complete, gcutil will issue requests to the local Flask application, get an OAuth token, issue requests to your CloudStack endpoint, and return the response in a GCE compatible format.

With the setup steps complete, you can start issuing standard gcutil commands. For illustration purposes, we use exoscale.

Note

Because there are several semantic differences, you will notice that as a project, we use the account information from CloudStack. Hence, we pass our email address as the project value. This is another area that could be improved.

Let’s start by listing the availability zones:

$ gcutil --cached_flags_file=~/.gcutil_params
         --project=runseb@gmail.com listzones
+----------+--------+------------------+
| name     | status | next-maintenance |
+----------+--------+------------------+
| ch-gva-2 | UP     | None scheduled   |
+----------+--------+------------------+

Let’s list the machine types (or, in CloudStack terminology, the compute service offerings and the list of available images):

$ gcutil --cached_flags_file=~/.gcutil_params
         --project=runseb@gmail.com listmachinetypes
+-------------+----------+------+-----------+-------------+
| name        | zone     | cpus | memory-mb | deprecation |
+-------------+----------+------+-----------+-------------+
| Micro       | ch-gva-2 |    1 |       512 |             |
+-------------+----------+------+-----------+-------------+
| Tiny        | ch-gva-2 |    1 |      1024 |             |
+-------------+----------+------+-----------+-------------+
| Small       | ch-gva-2 |    2 |      2048 |             |
+-------------+----------+------+-----------+-------------+
| Medium      | ch-gva-2 |    2 |      4096 |             |
+-------------+----------+------+-----------+-------------+
| Large       | ch-gva-2 |    4 |      8182 |             |
+-------------+----------+------+-----------+-------------+
| Extra-large | ch-gva-2 |    4 |     16384 |             |
+-------------+----------+------+-----------+-------------+
| Huge        | ch-gva-2 |    8 |     32184 |             |
+-------------+----------+------+-----------+-------------+
$ ./gcutil --cached_flags_file=~/.gcutil_params
           --project=runseb@gmail.com listimages
+---------------------------------+-------------+--------+
|              name               | deprecation | status |
+---------------------------------+-------------+--------+
| CentOS 5.5(64-bit) no GUI (KVM) |             | Ready  |
| Linux CentOS 6.4 64-bit         |             | Ready  |
| Linux CentOS 6.4 64-bit         |             | Ready  |
| Linux CentOS 6.4 64-bit         |             | Ready  |
| Linux CentOS 6.4 64-bit         |             | Ready  |
| Linux CentOS 6.4 64-bit         |             | Ready  |
| Linux Ubuntu 12.04 LTS 64-bit   |             | Ready  |
| Linux Ubuntu 12.04 LTS 64-bit   |             | Ready  |
| Linux Ubuntu 12.04 LTS 64-bit   |             | Ready  |
| Linux Ubuntu 12.04 LTS 64-bit   |             | Ready  |
| Linux Ubuntu 12.04 LTS 64-bit   |             | Ready  |
| Linux Ubuntu 13.04 64-bit       |             | Ready  |
| Linux Ubuntu 13.04 64-bit       |             | Ready  |
| Linux Ubuntu 13.04 64-bit       |             | Ready  |
| Linux Ubuntu 13.04 64-bit       |             | Ready  |
| Linux Ubuntu 13.04 64-bit       |             | Ready  |
| Windows Server 2008 R2 SP1      |             | Ready  |
| Windows Server 2008 R2 SP1      |             | Ready  |
| Windows Server 2008 R2 SP1      |             | Ready  |
| Windows Server 2008 R2 SP1      |             | Ready  |
| Windows Server 2012             |             | Ready  |
| Windows Server 2012             |             | Ready  |
| Windows Server 2012             |             | Ready  |
| Windows Server 2012             |             | Ready  |
+---------------------------------+-------------+--------+

You can also list firewalls, which gstack maps to CloudStack security groups. To create a security group, use the firewall commands:

$ ./gcutil --cached_flags_file=~/.gcutil_params
           --project=runseb@gmail.com addfirewall ssh --allowed=tcp:22

And get the details of this firewall with getfirewall:

$ ./gcutil --cached_flags_file=~/.gcutil_params
           --project=runseb@gmail.com getfirewall ssh
+---------------+-----------+
|   property    |   value   |
+---------------+-----------+
| name          | ssh       |
| description   |           |
| creation-time |           |
| network       |           |
| source-ips    | 0.0.0.0/0 |
| source-tags   |           |
| target-tags   |           |
| allowed       | tcp: 22   |
+---------------+-----------+

To start an instance, you can follow the interactive prompt given by gcutil. You will need to pass the --permit_root_ssh flag, another one of those semantic and access configuration details that needs to be ironed out. The interactive prompt will let you choose the machine type and the image that you want; it will then start the instance:

$ ./gcutil --cached_flags_file=~/.gcutil_params
           --project=runseb@gmail.com addinstance foobar
Selecting the only available zone: CH-GV2
1: Extra-large  Extra-large 16384mb 4cpu
2: Huge Huge 32184mb 8cpu
3: Large    Large 8192mb 4cpu
4: Medium   Medium 4096mb 2cpu
5: Micro    Micro 512mb 1cpu
6: Small    Small 2048mb 2cpu
7: Tiny Tiny 1024mb 1cpu
7
1: CentOS 5.5(64-bit) no GUI (KVM)
2: Linux CentOS 6.4 64-bit
3: Linux CentOS 6.4 64-bit
4: Linux CentOS 6.4 64-bit
5: Linux CentOS 6.4 64-bit
6: Linux CentOS 6.4 64-bit
<...snip>
INFO: Waiting for insert of instance . Sleeping for 3s.
INFO: Waiting for insert of instance . Sleeping for 3s.

Table of resources:

+--------+--------------+--------------+----------+---------+
| name   | network-ip   | external-ip  | zone     | status  |
+--------+--------------+--------------+----------+---------+
| foobar | 185.1.2.3    | 185.1.2.3    | ch-gva-2 | RUNNING |
+--------+--------------+--------------+----------+---------+

Table of operations:

+--------------+--------+--------------------------+----------------+
| name         | status | insert-time              | operation-type |
+--------------+--------+--------------------------+----------------+
| e4180d83-31d0| DONE   | 2014-06-09T10:31:35+0200 | insert         |
+--------------+--------+--------------------------+----------------+

You can, of course, list (with listinstances) and delete instances:

$ ./gcutil --cached_flags_file=~/.gcutil_params --project=runseb@gmail.com
           deleteinstance foobar
Delete instance foobar? [y/n]
y
WARNING: Consider passing '--zone=CH-GV2' to avoid the unnecessary
         zone lookup which requires extra API calls.
INFO: Waiting for delete of instance . Sleeping for 3s.
+--------------+--------+--------------------------+----------------+
| name         | status | insert-time              | operation-type |
+--------------+--------+--------------------------+----------------+
| d421168c-4acd| DONE   | 2014-06-09T10:34:53+0200 | delete         |
+--------------+--------+--------------------------+----------------+

gstack is still a work in progress, but it is now compatible with the GCE GA v1.0 API. The few differences in API semantics need to be investigated further and additional API calls need to be supported. However, it provides a solid base to start working on hybrid solutions between a GCE public cloud and a CloudStack-based private cloud.

4.9. Supporting the OCCI Standard in CloudStack

The Open Cloud Computing Interface (OCCI) is a standard from the Open Grid Forum (OGF). OCCI was originally created to be a remote management API for the IaaS layer of cloud computing but has since evolved to also address the PaaS and SaaS layers. With CIMI, it is one of the two cloud standards for cloud providers’ APIs backed by a standards organization. As we mentioned several times already, CloudStack has its own API, which is not a standard. AWS EC2 and Google GCE are not standards either. Cloud wrappers like libcloud and jclouds work well as unofficial standards that abstract differences in cloud providers’ APIs. Users interested in using OCCI or CIMI will need a wrapper on top of the CloudStack API that will provide a CIMI or OCCI implementation. There is currently no CIMI interface for CloudStack, but through rOCCI, there is an OCCI interface. The rest of this recipe goes through installation, configuration, and usage of the rOCCI CloudStack driver.

Note

There are several implementations of OCCI. rOCCI is one of them and is currently going through some refactoring.

Problem

The CloudStack API is very nice, but it is not backed by a standards organization. You care about standards and you would like to use OCCI from the Open Grid Forum to interact with your CloudStack cloud, removing any potential issues with a nonstandard API.

Solution

Install a rOCCI server in your infrastructure and configure to use the CloudStack driver. You can then use any OCCI client to issue cloud requests to it. rOCCI will forward the requests to the CloudStack API server and send the appropriate response back.

Discussion

Using OCCI with CloudStack involves running the rOCCI server and using an OCCI client to interface to it. The rOCCI server provides the API mapping between the OCCI standard API and the CloudStack API. In this discussion, the rOCCI client is used to issue OCCI API requests instead of using a CloudStack-specific client like CloudMonkey.

Install and run the rOCCI server

As we have done several times now, you can install the rOCCI server by cloning the project on GitHub, doing a build using Ruby’s bundler, and setting up some configuration files:

$ git clone https://github.com/isaacchiang/rOCCI-server.git
$ gem install bundler
$ bundle install
$ cd etc/backend
$ cp cloudstack/cloudstack.json default.json

Note

The rOCCI CloudStack backend is experimental and not merged in the rOCCI server project yet.

Edit the default.json file to contain the information about your CloudStack cloud, your endpoint, and your API keys. Start the rOCCI server in one shell:

$ bundle exec passenger start

The server should be running on http://0.0.0.0:3000, and you can try to run the basic tests:

 $ bundle exec rspec

You are ready to install an OCCI client and use it to talk to the rOCCI server you just started. The requests will be forwarded to your CloudStack endpoint.

Install the rOCCI client

Clone the rOCCI-cli client from GitHub. Use bundler again to build it and rake to install it:

$ git clone https://github.com/gwdg/rOCCI-cli.git
$ cd rOCCI-cli
$ bundle install
$ bundle exec rake test
$ rake install

An occi binary should now be in your path. Try to use it by running the --help option:

$ occi --help

Test the OCCI client against the server

With your cloud endpoint and your API keys properly defined earlier in the json configuration file, you can use the OCCI client to list templates, locations, and start an instance. Similar to EC2Stack, you need to specify the endpoint of the rOCCI server that’s running. Try a couple OCCI client commands:

$ occi --endpoint http://0.0.0.0:3000/ --action list --resource os_tpl

Os_tpl locations:
    os_tpl#6673855d-ce9b-4997-8613-6830de037a8f

$ occi --endpoint http://0.0.0.0:3000/ --action list --resource resource_tpl

Resource_tpl locations:
    resource_tpl##08ba0343-bd39-4bf0-9aab-4953694ae2b4
    resource_tpl##f78769bd-95ea-4139-ad9b-9dfc1c5cb673
    resource_tpl##0fd364a9-7e33-4375-9e10-bb861f7c6ee7

You will recognize the uuid from the templates and service offerings that you have created in CloudStack. These uuid will be different. To start an instance:

$ occi --endpoint http://0.0.0.0:3000/
       --action create
       --resource compute
       --mixin os_tpl#6673855d-ce9b-4997-8613-6830de037a8f
       --mixin resource_tpl#08ba0343-bd39-4bf0-9aab-4953694ae2b4
       --resource-title foobar

And voilà! The holy grail again, you started an instance on a CloudStack cloud using an OCCI client and an OCCI implementation on top of the CloudStack API.

Get 60 Recipes for Apache CloudStack now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.