Chapter 4. API Interfaces
4.1. Installing and Configuring EC2Stack
CloudStack features a native EC2 query interface called awsapi that can be run on the
management server.
EC2Stack is a new project by
CloudStack committer Ian Duffy and his friend Darren Brogan from
University College Dublin. They did this as part of their third-year
school project. Building on their previous experience with
gstack (see Recipe 4.7), a GCE interface to
CloudStack, they wrote a brand new EC2 interface to CloudStack.
The interface uses Flask microframework and is written 100% in Python. It also features a Vagrant box for easy testing, lots of unit tests, and automatic build tests (pep8, pylint and coverage) via Travis CI. As part of Google Summer of Code 2014, Darren Brogan is enhancing EC2Stack with additional API and unit tests.
Problem
You want an AWS EC2 compliant interface to your CloudStack cloud in order to use AWS clients like the AWS CLI or Python Boto (Recipe 4.4).
Solution
Download EC2Stack from the Python package index or install it from source by cloning the GitHub repository.
Discussion
Install EC2Stack using pip in a single operation:
$ sudo pip install ec2stack
If you want to do it from source and check out the code, then clone the Git repository and install it by hand:
$ git clone https://github.com/BroganD1993/ec2stack.git $ sudo python ./setup.py install
You will now have ec2stack and ec2stack-configure binaries in your path. Before running the application, you will need to configure it. As an example, to set it up with exoscale do the following:
$ ec2stack-configure EC2Stack bind address [0.0.0.0]: EC2Stack bind port [5000]: Cloudstack host [localhost]: api.exoscale.ch Cloudstack port [8080]: 443 Cloudstack protocol [http]: https Cloudstack path [/client/api]: /compute Cloudstack custom disk offering name [Custom]: Cloudstack default zone name: CH-GV2 Do you wish to input instance type mappings? (Yes/No): Yes Insert the AWS EC2 instance type you wish to map: m1.small Insert the name of the instance type you wish to map this to: Tiny Do you wish to add more mappings? (Yes/No): No INFO [alembic.migration] Context impl SQLiteImpl. INFO [alembic.migration] Will assume non-transactional DDL.
Note
Note that we created a mapping between the AWS m1.small instance type and
the Tiny instance type in exoscale. You could add more mappings.
You are now ready to run ec2stack. This setup process will run the application
on the foreground. Because it is a Flask application, you can deploy it as a service in several
ways. The Flask Documentation has good tips to do this.
For testing, you can run it in the foreground; EC2Stack will be listening for requests and be ready
to forward them to your CloudStack cloud:
$ ec2stack * Running on http://0.0.0.0:5000/ * Restarting with reloader
4.2. Using the AWS CLI with EC2Stack
Problem
With EC2Stack running, you want to use the AWS CLI to make calls to your CloudStack cloud.
Solution
Install the AWS CLI from PyPI and configure it. Register your API keys with EC2Stack and start making requests to your cloud.
Discussion
Now that you are running EC2Stack on your local machine, you can use the AWS CLI to make calls to it. Install the CLI with:
$ sudo pip install awscli
Warning
The Python AWS CLI available on PyPI may change often, which can cause EC2Stack to break.
You can install a specific awscli package with:
$ sudo pip install awscli==1.3.10
Currently, EC2Stack stack has been tested with 1.3.10.
In addition, you need to register your API keys with the AWS CLI. If you have not used exoscale (as in the preceding example), then choose the keys of your own CloudStack deployment and use the appropriate region name:
$ aws configure AWS Access Key ID [None]: PQogHs2sk_3uslfvrASjQFDlZbt0mEDd14iN AWS Secret Access Key [None]: aHuDB2ewpgxVuQlvD9P1o313BioI1W4v Default region name [None]: CH-GV2 Default output format [None]:
You can see these settings in the ~/.aws/config file.
Check
the AWS CLI reference for further
customization. The output format can be json, text, or table.
With your AWS CLI installed and configured, the final configuration step is to register a user with EC2Stack.
To be on the safe side, upgrade the requests module:
$ sudo pip install --upgrade requests
Register your API keys like so:
$ ec2stack-register http://localhost:5000 <API accesskey> <API secret key>
The command should return a Successfully Registered! message. At this stage, you are
now ready to use the AWS CLI (or Boto) and send requests to the EC2Stack endpoint:
$ aws ec2 describe-images --endpoint=http://localhost:5000 $ aws ec2 describe-key-pairs --endpoint=http://localhost:5000 $ aws ec2 create-key-pair --endpoint=http://localhost:5000 --key-name=test
To start an instance, for example:
$ aws ec2 run-instances --image-id=20d4ebc3-8898-431c-939e-adbcf203acec --endpoint=http://localhost:5000
The image-id parameter is the CloudStack uuid corresponding to the
template that you want to start. You find it by running the aws
describe-images call.
4.3. Improving the EC2Stack API Coverage
Solution
Because EC2Stack is open source, you can easily contribute to it and add the API that you need. Fork the project on GitHub and submit a pull request.
Discussion
Looking at the code, only the following AWS APIs are covered (more are being added through a Google Summer of Code 2014 project):
def_get_action(action):actions={'AttachVolume':volumes.attach_volume,'AuthorizeSecurityGroupEgress':security_groups.authenticate_security_group_egress,'AuthorizeSecurityGroupIngress':security_groups.authenticate_security_group_ingress,'CreateKeyPair':keypairs.create_keypair,'CreateSecurityGroup':security_groups.create_security_group,'CreateTags':tags.create_tags,'CreateVolume':volumes.create_volume,'DeleteKeyPair':keypairs.delete_keypair,'DeleteSecurityGroup':security_groups.delete_security_group,'DeleteTags':tags.delete_tags,'DeleteVolume':volumes.delete_volume,'DescribeAvailabilityZones':zones.describe_zones,'DescribeImageAttribute':images.describe_image_attribute,'DescribeImages':images.describe_images,'DescribeInstanceAttribute':instances.describe_instance_attribute,'DescribeInstances':instances.describe_instances,'DescribeKeyPairs':keypairs.describe_keypairs,'DescribeSecurityGroups':security_groups.describe_security_groups,'DescribeTags':tags.describe_tags,'DescribeVolumes':volumes.describe_volumes,'DetachVolume':volumes.detach_volume,'GetPasswordData':passwords.get_password_data,'ImportKeyPair':keypairs.import_keypair,'RebootInstances':instances.reboot_instance,'RegisterSecretKey':register_secret_key,'RemoveSecretKey':remove_secret_key,'RevokeSecurityGroupEgress':security_groups.revoke_security_group_egress,'RevokeSecurityGroupIngress':security_groups.revoke_security_group_ingress,'RunInstances':instances.run_instance,'StartInstances':instances.start_instance,'StopInstances':instances.stop_instance,'TerminateInstances':instances.terminate_instance,}
Warning
Currently, EC2Stack is geared toward CloudStack basic zones. It is aimed at clouds that resemble AWS EC2 and want to allow access via SSH key pairs and security groups. Virtual private clouds (VPCs) are not currently supported.
The code is quite clean and it will be easy to add more actions and provide a much better coverage really soon. Pull requests are welcome if you are interested to contribute.
4.4. Using Python Boto with EC2Stack
There are many tools available to interface with an AWS-compatible API. In the previous recipe, we saw how to use the AWS CLI, but now let’s briefly look at Boto. Boto is a Python package that provides client-side bindings to work with the AWS API. It interfaces with EC2 but also with S3, CF, EMR, and so on. Boto has extensive documentation for each AWS service it supports.
Problem
You are familiar with Boto and want to use it with your CloudStack cloud.
Solution
Install Boto from the Python Package Index, and install and run an AWS EC2 interface to CloudStack. Write a Python script that imports the Boto module and creates a connection object to your EC2 interface endpoint.
Discussion
Installation is as easy as:
$ sudo pip install boto
With Boto installed on your client machine and an AWS EC2 interface running in front of CloudStack (see Recipe 4.1), you can now use the following script to start instances. Just replace the access and secret keys with your own and update the endpoint:
#!/usr/bin/env pythonimportsysimportosimportbotoimportboto.ec2region=boto.ec2.regioninfo.RegionInfo(endpoint="localhost")apikey='GwNnpUPrO6KgIdZu01z_ZhhZnKjtSdRwuYd4DvpzvFpyxGMvrzno2q05MB0ViBoFYtdqKd'secretkey='t4eXLEYWw7chBhDlaKf38adCMSHx_wlds6JfSx3z9fSpSOm0AbP9Moj0oGIzy2LSC8iw'defmain():'''Establish connection to EC2 cloud'''conn=boto.connect_ec2(aws_access_key_id=apikey,aws_secret_access_key=secretkey,is_secure=False,region=region,port=5000,path="/",api_version="2014-02-01")'''Get list of images that I own'''images=conn.get_all_images()myimage=images[0]'''Pick an instance type'''vm_type='m1.small'reservation=myimage.run(instance_type=vm_type,security_groups=['default'])if__name__=='__main__':main()
Warning
With Boto, you can also interact with other AWS services like S3. CloudStack has an S3 tech preview but it is backed by a standard NFS server and therefore is not a true scalable distributed block store. It will be removed from the code in an upcoming release. To provide an S3 service in your cloud, I recommend using other software like RiakCS, Ceph radosgw, or Glusterfs S3 interface. These systems handle large-scale chunking and replication.
This script will start an instance of type m1.small in a zone with security groups enabled.
You could pass additional parameters to the run method like a key pair.
If you are like me, you might like to have an interactive shell to your clouds, which means you might want to use Boto in a slightly different way. I use IPython to
get an interactive shell, with tab completion, history, and logging capabilities. My shell
script is:
#!/usr/bin/env pythonimportbotoimportboto.ec2fromIPython.terminal.embedimportInteractiveShellEmbedaccesskey="my api key"secretkey="my secret key"region=boto.ec2.regioninfo.RegionInfo(endpoint="localhost")conn=boto.connect_ec2(aws_access_key_id=accesskey,aws_secret_access_key=secretkey,is_secure=False,region=region,port=5000,path="/",api_version="2014-02-01")ipshell=InteractiveShellEmbed(banner1="Hello, Cloud Shell!")ipshell()
Starting this interactive shell, you can discover all the methods available in the connection object by entering conn. and pressing the Tab key.
The AWS interfaces to CloudStack do not yet have 100% fidelity with the AWS API, so keep in mind that not all Boto methods will work.
Note
In this example, I used the EC2Stack interface, but you could also use the interface
that comes natively bundled with CloudStack. Using the packages, it can be started
with service cloudstack-awsapi start. I personally prefer EC2Stack because
I was involved in the development and it supports a newer api_version.
4.5. Installing Eutester to Test the AWS Compatibility of Your CloudStack Cloud
Eutester was created by the folks at Eucalyptus to provide a framework to create functional tests for AWS zones and Eucalyptus-based clouds. What is interesting with Eutester is that it could be used to compare the AWS compatibility of multiple clouds. Therefore, you might be wonder, “Can we use Eutester with CloudStack?” And the answer is Yes. Certainly it could use more work, but the basic functionality is there. It allows you to write test scenarios and compare the results between an AWS EC2 availability zone and a CloudStack cloud.
Problem
You want to install Eutester to write integration tests for your AWS EC2 compliant cloud endpoints.
Solution
Grab the binary from the Python package index with pip or build
it from source from GitHub.
Discussion
Install eutester with:
$ sudo pip install eutester
Warning
The master branch of eutester may still cause problems to list images from a CloudStack
cloud. I recently patched
a fork of the testing branch and opened an issue on
their GitHub page. You might want to check its status if you want to use Eutester heavily.
To use eutester with CloudStack, clone the testing branch of my Eutester fork.
Then install it by hand:
$ git clone -b testing https://github.com/runseb/eutester.git $ cd eutester $ sudo python ./setup.py install
4.6. Using Eutester with EC2Stack to Write Functional tests
Problem
You have installed Eutester and want to write a Python script to issue requests to your CloudStack cloud.
Solution
Import the eucaops module in a Python script, and create a connection object using your endpoint information
and your API keys. To explore your cloud interactively and create testing scenarios, use IPython.
Discussion
Start a Python/IPython interactive shell or write a script that will import ec2ops
and create a connection object to your AWS EC2 compatible endpoint. For example, using EC2Stack from Recipe 4.1:
#!/usr/bin/env pythonfromeucaopsimportec2opsfromIPython.terminal.embedimportInteractiveShellEmbedaccesskey="my api key"secretkey="my secret key"conn.ec2ops.EC2ops(endpoint="localhost",aws_access_key_id=apikey,aws_secret_access_key=secretkey,is_secure=False,port=5000,path="/",APIVersion="2014-02-01")ipshell=InteractiveShellEmbed(banner1="Hello, Cloud Shell!")ipshell()
Eutester, at the time of this writing, has 145 methods. Only the methods available
through the CloudStack AWS EC2 interface that you will be using will be available. For example,
get_zones and get_instances would return:
In [3]: conn.get_zones()
Out[3]: [u'ch-gva-2']
In [4]: conn.get_instances()
[2014-05-21 05:39:45,094] [EUTESTER] [DEBUG]:
--->(ec2ops.py:3164)Starting method: get_instances(self, state=None,
idstring=None, reservation=None, rootdevtype=None, zone=None,
key=None, pubip=None, privip=None, ramdisk=None, kernel=None,
image_id=None, filters=None)
Out[4]:
[Instance:5a426582-3aa3-49e0-be3f-d2f9f1591f1f,
Instance:95ee8534-b171-4f79-9e23-be48bf1a5af6,
Instance:f18275f1-222b-455d-b352-3e7b2d3ffe9d,
Instance:0ea66049-9399-4763-8d2f-b96e9228e413,
Instance:7b2f63d6-66ce-4e1b-a481-e5f347f7e559,
Instance:46d01dfd-dc81-4459-a4a8-885f05a87d07,
Instance:7158726e-e76c-4cd4-8207-1ed50cc4d77a,
Instance:14a0ce40-0ec7-4cf0-b908-0434271369f6]This example shows that I am running eight instances at the moment in a zone called ch-gva-2,
one zone of the exoscale cloud. Selecting one of these instance objects will give you
access to all the methods available for instances. You could also list, delete, and create key pairs; list, delete, and create security groups; and so on.
Note
Eutester is meant for building integration tests and easily creating test scenarios. If you are looking for a client to build an application with, use Boto from Recipe 4.4.
4.7. Installing and Configuring gstack: The CloudStack GCE Interface
Google Compute Engine (GCE) is the Google public cloud. In December 2013, Google announced the General Availability (GA) of GCE. With AWS and Microsoft Azure, it is one of the three leading public
clouds in the market. CloudStack has a GCE compatible interface that lets users use the GCE clients (i.e., gcloud and gcutil) to access their CloudStack cloud.
Like EC2Stack, gstack is a Python Flask application that provides a REST API compatible with the GCE API and forwards the requests to the corresponding CloudStack
API. The source is available on GitHub and the binary is downloadable via PyPI.
Problem
You want to install gstack on your machine.
Solution
Grab the binary from the Python package index with pip or clone
the source code from GitHub.
Discussion
You can grab the gstack binary package from PyPI using pip in one single command:
$ sudo pip install gstack
Or, if you plan to explore the source and work on it, you can Clone the repository and install it by hand:
$ git clone https://github.com/NOPping/gstack.git $ sudo python ./setup.py install
Both of these installation methods will install a gstack and a gstack-configure binary in your path.
Before running gstack you must configure it. To do so, run the following:
$ gstack-configure
And enter your configuration information when prompted. You will need to specify
the host and port where you want gstack to run on, as well as the CloudStack endpoint
that you want gstack to forward the requests to. In the following example, we use the
exoscale cloud:
$ gstack-configure gstack bind address [0.0.0.0]: localhost gstack bind port [5000]: Cloudstack host [localhost]: api.exoscale.ch Cloudstack port [8080]: 443 Cloudstack protocol [http]: https Cloudstack path [/client/api]: /compute
The information will be stored in a configuration file available at ~/.gstack/gstack.conf:
$ cat ~/.gstack/gstack.conf PATH = 'compute/v1/projects/' GSTACK_BIND_ADDRESS = 'localhost' GSTACK_PORT = '5000' CLOUDSTACK_HOST = 'api.exoscale.ch' CLOUDSTACK_PORT = '443' CLOUDSTACK_PROTOCOL = 'https' CLOUDSTACK_PATH = '/compute'
You can start gstack as easily as this:
$ gstack
Tip
Like EC2Stack, this will run gstack in the foreground. This is acceptable for testing purposes
but if you want to run gstack as a service in production setup, look at some of the WSGI HTTP servers
that can be used to serve gstack. In production, you will also need to create a properly signed certificate for gstack and replace the self-signed certificate.
4.8. Using gstack with the gcutil Tool
Problem
With gstack installed and running on your machine, you want to use the gcutil command-line tool to issue
requests to your CloudStack cloud.
Solution
Install and configure the standalone gcutil tool and start issuing commands to CloudStack via your running gstack server.
Discussion
The current version of gstack only works with the standalone version of gcutil.
Warning
Do not use the version of gcutil bundled in the Google Cloud SDK. Instead, install the 0.14.2 version of gcutil.
gstack comes with a self-signed certificate for the local endpoint
gstack/data/server.crt, copy the certificate to the gcutil
certificates file gcutil/lib/httplib2/httplib2/cacerts.txt.
At this stage, your CloudStack API key and secret key need to be entered in
the gcutil auth_helper.py file in the
gcutil/lib/google_compute_engine/gcutil/ directory.
This is far from ideal. Thus, we opened a feature request with Google to
pass the client_id and client_secret as options to gcutil. Hopefully a
future release of gcutil will allow us to do so.
Create a cached parameters file for gcutil. Assuming you are running gstack on your local machine, use the defaults that were
suggested during the configuration phase. Modify ~/.gcutil_params with the following:
--auth_local_webserver --auth_host_port=9999 --dump_request_response --authorization_uri_base=https://localhost:5000/oauth2 --ssh_user=root --fetch_discovery --auth_host_name=localhost --api_host=https://localhost:5000/
Warning
Make sure to set the --auth_host_name variable to the same value as GSTACK_BIND_ADDRESS in your ~/.gstack/gstack.conf file.
Otherwise you will see certificates errors.
With this setup complete, gcutil will issue requests to the local Flask application, get an OAuth
token, issue requests to your CloudStack endpoint, and return the response in a GCE compatible format.
With the setup steps complete, you can start issuing standard gcutil commands. For illustration
purposes, we use exoscale.
Note
Because there are several semantic differences, you will notice that as a project, we use the account information from CloudStack. Hence, we pass our email address as the project value. This is another area that could be improved.
Let’s start by listing the availability zones:
$ gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com listzones
+----------+--------+------------------+
| name | status | next-maintenance |
+----------+--------+------------------+
| ch-gva-2 | UP | None scheduled |
+----------+--------+------------------+Let’s list the machine types (or, in CloudStack terminology, the compute service offerings and the list of available images):
$ gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com listmachinetypes
+-------------+----------+------+-----------+-------------+
| name | zone | cpus | memory-mb | deprecation |
+-------------+----------+------+-----------+-------------+
| Micro | ch-gva-2 | 1 | 512 | |
+-------------+----------+------+-----------+-------------+
| Tiny | ch-gva-2 | 1 | 1024 | |
+-------------+----------+------+-----------+-------------+
| Small | ch-gva-2 | 2 | 2048 | |
+-------------+----------+------+-----------+-------------+
| Medium | ch-gva-2 | 2 | 4096 | |
+-------------+----------+------+-----------+-------------+
| Large | ch-gva-2 | 4 | 8182 | |
+-------------+----------+------+-----------+-------------+
| Extra-large | ch-gva-2 | 4 | 16384 | |
+-------------+----------+------+-----------+-------------+
| Huge | ch-gva-2 | 8 | 32184 | |
+-------------+----------+------+-----------+-------------+$ ./gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com listimages
+---------------------------------+-------------+--------+
| name | deprecation | status |
+---------------------------------+-------------+--------+
| CentOS 5.5(64-bit) no GUI (KVM) | | Ready |
| Linux CentOS 6.4 64-bit | | Ready |
| Linux CentOS 6.4 64-bit | | Ready |
| Linux CentOS 6.4 64-bit | | Ready |
| Linux CentOS 6.4 64-bit | | Ready |
| Linux CentOS 6.4 64-bit | | Ready |
| Linux Ubuntu 12.04 LTS 64-bit | | Ready |
| Linux Ubuntu 12.04 LTS 64-bit | | Ready |
| Linux Ubuntu 12.04 LTS 64-bit | | Ready |
| Linux Ubuntu 12.04 LTS 64-bit | | Ready |
| Linux Ubuntu 12.04 LTS 64-bit | | Ready |
| Linux Ubuntu 13.04 64-bit | | Ready |
| Linux Ubuntu 13.04 64-bit | | Ready |
| Linux Ubuntu 13.04 64-bit | | Ready |
| Linux Ubuntu 13.04 64-bit | | Ready |
| Linux Ubuntu 13.04 64-bit | | Ready |
| Windows Server 2008 R2 SP1 | | Ready |
| Windows Server 2008 R2 SP1 | | Ready |
| Windows Server 2008 R2 SP1 | | Ready |
| Windows Server 2008 R2 SP1 | | Ready |
| Windows Server 2012 | | Ready |
| Windows Server 2012 | | Ready |
| Windows Server 2012 | | Ready |
| Windows Server 2012 | | Ready |
+---------------------------------+-------------+--------+You can also list firewalls, which gstack maps to CloudStack security groups. To create a security group, use the firewall commands:
$ ./gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com addfirewall ssh --allowed=tcp:22And get the details of this firewall with getfirewall:
$ ./gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com getfirewall ssh
+---------------+-----------+
| property | value |
+---------------+-----------+
| name | ssh |
| description | |
| creation-time | |
| network | |
| source-ips | 0.0.0.0/0 |
| source-tags | |
| target-tags | |
| allowed | tcp: 22 |
+---------------+-----------+To start an instance, you can follow the interactive prompt given by gcutil.
You will need to pass the --permit_root_ssh flag, another one of those semantic and access configuration details that needs to be ironed out.
The interactive prompt will let you choose the machine type and the image that you want; it will then start the instance:
$ ./gcutil --cached_flags_file=~/.gcutil_params
--project=runseb@gmail.com addinstance foobar
Selecting the only available zone: CH-GV2
1: Extra-large Extra-large 16384mb 4cpu
2: Huge Huge 32184mb 8cpu
3: Large Large 8192mb 4cpu
4: Medium Medium 4096mb 2cpu
5: Micro Micro 512mb 1cpu
6: Small Small 2048mb 2cpu
7: Tiny Tiny 1024mb 1cpu
7
1: CentOS 5.5(64-bit) no GUI (KVM)
2: Linux CentOS 6.4 64-bit
3: Linux CentOS 6.4 64-bit
4: Linux CentOS 6.4 64-bit
5: Linux CentOS 6.4 64-bit
6: Linux CentOS 6.4 64-bit
<...snip>
INFO: Waiting for insert of instance . Sleeping for 3s.
INFO: Waiting for insert of instance . Sleeping for 3s.
Table of resources:
+--------+--------------+--------------+----------+---------+
| name | network-ip | external-ip | zone | status |
+--------+--------------+--------------+----------+---------+
| foobar | 185.1.2.3 | 185.1.2.3 | ch-gva-2 | RUNNING |
+--------+--------------+--------------+----------+---------+
Table of operations:
+--------------+--------+--------------------------+----------------+
| name | status | insert-time | operation-type |
+--------------+--------+--------------------------+----------------+
| e4180d83-31d0| DONE | 2014-06-09T10:31:35+0200 | insert |
+--------------+--------+--------------------------+----------------+You can, of course, list (with listinstances) and delete instances:
$ ./gcutil --cached_flags_file=~/.gcutil_params --project=runseb@gmail.com
deleteinstance foobar
Delete instance foobar? [y/n]
y
WARNING: Consider passing '--zone=CH-GV2' to avoid the unnecessary
zone lookup which requires extra API calls.
INFO: Waiting for delete of instance . Sleeping for 3s.
+--------------+--------+--------------------------+----------------+
| name | status | insert-time | operation-type |
+--------------+--------+--------------------------+----------------+
| d421168c-4acd| DONE | 2014-06-09T10:34:53+0200 | delete |
+--------------+--------+--------------------------+----------------+gstack is still a work in progress, but it is now compatible with the GCE GA v1.0 API. The few differences in API semantics need to be investigated further and additional API calls need to be supported. However, it provides a solid base to start working on hybrid solutions between a GCE public cloud and a CloudStack-based private cloud.
4.9. Supporting the OCCI Standard in CloudStack
The Open Cloud Computing Interface (OCCI) is a standard from the Open Grid Forum (OGF). OCCI was originally created to be a remote management API for the IaaS layer of cloud computing but has since evolved to also address the PaaS and SaaS layers. With CIMI, it is one of the two cloud standards for cloud providers’ APIs backed by a standards organization. As we mentioned several times already, CloudStack has its own API, which is not a standard. AWS EC2 and Google GCE are not standards either. Cloud wrappers like libcloud and jclouds work well as unofficial standards that abstract differences in cloud providers’ APIs. Users interested in using OCCI or CIMI will need a wrapper on top of the CloudStack API that will provide a CIMI or OCCI implementation. There is currently no CIMI interface for CloudStack, but through rOCCI, there is an OCCI interface. The rest of this recipe goes through installation, configuration, and usage of the rOCCI CloudStack driver.
Note
There are several implementations of OCCI. rOCCI is one of them and is currently going through some refactoring.
Problem
The CloudStack API is very nice, but it is not backed by a standards organization. You care about standards and you would like to use OCCI from the Open Grid Forum to interact with your CloudStack cloud, removing any potential issues with a nonstandard API.
Solution
Install a rOCCI server in your infrastructure and configure to use the CloudStack driver. You can then use any OCCI client to issue cloud requests to it. rOCCI will forward the requests to the CloudStack API server and send the appropriate response back.
Discussion
Using OCCI with CloudStack involves running the rOCCI server and using an OCCI client to interface to it. The rOCCI server provides the API mapping between the OCCI standard API and the CloudStack API. In this discussion, the rOCCI client is used to issue OCCI API requests instead of using a CloudStack-specific client like CloudMonkey.
Install and run the rOCCI server
As we have done several times now, you can install the rOCCI server by cloning the project on GitHub, doing a build using Ruby’s bundler, and setting up some configuration files:
$ git clone https://github.com/isaacchiang/rOCCI-server.git $ gem install bundler $ bundle install $ cd etc/backend $ cp cloudstack/cloudstack.json default.json
Note
The rOCCI CloudStack backend is experimental and not merged in the rOCCI server project yet.
Edit the default.json file to contain the information about your CloudStack cloud, your endpoint, and your API keys. Start the rOCCI server in one shell:
$ bundle exec passenger start
The server should be running on http://0.0.0.0:3000, and you can try to run the basic tests:
$ bundle exec rspec
You are ready to install an OCCI client and use it to talk to the rOCCI server you just started. The requests will be forwarded to your CloudStack endpoint.
Install the rOCCI client
Clone the rOCCI-cli client from GitHub. Use bundler again to build it and rake to install it:
$ git clone https://github.com/gwdg/rOCCI-cli.git $ cd rOCCI-cli $ bundle install $ bundle exec rake test $ rake install
An occi binary should now be in your path. Try to use it by running the --help option:
$ occi --help
Test the OCCI client against the server
With your cloud endpoint and your API keys properly defined earlier in the json configuration file, you can use the OCCI client to list templates, locations, and start an instance. Similar to EC2Stack, you need to specify the endpoint of the rOCCI server that’s running. Try a couple OCCI client commands:
$ occi --endpoint http://0.0.0.0:3000/ --action list --resource os_tpl
Os_tpl locations:
os_tpl#6673855d-ce9b-4997-8613-6830de037a8f
$ occi --endpoint http://0.0.0.0:3000/ --action list --resource resource_tpl
Resource_tpl locations:
resource_tpl##08ba0343-bd39-4bf0-9aab-4953694ae2b4
resource_tpl##f78769bd-95ea-4139-ad9b-9dfc1c5cb673
resource_tpl##0fd364a9-7e33-4375-9e10-bb861f7c6ee7You will recognize the uuid from the templates and service offerings
that you have created in CloudStack. These uuid will be different. To start an instance:
$ occi --endpoint http://0.0.0.0:3000/
--action create
--resource compute
--mixin os_tpl#6673855d-ce9b-4997-8613-6830de037a8f
--mixin resource_tpl#08ba0343-bd39-4bf0-9aab-4953694ae2b4
--resource-title foobarAnd voilà! The holy grail again, you started an instance on a CloudStack cloud using an OCCI client and an OCCI implementation on top of the CloudStack API.
Get 60 Recipes for Apache CloudStack now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
