Getting Started with Fluidinfo

Chapter 4. Programming with Fluidinfo

Client Libraries

The easiest way to make use of Fluidinfo in your own programming projects is to use an existing client library. There are many open source libraries available in many different languages. A comprehensive list can be found on Fluidinfo’s developer page. Such client libraries can be categorized in two ways:

• Asynchronous versus synchronous clients^[22]

• Thin layers just above the HTTP API versus programmer-friendly abstractions (for example, an object-oriented mapping to Fluidinfo)

This chapter and the two that follow introduce three libraries that illustrate the categories described above:

fluidinfo.py

A blocking client that provides a bare-minimum client library for Fluidinfo’s HTTP API (presented in this chapter)

FOM (the Fluid Object Mapper)

A blocking, object-oriented abstraction for using Fluidinfo (presented in Chapter 5)

fluidinfo.js

An asynchronous library that provides both low-level interactions with the API and high-level functions that abstract common tasks (presented in Chapter 6)

Although there is a danger that programs built upon synchronous libraries can become unresponsive while they wait for responses from Fluidinfo,^[23] it does mean that the code written with these libraries is easy to read and understand.

Alternatively, if you’d like to use the API directly, you should refer to Chapter 7, which explains and illustrates the raw HTTP-based API in detail.

Introducing fluidinfo.py

The fluidinfo.py library is a very thin wrapper around Fluidinfo’s HTTP-based API. It provides just enough functionality that you don’t have to worry about how your interaction with Fluidinfo happens. You only need to understand the API and make obvious calls to it via the fluidinfo module.

fluidinfo.py requires Python 2.6 (or later) to work, so we assume you have followed the instructions at the Python website to install an appropriate version on your system. With this requirement met, there are two platform-independent ways to install the library:

• Use Python’s own installation tools: easy_install or pip.

• Install from the source code.

The first option assumes you have Python’s setuptools or pip installed. With one of these, you should be able to install fluidinfo.py by typing commands similar to the following into your terminal:

$ easy_install fluidinfo.py
$ pip install -U fluidinfo.py

If all goes well, some messages will scroll down the screen that end with an indication that the package is installed.

Alternatively, download the source code from the project’s website and use the setup.py installation script.

The fluidinfo.py source code is freely available and there’s an obvious link to download the code as a compressed file (which you should uncompress to somewhere on your file system). Alternatively, use the Git source control tool to track the project repository on your local file system.

In either case you’ll end up with a directory containing the source code. Change into the directory and install fluidinfo.py by typing the following command into your shell:

$ python setup.py install

(You may need some sort of administrative privileges for this to work.)

To check everything works, simply start a Python session and type in the following:

$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import fluidinfo
>>> exit()

If you get an “ImportError” message, it’s likely that the library isn’t in your Python path. Check the Python documentation for more information about how to set the path correctly.

Fluidinfo.py Fundamentals

The primary goals of fluidinfo.py are simplicity, ease of use, and the principle of least surprise. Ultimately, if you know the HTTP API, it should be obvious what to do with this library. The following example from an interactive Python session illustrates this:

>>> import fluidinfo 
>>> fluidinfo.login('username', 'password') 
>>> fluidinfo.get('/users/alice') 
({  'status': '200',
    'content-length': '71',
    'content-location': 'https://fluiddb.fluidinfo.com/users/alice',
    'server': 'nginx/0.7.65',
    'connection': 'keep-alive',
    'cache-control': 'no-cache',
    'date': 'Thu, 14 Jul 2011 12:36:29 GMT',
    'content-type': 'application/json'},
 {  u'name': u'Alice Liddell',
    u'id': u'1494b708-0665-4608-9366-e3c161ef313c'}
)

Import the fluidinfo library into the Python session.

Log in, passing in your username as the first argument followed by your password. It’s a good idea never to hardcode a password into code.

Make an HTTP GET request to the /users/alice endpoint.

The example shows how to get information about the user alice. The result consists of two dictionaries—the HTTP response headers and the JSON payload returned by Fluidinfo. Notice how fluidinfo.py takes care of authentication and silently translates the raw JSON response from Fluidinfo into a native Python dictionary.

There are functions for each of the HTTP methods^[24] used with the HTTP API. For example, creating a new object involves sending an HTTP POST request to the /objects endpoint with a JSON specification of the about value. As the following example shows, simply call the post function with the appropriate arguments (a single JSON payload describing the new object):

>>> payload = {'about': 'wonderland'}
>>> headers, response = fluidinfo.post('/objects', payload)
>>> headers
{'status': '201',
 'content-length': '131',
 'server': 'nginx/0.7.65',
 'connection': 'keep-alive',
 'location': 'https://fluiddb.fluidinfo.com/objects/1f296e02-6e5a-41e0-acb8-145d8063c549',
 'cache-control': 'no-cache',
 'date': 'Wed, 13 Jul 2011 14:08:13 GMT',
 'content-type': 'application/json'}
>>> response
{u'id': u'1f296e02-6e5a-41e0-acb8-145d8063c549',
 u'URI': u'https://fluiddb.fluidinfo.com/objects/1f296e02-6e5a-41e0-acb8-145d8063c549'}

Notice how fluidinfo.py has silently translated the native Python dictionary (containing the definition of what the new object is about) into the JSON payload. Furthermore, the Python tuple returned by fluidinfo.py is assigned to two variables representing the headers and response.

Tagging an object requires a single HTTP PUT request and retrieving a tag value is a single GET request:

>>> headers, response = fluidinfo.put('/about/wonderland/alice/rating', 11) 
>>> headers, response = fluidinfo.get('/about/wonderland/alice/rating') 
>>> response
11

Add/update the integer value 11 to the object about “wonderland” via the alice/rating tag.

Return the value of the tag.

The simple pattern for creating a request with fluidinfo.py is to call a function named after the HTTP method, with the path to the desired endpoint and (optional) value to upload as the two arguments:

        fluidinfo.METHOD('/PATH/TO/ENDPOINT', VALUE)

Some calls, following Fluidinfo’s HTTP API, require additional arguments. For example, when retrieving values from objects that match a particular query, you need to supply both the list of tags whose values you want to retrieve and the query itself (the equivalent of a SELECT/WHERE statement in SQL):

>>> tag_list = ['fluiddb/about', 'alice/rating', 'oreilly.com/title']
>>> query_to_match = 'oreilly.com/title matches "Python"'
>>> headers, result = fluidinfo.get('/values', tags=tag_list,
...    query=query_to_match)

At a low level there needs to be an HTTP GET call to the /values endpoint. However, the HTTP API needs appropriately URL-encoded arguments for the tag list and query. Happily, fluidinfo.py does the heavy lifting and makes the appropriate translation from the arguments passed into the Python function to those needed by the Fluidinfo API.

Another similar requirement is to append arguments to an endpoint to tell Fluidinfo how much information you want returned. For example, when getting information about a namespace, you can optionally ask for its description, child namespaces, and tags. The Fluidinfo API states that these should be passed as arguments in the URL, so fluidinfo.py helps by turning any arguments you pass (in addition to the endpoint and value) into URL arguments:

        >>> fluidinfo.get('/namespaces/boingboing.net',
        ... returnDescription=True, returnNamespaces=True, returnTags=True)

The result is a GET request to the following URL:

https://fluiddb.fluidinfo.com/namespaces/boingboing.net?returnDescription=True&
returnNamespaces=True&returnTags=True

Common Tasks Using fluidinfo.py

The fluidinfo.py library makes it very easy to explore Fluidinfo’s API in an interactive Python session. What follows is a list of basic tasks users most often want to do with the Fluidinfo API along with examples of how to make them work using fluidinfo.py. A growing list of such “recipes” can be found on Fluidinfo’s cookbook page.

Create a New Object

There are two ways to create a new object:

1. HTTP POST to the /objects endpoint to create objects that may or may not have an associated about value.

2. HTTP POST to the /about endpoint.

The following session demonstrates how to use the /about endpoint:

>>> headers, response = fluidinfo.post('/about/terry jones')
>>> headers
{'cache-control': 'no-cache',
 'connection': 'keep-alive',
 'content-length': '131',
 'content-type': 'application/json',
 'date': 'Tue, 29 Aug 2011 11:52:16 GMT',
 'location': 'https://fluiddb.fluidinfo.com/about/terry%20jones',
 'server': 'nginx/0.7.65',
 'status': '201'}
>>> response
{u'URI': u'https://fluiddb.fluidinfo.com/about/terry%20jones',
 u'id': u'79a9a9ed-15cd-4357-a434-f9b322ff8074'}

Notice how Fluidinfo responds with the canonical URI for the object along with its unique ID. The 201 status in the headers indicates that the operation succeeded. If there is already an object about "terry jones", the call returns details of the existing object (in other words, no new object is created).

Tag an Object with a Value

As with the creation of objects, it is possible to use both the /objects and /about endpoints for manipulating values tagged on objects. The following examples use the /about endpoint, because using an “about” value makes them easier to understand.

Two types of value can be stored in Fluidinfo. Primitive values can be used in search queries and include numbers, Boolean values, strings, and lists of strings. Opaque values include any other type of data with an associated MIME type and cannot be used in a search query.

Note

A MIME type (Multipurpose Internet Mail Extension) is a two-part identifier for file formats. Originally developed to allow non-ASCII multimedia content in mail messages, its most common use now is perhaps as the value that’s passed via the content-type header in HTTP. Its purpose is to indicate the type of data being transmitted. For example, the MIME type for a PNG image is image/png (notice the two-part type/subtype distinction). In the context of Fluidinfo, you must indicate a MIME type for all opaque values. When the value is subsequently retrieved, Fluidinfo adds the appropriate content-type header to the response. (Fluidinfo uses the MIME type application/vnd.fluiddb.value+json to indicate simple tag values.)

Remember, to tag an object with a primitive value, simply pass the correct endpoint path (consisting of the object’s about value with the tag’s path appended to the end) and the actual Python value to the put function. The library takes care of the rest:

>>> headers, response = fluidinfo.put('/about/fluidinfo/alice/rating', 10)
>>> headers
{'cache-control': 'no-cache',
 'connection': 'keep-alive',
 'content-type': 'text/html',
 'date': 'Tue, 29 Aug 2011 20:13:25 GMT',
 'server': 'nginx/0.7.65',
 'status': '204'}

The 204 status code in the headers indicates the operation succeeded.

Tagging an opaque value works in exactly the same way but with the addition of a mime argument:

>>> headers, response = fluidinfo.put('/about/fluidinfo/alice/comment',
... '<html><body><h1>Fluidinfo</h1><p>An interesting project.</p></body></html>',
... mime='text/html')

Notice that we are declaring the text to have the MIME type text/html, rather than letting the library convert it to Fluidinfo’s primitive string type. To store a binary value, such as an image, you might do something like this:

>>> kittyImage = open('kitten.png')
>>> headers, response = fluidinfo.put('/about/fluidinfo/alice/kitten',
... kittyImage.read(), mime='image/png')

Get a Specific Value from an Object

To retrieve individual values from an object, simply send an HTTP GET request to the appropriate path:

>>> headers, response = fluidinfo.get('/about/fluidinfo/alice/rating')
>>> headers
{'cache-control': 'no-cache',
 'connection': 'keep-alive',
 'content-length': '2',
 'content-location': 'https://fluiddb.fluidinfo.com/about/fluidinfo/alice/rating',
 'content-type': 'application/vnd.fluiddb.value+json',
 'date': 'Tue, 29 Aug 2011 21:26:12 GMT',
 'server': 'nginx/0.7.65',
 'status': '200'}
 >>> response
 10

This example returns the value of the alice/rating tag on the object about fluidinfo.

Notice that the library has converted the response into a Python integer type. It knows to do this because the content-type header has the value application/vnd.fluiddb.value+json. This is the MIME type used by Fluidinfo to indicate that the payload of a response or request is a primitive type. If the value being returned was an opaque value, the content-header would reflect this. For example, if an image were being returned, the content-header would be something like image/png.

Delete a Specific Value from an Object

Deleting a specific value from an object requires that you make an HTTP DELETE request to an endpoint that references the object and affected tag:

>>> headers, response = fluidinfo.delete('/about/fluidinfo/alice/rating')
>>> headers
{'cache-control': 'no-cache',
 'connection': 'keep-alive',
 'content-type': 'text/html',
 'date': 'Tue, 29 Aug 2011 21:37:21 GMT',
 'server': 'nginx/0.7.65',
 'status': '204'}

This example deletes the value associated with the alice/rating tag from the object about fluidinfo.

The status header’s 204 value informs us that the tag/value was deleted. Fluidinfo doesn’t return anything in the response.

Query for Specified Values on Matching Objects

Queries are specified with the language described in the Appendix A. By specifying one or more tags it is possible to build a request that is the equivalent to the following SQL-like query:

SELECT tag1, tag2, tag3 FROM fluidinfo WHERE query=…

This is achieved with a GET request to a /values-based URL that has, in addition to the query argument, a list of the tags whose values should be returned:

>>> headers, response = fluidinfo.get('/values',
... tags=['alice/rating', 'alice/comment', 'alice/kitten'],
... query='has alice/rating')
>>> response
{u'results':
  {u'id':
    {u'0ff5ba6a-b901-41c6-ab76-516183379e41':
      {u'alice/rating': {u'value': 10}},
     u'2eecb420-9a1f-497b-9654-c07dd86fa926':
      {u'alice/rating': {u'value': 5},
       u'alice/comment': {u'value': u'It was ok'},
       u'alice/kitten': {u'size': 3952, u'value-type': u'image/png'}
      }
    }
  }
}

This example requests the values of the alice/rating, alice/comment, and alice/kitten tags on all objects that have been tagged with the alice/rating tag. The result is a Python dictionary created from the raw JSON that Fluidinfo responded with.

It is important to note the following about the result:

• Each matching object is referenced by its object ID.

• No value component is returned if the matching object doesn’t have a selected tag (see the first result, which has no alice/comment or alice/kitten values).

• If the value is a primitive type, it will be returned inline (see the results for the alice/rating and alice/comment tags).

• If the value is an opaque type, its size and MIME type will be returned (see the result for the alice/kitten tag). In order to get the actual value, you should GET the individual tag as described in Get a Specific Value from an Object.

Armed with these examples and a specification of the Fluidinfo HTTP API, it is a relatively simple exercise to build quite complex requests with the minimum of effort. However, such interactions can sometimes feel clumsy, because the RESTful nature of Fluidinfo’s API may not fit with the programming paradigm being used in your application. These problems are addressed by more abstract libraries such as FOM, which we’ll explore in the next chapter.

---

^[22] Asynchronous clients make a request to Fluidinfo, continue with what they’re doing, and associate a piece of code to handle the response when it arrives (these are usually known as callbacks). In contrast, synchronous clients are said to block: when they make a call to Fluidinfo the program waits for a response before continuing.

^[23] A problem that can easily be overcome through sensible use of threading or other programming mechanisms.

^[24] GET, POST, PUT, DELETE, and HEAD.