Fluidinfo is an online storage system in which there is a place for information about everything—everything that exists, everything that could exist, and everything that can be imagined. It allows anyone to store any information, about anything, in any digital form. And Fluidinfo makes it easy to find information in the system, and to extract it, using an unusually simple query language.
Fluidinfo is a first-class Internet citizen, exposing all its functionality through HTTP, the core protocol that underpins the World Wide Web. Programmers can take advantage of its RESTful API, which makes it easy to integrate with other applications.
Finally, Fluidinfo is social: users can exercise fine control over who can read their data and can even enable other chosen users and applications to write data on their behalf.
One of the key aspects of Web 2.0 was the increasing writability of the Web. Although its inventor, Tim Berners-Lee, always envisaged the Web as openly writable, in its early years there were many more readers than writers. The advent of blogging tools, social networks, user product reviews, and online data services have all contributed to a significant reduction in the asymmetry between Internet authors and consumers. Yet even today, the extent of the Web’s writability remains strictly limited.
The motivating concept behind Fluidinfo is to provide a shared data space where anyone can attach granular data of any kind to anything on the Web or anything in the world. Of course, physical objects like the Eiffel Tower don’t exist on the Web, or in Fluidinfo, so digital data can’t literally be attached to them. But the open structure of Fluidinfo and the emerging conventions for how to refer to anything—digital or physical, real or imaginary—offer a very real sense in which Fluidinfo has a nominated container ready to accept information about anything. Any Fluidinfo user can add data to any such container.
Fluidinfo seeks not only to provide a place for arbitrary data, and mechanisms for reading and writing it, but also to leave the creator of that data firmly in control of it. We are all familiar with web services that encourage us to create data inside them, and to import data to them, but which make it much harder to get that data out, still less to allow other applications and services to access it. Fluidinfo takes a different approach: it not only provides excellent mechanisms for extracting and exporting data, but it includes facilities designed to encourage users and applications to share, remix, and reuse data in ways that will often not have been anticipated when it was created. Fundamentally, Fluidinfo leaves the data creator (or owner) in full control, not only in terms of who can read and write that data, but also which applications can use, modify, and extend it.
Fluidinfo’s radical reimagining of the way data can be created, shared, and controlled online marks a significant departure, which we believe more fully realizes Tim Berners-Lee’s original vision of the Web as a read-write medium. This book is a practical guide to several ways to access and use Fluidinfo; we hope it will help you to embrace the writable Web.
Some systems gain their power through complexity, others through simplicity. Fluidinfo is among the latter. Five core concepts cover most of it:
Objects represent things (real or imaginary).
Tags attach information to objects.
Users use their own tags to attach data to shared objects.
The permissions system controls who can read and write each tag. (Objects do not have owners, so anyone can tag any object.)
Queries select objects by specifying properties of their tags and values. The selected objects can be read or tagged.
Fluidinfo is a collection of objects that function as shared data containers. Data is stored by attaching it to the relevant object using a tag—a kind of digital Post-it note.
Most objects are used to store data about something specific and the object is said to be about that thing. It is possible to tell what an object is about in two ways:
By checking the value of the special, globally unique about tag (
fluiddb/about) attached to the object.[2]From the context given by arbitrary tags attached to the object. The about tag is the only tag that the system guarantees to be unique and immutable. If you query for objects on the basis of other tags, you may see the tags, and therefore the objects retrieved, change over time.
In some sense Fluidinfo has an object for everything. This is reminiscent of Jorge Luis Borges’s 1941 story The Library of Babel, in which he describes a library containing all the books that could be written. Unfortunately the set of all books that could be written contains far more books full of lies, nonsense, and errors than it does accurate or useful books. The Library of Babel urgently needed a way to find the valuable material.
Most Fluidinfo users are well intentioned, so the quality of information is generally rather higher than in the Library of Babel. Fluidinfo also provides a mechanism to help judge information’s trustworthiness by starting every tag with its owner’s name. In this way, the system facilitates the establishment of networks of trust.
Tags attach information to objects.
Fluidinfo users can add data to any object by tagging it. A tag
has a name, and its mere presence on an object can carry meaning: Alice
might tag books with an alice/has-read tag to indicate that she has
read them, while Bert might tag things he wishes to acquire with a
bert/wants tag. Even the
about tag is owned by a user—the Fluidinfo
superuser, fluiddb.[3] This is why the full name for the
about tag is fluiddb/about.
Fluidinfo tags can also store data in the form of a
value, which can be a piece of text, a number, an
image, or any other digital information. For example, if Alice likes the
book Alice in Wonderland, she might add tags with
values such as alice/rating=10 or
alice/comment="I ♥ Lewis Carroll", as
well as the valueless alice/has-read,
to Fluidinfo’s Alice in Wonderland object (Figure 1-1).
Tags whose name starts with alice are owned and controlled by Alice
(username alice). By default, only
she can tag objects using her tags, while anyone can read them, but
Alice can choose who can read and write her tags on a tag-by-tag basis.
Users can assess the trustworthiness of information by examining its
provenance and who controls it, which is visible in every tag’s
name.
The alice at the start of all
Alice’s tags is an example of a namespace, which
groups tags together in much the same way that a folder groups files
together in a file system. Users can create further sub-namespaces if
they wish.
Anyone can read public data from Fluidinfo without even logging in, but in order to write data you need to authenticate with a valid username and password. Since all Fluidinfo tags are identifiable as being owned by a named user, the system is ideally suited to storing opinions and other personalized data. Because Fluidinfo users can also be applications or organizations (usually using their website’s address as a username), the system can also comfortably accommodate application data and institutional data. The permissions system can be used to enable groups of users, or even the entire user base, to have shared write access to tags, allowing data to be formed collaboratively, and in the most extreme cases, in a wiki-like manner.
Permissions control who can read and write each tag. They do not apply to objects, leaving Fluidinfo openly writable; in other words, any user can tag any object.
Fluidinfo permissions are expressed in two parts: a policy that is
either open or closed and a list of users who are exceptions
to the policy.
For example, the write permission on the
alice/rating tag has a closed policy with one exception—alice herself. As a result, only Alice can tag
objects with alice/rating, and only
she can remove such tags. In contrast, the read
permission for the same tag has an open policy with no exceptions, which means
that anyone can read Alice’s ratings.
Tag permissions can be set programmatically, or using tools such as Fish (Chapter 2) or libraries (see Chapters 4, 5, and 6).
Fluidinfo has a simple query language that makes it easy to find and retrieve information. Many queries look very similar to their description in “natural” English.
Here are some examples of some of the more common types of queries used within Fluidinfo.
- Tag presence
We can select all the objects that have a particular tag with a query like this:
has alice/rating
meaning
all the objects that have an
alice/ratingtag attached to them.- Comparisons
We can specify numeric values with ordinary equality and inequality tests:
alice/rating >= 7
meaning
all the objects that have an
alice/ratingof seven or more.- Text search
We can specify words that tag values should contain using the
matchesoperator:alice/comment matches "fantastic"
meaning
all the objects that have an
alice/commentcontaining the word “fantastic.”- Multiple conditions
We can combine any of the previous queries using
and,or, andexcept, adding parentheses as necessary:(alice/rating >=7 or alice/comment matches "fantastic") except has alice/owns
meaning
all the objects that have an
alice/ratingof seven or more or analice/commentthat contains “fantastic,” other than those with analice/ownstag.
Users can use any tags they can read in their queries.
A single Fluidinfo query can refer to tags from multiple users. For example, the following query matches all the objects for books that Alice rates highly (6 or more) but that Bert hasn’t read:
(has alice/books/title and alice/rating >=6) except has bert/has-read
(The first clause, has
alice/books/title is used here to narrow the set of objects to
books Alice knows about, as opposed to all the other things she might
rate.)
As ever more of the world is digitized, the task of managing our data becomes increasingly complex. The overlapping challenges we face include:
Finding a particular piece of information within our own data
Keeping data in sync across multiple devices and the Web
Extracting or integrating data we have stored using different applications and services that may offer little or no support for such export
Organizing and preserving our data
Annotating both our own data and information we may not own but can access
Giving other people and applications appropriate access to our data without losing control of that data
There are many partial solutions to these problems, most of which reduce complexity for the user in exchange for migrating all data to a single place, sometimes with some support for satellite copies. This usually comes at the price of reduced control and flexibility. There are lots of online services that will manage many classes of data for you, and offer good import mechanisms, but once you have exported your digital life to one of these, and perhaps used the tools to annotate and organize the data better, you may find it difficult to reclaim your data.
Fluidinfo grew out of a different vision of how data should be stored and managed. It is not a panacea—the issues involved are probably too complex for any complete technical “solution”—but it does represent a coherent reimagining of the information landscape that speaks to each of these issues with a distinctive voice.
An obvious reference point for Fluidinfo is Wikipedia. In a sense,
both Wikipedia and Fluidinfo have a (potential) place for information on
any topic. In Wikipedia’s case, the repository for a topic is a
Wikipedia article, identified by a URL fragment; in the case of
Fluidinfo, the repository is an object, and the
about tag, fluiddb/about, acts as the identifier.
The two most fundamental divergences between Wikipedia and Fluidinfo concern structure and point of view.
- Structure
A Wikipedia page is largely unstructured in the technical sense that the information it contains is mostly free-form text, often augmented with images, tables, and other elements designed to be read by a human being. In contrast, the Fluidinfo object is structured, typically consisting of a large number of named pieces of information, each having a type. Fluidinfo’s query language and API are also relevant differences, but are natural given the more structured nature of Fluidinfo data.
- Point of view
An explicitly articulated goal of Wikipedia is to achieve a “neutral point of view,” which is developed through processes of cooperation, collaboration, negotiation, mediation, and (in some cases) edit wars, moderation, and locking. Although a history of changes is maintained, the live Wikipedia page on a topic presents a single version of the truth.
In contrast, all data in Fluidinfo is clearly associated with an owner and different users can all attach their own data to the same shared item. Fluidinfo is very suitable for opinions and other personal data, in a way that Wikipedia is certainly not. Fluidinfo’s permissions system provides extra flexibility around data ownership and control.
Because of this key difference, there are no edit wars in Fluidinfo.
Another reference point for Fluidinfo comes from social bookmarking sites such as Delicious and Pinboard. These sites provide online storage for bookmarks to web pages, and allow users to organize their bookmarks by attaching any number of simple tags to them. In this context, a tag is just a word, and users normally attach a number of tags to each URL bookmarked to make finding pages on particular subjects easy.
The sense in which these sites are social is that, by default, a user’s bookmarks are publicly visible. An interesting consequence of this is that searching for pages that many users have tagged with particular terms tends to produce highly relevant results for that term; the degree of coherence of the emergent taxonomic structure (sometimes known as a tagsonomy) is remarkable.
Fluidinfo can be viewed as a social bookmarking site in which the bookmarks need not point to web pages but can represent anything (for example, books, songs, films, locations, stocks, Twitter users, people, cars, buildings, taxis, words, concepts, equations, truths, falsehoods, colors, or similes). Fluidinfo further differentiates itself by allowing its tags to have typed values,[4] by providing a query mechanism, by allowing more sophisticated and finer-grained permissions, and by providing a correspondingly more powerful API.
As we have seen, where Wikipedia is a collection of collaboratively produced encyclopedia articles, and Delicious is a collection of tagged and annotated URLs, Fluidinfo is a collection of shared objects, each of which represents something.
When an object is created in Fluidinfo, the creator can specify
any text as the value of the special tag fluiddb/about, which is normally referred to
simply as the about tag. If an object already
exists with the nominated about tag, rather than
creating a new one, the system reuses the existing object; if the
about tag doesn’t exist, a new object is created.
Once created, objects are never destroyed and about
tags are never changed. As a result, about tags act
as permanent, unique identifiers for objects, and they are the primary
way of specifying objects directly.
Fluidinfo itself attaches no significance to the value of the
about tag, so the choice of which object to use to
store data is left to users and is a matter of convention. In simple
cases, such as URLs, the choice is simple and natural—people invariably
use the full URL directly as the about tag (for
example, fluiddb/about="http://fluidinfo.com"), though
there is slight ambiguity around whether to include a trailing slash
(most people do not). In more complex cases, it is important to adopt
sensible conventions if you want your data to end up on the same objects
as other people’s. Such conventions, and tools for helping to construct
the conventional about tags, are discussed in more
detail in Chapter 9.
If you want to be able to write data in Fluidinfo, you will need to sign up for an account by visiting Fluidinfo’s sign-up page. There are two ways to do this. If you have a Twitter account and would like to use the same username in Fluidinfo, just click the “Sign in with Twitter” button and authorize Fluidinfo on Twitter’s site; a new Fluidinfo account will be created for you with your Twitter username (if available). If you don’t have a Twitter account, or would prefer not to link your accounts, simply supply your name, a unique username (which will identify all of your tags), and a working email address.[5]
Once you’ve agreed to the terms of use and proved yourself to be human by answering the CAPTCHA[6] Fluidinfo will email you a verification link.
After you have clicked the verification link, your account will be activated.
The next few chapters introduce three different ways of interacting with Fluidinfo. In Chapter 2, we introduce the Fluidinfo Shell, Fish, which provides a mechanism for sending commands directly to Fluidinfo. Then, in Chapter 3, we survey various web applications that use Fluidinfo to store their data. Finally, we show how Fluidinfo can be used programmatically, first from Python (Chapters 4 and 5), then with JavaScript (Chapter 6), and finally directly using the HTTP API (Chapter 7).
Note
As this book goes to press, Fluidinfo (the company) is just releasing an experimental new interface to Fluidinfo (the storage system) at http://fluidinfo.com. This experimental interface is not described in this book at present, because it is in a constant state of flux, often receiving more than one update in a day. We plan to add a chapter describing it when the interface stabilizes.
[2] The selection and meaning of about tags are determined by emerging conventions discussed in Chapter 9.
[3] Fluidinfo was originally called FluidDB, and the name of the superuser has not been changed.
[4] The note that can be attached to a bookmark in Delicious and Pinboard can be seen as a single example of a special tag with a value in these systems.
[5] You can have more than one account associated with a single email address.
[6] A type of challenge/response test used to ensure that a response is generated by a person.
Get Getting Started with Fluidinfo now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
