Building the Realtime User Experience

Chapter 2. Realtime Syndication

Interacting on the realtime web involves a lot of give and take; it’s more than just removing the need to refresh your web browser and having updates filter in as they happen. Acquiring content from external sources and publishing it back also must happen in realtime. On the Web, this is called syndication, a process in which content is broadcasted from one place to another.

Most syndication on the Web happens through the transmission of XML files, specifically RSS or Atom, from the publisher to the consumer. This model has always been fairly simple: a publisher specifies a feed location and updates the content in that file as it’s posted to the site. Consumers of this content, having no way of knowing when new content is posted, have to check that file every half hour or so to see whether any new content has arrived. If a consumer wanted the content faster, they’d have to check the feed more often. However, most publishers frown upon that type of activity and specifically prohibit it in their terms of service. If too many consumers start downloading all of the feeds on a site every minute, it would be very taxing on the server.

Although this has been a problem on the Web for as long as RSS feeds have been around, only recently have people put serious effort into fixing the issue. There are a good number of competing standards aimed at solving this problem. Each of these solutions has had varying degrees of success in getting sites to adopt ...