O'Reilly logo

XMPP: The Definitive Guide by Remko Tronçon, Kevin Smith, Peter Saint-Andre

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Preface

Why XMPP?

In 1800, it took one or two years to send a message from London to Calcutta and receive a reply. You needed to find a ship’s captain you trusted, who piloted his sailing ship around the Cape of Good Hope and probably stopped in various ports along the way. Then your contact in Calcutta needed to write a reply and send it back to London in a similar fashion. Not exactly instant messaging!

With the invention of the steamship and the opening of the Suez Canal, the time was reduced to a month or two. Air mail reduced the time further to a week or two, and eventually to a few days (“when it absolutely, positively has to be there overnight”). The deployment of commercial email systems introduced us to wait times of only a few minutes (depending on how often you polled your server). And instant messaging (IM) systems such as ICQ® took communication to its logical conclusion: nearly immediate interaction.

As a result of these developments, the useful half-life of information has shrunk significantly, in many cases to mere seconds. For many people, IM trumps email. Blogging trumps newspapers and magazines. Microblogging trumps blogging. Groupchat trumps email discussion lists. Shared editing and whiteboarding trump carefully crafted presentations. Immediate notifications trump once-a-day updates. And the list goes on.

What all these technologies have in common is that the interactions happen in close to real time. To make this possible, we need technologies for real-time communication. Ideally such technologies would be open standards providing the real-time equivalent of HTTP, HTML, and the other building blocks of today’s Internet, because over the long term open standards provide stronger security, greater extensibility, and the possibility for more innovation at the edges than do closed technologies.

The Extensible Messaging and Presence Protocol (XMPP) is just such an open technology for real-time interaction. Consider some of its advantages:

  • XMPP is proven. Over 10 years of development has resulted in a stable, widely deployed, seriously tested, Internet-scale technology, with dozens of interoperable codebases, tens of thousands of deployed services, and millions of end users.

  • XMPP is secure. It provides built-in support for channel encryption and strong authentication, inherent resistance to many forms of malware, a diverse ecosystem of implementations, a decentralized network without a single point of failure, and significant deployment at some of the most security-conscious financial organizations and government agencies worldwide. Work on more advanced features (such as user-friendly end-to-end encryption) continues so that XMPP will be even more secure.

  • XMPP is decentralized. Unlike standalone communication silos, XMPP technologies are deployed in a decentralized client-server architecture with an unlimited number of servers. Any person or organization can run their own XMPP server and connect it to the rest of the network using standard Internet infrastructure such as the Domain Name System (DNS), and certificates are freely available through the XMPP Standards Foundation (XSF) to enable secure federation of XMPP traffic.

  • XMPP is extensible. Because XMPP is at its core a technology for rapidly delivering XML from one place to another, it has been used for a wide range of applications beyond instant messaging, including gaming, social networking, Voice over IP (VoIP), real-time collaboration, alerts and notifications, data syndication, geolocation, intelligent workflows, machine-to-machine communication, and custom applications.

  • XMPP is scalable. The “push” model of information transfer used in XMPP solves serious scaling problems associated with traditional HTTP-based polling approaches; as a result, it enables you to build applications that were literally impossible until now.

  • XMPP is a standard. The core aspects of XMPP have undergone rigorous public review within the Internet Engineering Task Force (IETF), and extensions to XMPP are published in an open, developer-oriented standards process run by the XSF. This approach has resulted in strong technologies that can be freely implemented under any licensing terms, from open source to shareware to proprietary code.

  • XMPP is a community. Open standards, a large number of software products, and a communications network are all good, but the “secret sauce” of XMPP may be its vibrant and friendly community of technologists, developers, open source projects, commercial software companies, service providers, and end users. This community is committed to working together to solve problems and build great new applications.

For these reasons, more and more software developers and service providers are using XMPP to build real-time applications or add real-time interfaces to existing applications. And you can, too, because XMPP provides a simple but powerful set of tools that can help you solve real-world problems. This book will show you how.

Jabber and XMPP

Throughout this book, we use the terms “Jabber” and “XMPP” interchangeably. These technologies were originally developed by Jeremie Miller and the Jabber open source community in 1998–1999. When the community submitted its core protocols to the Internet Engineering Task Force (IETF) in 2002, it chose the name “Extensible Messaging and Presence Protocol” to distinguish the protocol from the broader technology and developer community. You can think of the relationship as “XMPP is to Jabber as HTTP is to the Web.” The term Jabber was proactively trademarked by Jabber, Inc. (now part of Cisco Systems, Inc.) in 2000 to protect the open source community, but the XSF sublicenses the term for use in open source projects and other community activities.

Is This Book for You?

This book may be for you if:

  • You are a software developer who needs a helpful guide to building a real-time application or extending an existing system, as well as relevant reference materials to use during your project.

  • You are a product manager or software architect who is looking for suggestive ideas and case studies regarding real-time systems.

  • You are a software architect or developer who needs a brief but thorough overview of XMPP.

  • You are a researcher, teacher, or student who is designing a research project.

  • You are interested in new technologies and the emergence of the real-time Internet.

Above all, this book provides a practical guide to XMPP. Because XMPP is a well-documented protocol, we regularly refer you to the XMPP specifications for relevant details (these specifications come in two flavors: the core protocols are defined in the Requests for Comments or “RFC” series published by the IETF, and dozens of extensions are defined in the XMPP Extension Protocol or “XEP” series published by the XSF). Because XMPP is widely supported by a large number of servers, clients, and code libraries, both open source and commercial, we refer you to those projects for assistance with real-world implementation. Instead of covering all protocol details and possible implementations, we show how XMPP technologies can be used to solve certain classes of problems by helping you to “think in XMPP” and covering the “gotchas” that can trip up those who are new to XMPP technologies.

Throughout this book, we assume that you are familiar with the very basics of computer networking, common Internet applications (such as email and the World Wide Web), and structured data formats (such as HTML). However, we often treat these technologies as the starting points for our discussion or as “contrast objects” for XMPP, which differs from applications such as the Web in important ways that we’ll describe as we go. Finally, we include some examples using the Python programming language, so some familiarity with Python can also help you understand the concepts we describe.

Getting the Most Out of This Book

To get the most out of this book, we do not recommend that you read it cover to cover in one sitting (although you are welcome to do so!). Instead, first explore the sections that interest you or that you need to complete a particular task, perhaps after reading the introductory materials in Part I. You might also consider skimming over the details of each XML example on your first reading so that you get the general idea of each use case and protocol extension.

The book is organized as follows:

  • Part I provides an overview of XMPP. The first chapter talks about XMPP at a high level and introduces you to some ways XMPP is being used to build real-time applications. The second chapter describes the basics of XMPP technologies, including architectural issues, addressing, and communication primitives. Read this section first if you’d like a relatively quick orientation to XMPP technologies.

  • Part II consists of a series of “developer stories” that illustrate how the tools in the XMPP toolkit can help you solve particular classes of problems. Each chapter in Part II introduces the XMPP concepts and services that you need in a given problem domain, describes how to use those tools, and provides examples showing how specific protocols come into play. Read the chapters here that interest you most. The order doesn’t matter, because we recap concepts where needed, and provide cross-references to more detailed treatments in other chapters.

  • Part III shows you how to put it all together by walking you through the thought processes and design decisions involved in building an XMPP-based application. Read this part after you have a feel for XMPP from the first two parts, and as you begin to dig into a large project that uses XMPP to construct a business application or real-time service.

  • Part IV consists of the appendixes, which help you understand the terminology of XMPP; introduce you to the wealth of XMPP servers, clients, and code libraries; and guide you through the large “stack” of XMPP protocol specifications so you can quickly find what you need. Use these appendixes as reference material on an ongoing basis, or as a quick index to the myriad of XMPP resources available on the Internet.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for protocol examples and sample code, as well as within paragraphs to refer to protocol aspects such as XML elements, attributes, and namespaces, code features such as variable and function names, databases, data types, environment variables, statements, keywords, etc.

Constant width bold

Indicates user input in examples showing an interaction. Also indicates emphasized code elements to which you should pay particular attention.

Tip

This icon signifies a tip, suggestion, or general note.

Caution

This icon indicates a warning or caution.

About the Examples

In Parts I and II, we include a large number of protocol examples (but not nearly as many as you will find in the XMPP specifications, which are extremely thorough). Each example contains a snippet of XML that would be sent over the wire to communicate a message, share presence information, retrieve data, initiate a command sequence, return an error, and the like. These chunks of XML are essentially copied directly from the XMPP specifications with additional notes to highlight their most important and relevant aspects. However, sometimes our examples are incomplete or shortened for readability, so be sure to check the official XMPP specifications for the most accurate examples and protocol descriptions!

Most of the examples in this book use Lewis Carroll’s Alice’s Adventures in Wonderland as the backdrop (Alice and her friends are much more interesting than “User A” and “User B”!). The domain names in these examples are things like wonderland.lit, which clearly don’t work on today’s Internet, because the .lit top-level domain has not yet been assigned. This is intentional (we don’t want to bother anyone who owns a real domain name like wonderland.com).

In Part III, we intersperse protocol examples with software code showing one possible implementation of several protocol interactions. This software code is written in the Python programming language, a popular, easy-to-read language for scripting and application development.

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “XMPP: The Definitive Guide, by Peter Saint-Andre, Kevin Smith, and Remko Tronçon. Copyright 2009 Peter Saint-Andre, Kevin Smith, and Remko Tronçon, 978-0-596-52126-4.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at .

Safari® Books Online

Note

When you see a Safari® Books Online icon on the cover of your favorite technology book, that means the book is available online through the O’Reilly Network Safari Bookshelf.

Safari offers a solution that’s better than e-books. It’s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://my.safaribooksonline.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at:

http://www.oreilly.com/catalog/9780596521264/

To comment or ask technical questions about this book, send email to:

For more information about our books, conferences, Resource Centers, and the O’Reilly Network, see our website at:

http://www.oreilly.com

Finally, the authors of this book can usually be found on the XMPP network in the jdev@conference.jabber.org chat room.

Acknowledgments

We would like to thank Mary Treseler for her editorial guidance throughout this project, and her patience with an enthusiastic but not entirely disciplined group of authors. We’d also like to thank our technical reviewers for their thorough comments on, and insight into, the contents of this book; it was greatly improved by their input. Thank you Dave Cridland, Brian Dainton, Kellan Elliott-McCrea, Michelle Fisher, Nathan Fritz, and Jack Moffitt.

Peter Saint-Andre

Thanks are due to the many developers who helped me understand these technologies as they were being designed in the early days of the Jabber open source community. I would like to especially recognize the help of my friend Peter Millard, who patiently answered my never-ending questions about Jabber technologies from 1999 until his death in 2006. I dedicate my work on this book to his memory.

I would not have been able to contribute to XMPP all these years without the generous support of my employer, Jabber, Inc. (now part of Cisco Systems, Inc.).

Most fundamentally, my wife, Elisa, has always cheerfully tolerated my obsession with XMPP despite countless hours working on specs, posting to discussion lists, writing blog entries, traveling to conferences, and all the rest.

Kevin Smith

Many of the members of the XMPP community have been supportive over the last seven or so years since my first involvement, and I’d like to acknowledge particularly those people I’ve worked with in the Psi and Sleek projects, and those I’ve worked with on the XMPP Council in expanding my knowledge of XMPP and software development. Thanks to Peter and Remko especially, for all the fun we’ve had with this book.

My wife, Cath, has my unending gratitude for her support in my numerous XMPP-related and other free-time-swallowing commitments.

Remko Tronçon

My first words of gratitude go to my coauthors, Peter and Kevin. Not only did they make the writing of this book an incredibly fun experience, but they are also the reason why I got into XMPP in the first place. Thanks to Kevin, the other Psi developers, and the whole Psi userbase, I got the chance to take my first steps into the XMPP world. Thanks to the support of “patron saint” Peter and the rest of the XMPP community, I was able to take this involvement one step further, and joined the conversation to define the XMPP standards. The XMPP community is without a doubt one of the most pleasant and accessible groups of people out there on the interwebs. Thanks to everyone out there who ever talked to me!

My most important source of inspiration, however, comes from outside the digital world. Kim has always unconditionally supported me in all my time-consuming activities, and has continuously pushed me to work harder, even in times when she hardly received any of the attention she deserved.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required