Making Musical Apps

Chapter 1. Introduction

Pure Data is a great audio engine; it is powerful, flexible, and extensible. With the appearance of libpd, its range has grown beyond the desktop to mobile and embedded settings. Its permissive BSD license allows developers to add libpd to virtually any project. The complete absence of dependencies means that you can build libpd as soon as you have a C compiler. Ready-made bindings for Java and Objective-C as well as support for Android and iOS help you get mobile apps off the ground in a hurry.

One of the most attractive features of Pd is its instantaneous interactive nature; you change the signal processing graph on the screen while Pd is running, and you hear the effect immediately. This makes it an excellent tool for prototyping audio components, especially for games.

In the past, game developers have rigged up development setups that connect a game to Pd using the networking capabilities of Pd. While this is an effective way of creating preliminary audio components, a prototype created in this way will always be a prototype, and the use of network objects in Pd introduces some friction into the patching process. In order to deploy components created in this manner, developers have to reimplement them in a format that can be shipped with the game.

With libpd, all these inefficiencies disappear. Sound designers can prototype audio components in Pd, using the same objects and techniques that they would use when creating a patch for any other purpose. When the patch is done, the app developer simply adds it to the resources of the app, much like a media file. The UI elements that the designer uses when creating and testing a patch become the conduit through which the app controls the patch.

The design of libpd achieves complete separation of concerns between audio development and app development, as well as a smooth workflow from prototyping to production. With libpd, the prototype is the production code.

A Great Investment

Learning Pure Data in general and libpd in particular is a great investment. In a technological environment that changes so quickly that many books are out of date before they are published, Pure Data is a rare pocket of stability. Realizing that many works in computer music are at risk of being lost because they depend on obsolete technology, Miller Puckette designed Pd to be a durable and stable platform, and he started the Pd Repertory project, which aims to port valuable compositions to Pd in order to preserve them for posterity.

In a similar vein, libpd aims to provide a stable API for embedded audio that behaves in much the same way across a wide variety of platforms and languages and protects the developer from the ever-changing arcana of current technology, providing a level of abstraction that lets you focus on the work at hand. Think of it as your insanity abstraction layer. That doesn’t mean that libpd won’t change; both Pd and libpd are under active development, but most changes will happen under the hood, without affecting the API that client code depends on.

At the Pure Data Convention 2011 in Weimar, I participated in a panel discussion on the future of Pd. There was no shortage of ideas for improvements; a better user interface, faster release cycle, optimizations for modern processors, etc. What really struck me, however, was that nobody suggested new audio processing capabilities. Fifteen years after the appearance of Pd, people have yet to reach the limits of Pd as a synthesis engine. The mathematics at the core of Pd is as powerful now as it was in 1996, and Miller’s understanding of it is second to none. That’s the real reason why Pd is here to stay.

Resources

The main project repository of libpd is available at GitHub. In addition to the core library and components for Android and iOS, you can find support for other languages as well as some loosely associated projects. If you’re handy with a soldering iron and not afraid to try experimental hardware, you can find instructions for building and using a Bluetooth-MIDI interface for Android devices. You can also find much documentation in the project wiki at GitHub.

The libpd community meets at Pd Everywhere; you can find a link at http://shop.oreilly.com/product/0636920022503.do. In less than a year, Pd Everywhere has grown from a fledgling site to an active group of more than 200 developers, ranging from core maintainers to casual users. The tone is always helpful and civil, with an amazing signal-to-noise ratio. If you run into a problem with libpd, you should turn to Pd Everywhere first. We’ll be happy to hear from you. Finally, if you have a question about Pd itself rather than the libpd wrapper, you can turn to the Pd mailing list.

Prerequisites

This book assumes that you have some working knowledge of Android or iOS programming, as well as a working development environment. If you’re just getting started, I recommend Learning Android by Marko Gargenta (O’Reilly) as well as Professional Android 2 Application Development by Reto Meier (Wrox) for Android; and Programming iOS 5 by Matt Neuburg (O’Reilly) as well as Learning iOS Programming by Alasdair Allan (O’Reilly) for iOS.

You will need to be able to run sample applications on an actual device instead of just emulating or simulating one in software. Only testing an app with the Android emulator or iPhone simulator is not enough. This is a common admonition, but in the case of musical apps it applies twofold. Generally speaking, you need to test your apps on an actual device because an app that works just fine in simulation may fail in the wild. When doing audio, however, you also have the converse problem—an app that produces glitchy audio in simulation may work just fine when running on a real device. One way or another, testing in simulation will tell you little about the quality of an app.

Android Setup

The Android team recommends Eclipse for Android development, and so that’s what we’ll be using. You’ll need a recent version of the Android Software Development Kit (SDK), a recent version of Eclipse, and the Android Development Tools (ADT) for Eclipse. The Android developer site explains how to set all this up (http://developer.android.com/sdk/installing.html). Make sure to use Eclipse 3.7 or later; older versions will give you much grief. Even though libpd itself is written in C, you won’t need to install the Android Native Development Toolkit (NDK) if you just want to write libpd-based apps in Java.

The Android development environment can be tricky to set up. Follow the steps very carefully and try a few of the sample apps that come with the SDK to convince yourself that your installation is ready for Android development.

Warning

A complete Android development environment consists of several separate but interconnected pieces: Eclipse, Java Development Kit (JDK), ADT, SDK, and NDK. This is an occasional cause of confusion because they are all updated independently; you need to choose versions that will work together. As a general rule, the maintainers of libpd aim to make things work with the latest version of all components, but sometimes it may take a while to catch up. When in doubt, check the libpd wiki for the currently recommended setup.

The installation tool of the Android SDK will ask you which SDK versions to download. I recommend that you select all of them and install new ones as they become available, even if you only intend to target older versions. You can deploy libpd-based apps to API Level 3 (Cupcake) or higher, but you will need at least API Level 10 to build an app with libpd.

In addition to Android SDK versions, you need to pay attention to Java compiler settings. Pd for Android requires Java 1.6. Each project in Pd for Android explicitly enforces this requirement, but you should still select Preferences → Java → Compiler and make sure that the compiler compliance level of your workspace is set to 1.6 and that Java 1.6 is installed on your machine. If this conflicts with other projects of yours, I recommend that you create a new workspace for projects that use libpd.

Note

In the past, different versions of the ADT have taken different approaches to laying out and configuring projects. The Android branch of libpd tracks the development of the ADT, and we make an effort to remain compatible with the latest version. This means that you will need Version 15 or later of the ADT, regardless of which version of Android you are developing for.

If a future upgrade of the ADT breaks a project of yours, you can update your project configuration by opening its context menu and selecting Android Tools → Fix Project Properties.

The Android emulator is extremely limited in its audio capabilities. At the time of writing, it will only do audio at a sample rate of 8000Hz, and the round-trip audio latency from microphone to speaker is quite large. You will need to run your app on an actual device if you want to test its audio performance.

iOS Setup

Setting up a Mac for iOS development is straightforward; if you install Xcode 4.2, you’ll automatically get all the components you need, including the revision control system Git. Xcode 4.2 is quite different from earlier versions, and you should consider upgrading if you haven’t done so already.

In particular, Xcode 4.2 introduced Automatic Reference Counting (ARC). The iOS branch of libpd is built without ARC, but this choice does not affect client code. Projects using libpd will work equally well regardless of whether ARC is enabled. Since Apple recommends ARC for new development, the sample projects in this book will use it, but we will also point out the minor differences to keep in mind when building projects without ARC.

You will need an iOS device for testing, and you need to be able to run your own apps on it. That requires enrolling in Apple’s iOS Developer Program and paying a fee. The simulator tends to produce glitchy audio, and it doesn’t support all audio session categories and channel configurations. The maintainers of the iOS branch of libpd are making every effort to allow for at least some basic testing in simulation, but the simulator has been a moving target so far. Give it a try, but don’t expect it to work in all situations.

Git

Git is a relatively new revision control system that’s becoming more and more popular. You will need it for downloading and updating all libraries discussed in this book. Knowledge of Git is not a prerequisite, and we will list all Git commands that you will need to work with libpd. If you intend to manage your libpd-based projects with Git (and I highly recommend that you do), then you need to study up on Git. Many excellent tutorials are available online.

Coming from the Linux world, Git is most at home in a terminal. There are a few graphical frontends for Git, but none of them seem entirely convincing at this time. If you’re using Linux or Mac OS X, you can use the built-in terminal. If you’re using Windows, use the GitBash tool that comes with the distribution of Git. If you’re unfamiliar with the Unix command line, you should study up on basic shell commands. It won’t take long, and it’s a useful skill to have.

The way to install Git depends on your operating system. If you’re using a Mac, Git will be included in the installation of Xcode 4.2. If you’re using one of the popular Linux distributions, search for Git in your package manager and install the basic Git package. Windows users can find installation archives online. The most basic one will do—just make sure to select GitBash when prompted.

Note

If you’re doing Android development, you may be aware of EGit, a Git plugin for Eclipse. EGit is not adequate for our purposes because libpd and its branches are organized in terms of Git submodules and EGit doesn’t support submodules yet.

On the iOS side, Xcode 4.2 has some Git support baked right into it, but that won’t cut it, either, for the same reason: poor handling of submodules.