Chapter 4. Embedding Pure Data with libpd

Pure Data was originally designed to be an interactive tool for computer music and multimedia, tightly integrating a dataflow programming language and signal processing with a graphical user interface and support for various audio and MIDI interfaces. In this capacity, Pd runs as the top-level application, managing most aspects of its operation and only intermittently delegating control to the audio subsystem of the operating system. The usual way to port Pd to a new platform is by extending Pd, adding support for a new audio API and possibly new objects, known as externals, that make special capabilities of the new platform available to Pd.

When I started thinking about porting Pd to Android, I quickly realized that the usual approach wouldn’t work. At the time, it was possible for Android apps to have native components written in C, but the main body of an app still had to be written in Java. This restriction has since been lifted, but at the time it was impossible to run an extension of Pd as an Android app. Rather, I had to turn the usual model on its head; instead of extending Pd for a new purpose, I had to find a way to embed Pd into Android apps.

Introducing libpd

After much refactoring, the first prototype of an Android port of Pd fell into five main pieces: Pd itself; a thin wrapper on top of Pd that turns it into an embeddable audio library; Java bindings for this library; some platform-specific glue that ties the Java bindings into the audio architecture of Android and provides some common utilities; and the top-level application code. This structure turned out to be highly versatile and applicable to a range of uses far beyond Android, and virtually all libpd-based applications adhere to the same layer model (Figure 4-1).

Layer model of libpd development
Figure 4-1. Layer model of libpd development

In the strictest sense, the term libpd only refers to the second layer (the library wrapper for Pd), but we will play fast and loose with the terminology and include language bindings and audio glue when we talk about libpd.

In this model, each layer only communicates with the layers immediately next to it. Pure Data is written in C, using custom datatypes. The libpd wrapper is also written in C, but its API uses standard datatypes as much as possible. In those few cases where libpd exposes pointers to a Pd-specific datatype, it also provides convenience functions that allow developers to treat those pointers as opaque, without requiring any knowledge of the underlying datatype. The language bindings do away with custom datatypes altogether and exclusively use built-in types of the target language, such as floats, strings, arrays, and lists.

Note

We won’t be writing any code in this chapter, but if you want to download a copy of libpd for closer examination, you can open a terminal and say git clone git://github.com/libpd/libpd.git. Keep in mind, however, that in subsequent chapters we’ll pull in a separate copy of libpd as a submodule of Android or iOS projects. If you clone libpd now, you’ll have two copies of it later, and you’ll need to be careful not to get them mixed up.

API Overview

In this chapter, we will discuss the parts of libpd that are common to both Android and iOS, up to some minor concessions to language and idiom. Those parts include methods for opening and closing patches, for sending messages to and receiving messages from Pd, and for reading and writing arrays in Pd. That’s almost all of libpd, except for the most important parts, the methods that perform the actual audio processing. Musical apps will never call those directly; rather, they will use them by way of platform-specific audio glue.

In both Java and Objective-C, the low-level bindings for libpd are provided by a class called PdBase that is designed to be as small and thin as possible. Its methods are meant to preserve the flavor of old-school C programming that marks Pd itself. In particular, all methods of PdBase are static, and most of them don’t throw exceptions; if something goes wrong, they return a nonzero error code. In some cases, it may make sense to create a class that wraps PdBase in a more object-oriented way; this can be useful if you want to mock out the Pd components for testing, or if you want to provide high-level exception handling. Still, in the spirit of minimality, libpd refrains from introducing unnecessary layers of indirection.

All public methods in PdBase are synchronized, providing a basic level of thread safety. In particular, they guarantee that you can use PdBase in a threaded setting without having to worry about crashes due to race conditions or visibility issues. This is an important point because virtually all libpd-based apps will have at least two threads, one for the user interface and one for audio.

Warning

For the vast majority of apps, the synchronization provided by PdBase is all the thread safety you’ll need. It is not a silver bullet, however, and concurrency-related issues may still sneak up on you. In particular, if you need to be sure that the state of Pd won’t change between subsequent invocations of methods in PdBase, you have to consider additional synchronization.

Opening Patches

Opening patches in Java

static int openPatch(File file) throws IOException;
static int openPatch(String path) throws IOException;
static void closePatch(int handle);

Opening patches in Objective-C

+(void *)openFile:(NSString *)baseName path:(NSString *)pathName;
+(void)closeFile:(void *)handle;
+(int)dollarZeroForFile:(void *)handle;

The calls for opening and closing patches differ slightly between Java and Objective-C, but the basic behavior is the same. You open a patch by sending the path to the patch to libpd, and you get back a handle that identifies the patch. When you want to close the patch, you pass its handle to the close method. You can open multiple copies of the same patch and tell them apart by their handles.

In Objective-C, the handle is simply a pointer to the data structure that represents the patch in Pd, but you can treat it as an opaque pointer in your application code. If you pass this pointer to dollarZeroForFile, you will get the $0 tag of the topmost patch in your file.

As we saw in Chapter 2, the $0 tag is a unique identifier that Pd assigns to a patch, and it is frequently used when creating symbols that are local to a patch. In Java, we can’t use pointers to refer to a patch, and so the openPatch methods in the Java version of PdBase simply return the $0 tag as a handle. Unlike the rest of PdBase, the openPatch methods in Java will throw an exception if something goes wrong.

You don’t have to worry about initializing libpd; the first time you call a method in PdBase, libpd will be initialized automatically.

Finding Resources

Setting the search path in Java

static void clearSearchPath();
static void addToSearchPath(String path);

Setting the search path in Objective-C

+(void)clearSearchPath;
+(void)addToSearchPath:(NSString *)path;

If your patch uses additional resources, such as wav files or abstractions, then it is good practice to package those resources with your patch and to refer to them by relative paths only. In some situations, however, you may have to keep some resources separate from your patch. In this case, you need to add their locations to the search path of Pd so that Pd will be able to find them.

Note

You may be aware that Pd has two search paths, the regular one and the extra path. The extra path exists to solve a problem that won’t occur when working with libpd, and so libpd just leaves it blank.

Sending Messages to Pd

Sending messages in Java

static int sendBang(String receiver);
static int sendFloat(String receiver, float value);
static int sendSymbol(String receiver, String symbol);
static int sendList(String receiver, Object... list);
static int sendMessage(String receiver, String message, Object... list);

Sending messages in Objective-C

+(int)sendBangToReceiver:(NSString *)receiverName;
+(int)sendFloat:(float)value toReceiver:(NSString *)receiverName;
+(int)sendSymbol:(NSString *)symbol toReceiver:(NSString *)receiverName;
+(int)sendList:(NSArray *)list toReceiver:(NSString *)receiverName;
+(int)sendMessage:(NSString *)message withArguments:(NSArray *)list
        toReceiver:(NSString *)receiverName;

Sending messages to Pd is perfectly straightforward. For each supported message type (bang, float, symbol, list, typed message), there is a method that sends a message to a receive symbol in Pd. For instance, in order to send a pitch value of 72 to our synthesizer patch from Chapter 2, we would say PdBase.sendFloat("midinote", 72) in Java, or [PdBase sendFloat:72 toReceiver:@"midinote"] in Objective-C.

The elements of list or typed messages must be strings or numbers, i.e., they must be of type String, Integer, Float, or Double in Java, and NSString or NSNumber in Objective-C. The return value is an error code that will be nonzero if something went wrong, e.g., if the given receiver does not exist or if list objects are of the wrong type. Failures of send methods are rare and usually benign, and so most apps will just ignore their return values.

Note

libpd supports all message types in Pd except pointer messages. The omission of pointer messages was deliberate because Pd pointers have no semantics outside of Pd itself. If you need your app to interact with pointers in Pd, you can store them in a pointer object in your patch and trigger them with a bang from libpd.

Receiving Messages from Pd

Receiving messages from Pd requires several steps. First we need to create a receiver class that implements callbacks for handling messages from Pd, then we register an instance of this class with libpd, and finally we let libpd know which send symbols in Pd we want to subscribe to. This quickly leads to repetitive code, especially when we are receiving messages from multiple send symbols in Pd, and so we won’t discuss the general approach in detail.

Instead, we’ll focus on a special case, a utility class that encapsulates the routine aspects of handling messages from Pd and lets the developer focus on the interesting bits. This utility class, PdDispatcher, is included in the distribution of libpd, for both Java and Objective-C. It implements a publisher-subscriber (pub/sub) pattern for routing messages from Pd.

In order to receive messages from Pd, we register an instance of PdDispatcher with libpd. For each send symbol in Pd that we’re interested in, we register one or more listeners with the dispatcher; the listeners implement callback methods that are ultimately responsible for handling events from Pd (Figure 4-2). Behind the scenes, the dispatcher will subscribe to messages from those send symbols. When a Pd message is sent to one of those symbols, the dispatcher will look up the associated listeners and invoke their callback methods for this type of message.

Pub/sub messaging with PdDispatcher
Figure 4-2. Pub/sub messaging with PdDispatcher

Setting up PdDispatcher in Java

PdDispatcher dispatcher = new PdUiDispatcher();
// Note that we're instantiating a subclass, PdUiDispatcher,
// for reasons explained later on in the text.
PdBase.setReceiver(dispatcher);

Setting up PdDispatcher in Objective-C

dispatcher = [[PdDispatcher alloc] init];
[PdBase setDelegate:dispatcher];

The code snippets in the sections Setting up PdDispatcher in Java and Setting up PdDispatcher in Objective-C create a dispatcher object and register it with PdBase. Note that the setDelegate method in Objective-C retains the dispatcher object. If you aren’t using automatic reference counting (ARC), you’re free to call [dispatcher release] after you’ve registered it with PdBase. Regardless of whether you’re using ARC, you will need to call [PdBase setDispatcher:nil] if you want to unregister and release the dispatcher object altogether.

In addition to the message types we already discussed, dispatchers will also handle printing from Pd, i.e., the messages that appear in the main window when you’re working with Pd itself. Printing is generally not suitable for sending data from Pd to the application code, but it pays to log what Pd prints because you occasionally get useful debugging output that way.

When working with audio, you need to have some awareness of concurrency issues because audio apps typically have at least two threads, the audio thread and the main thread. (In practice, the operating system may spin off many more threads for each app, but these two are the ones that most developers should know of.) Messages generated by Pd originate from the audio thread, but in the vast majority of cases they will affect the user interface, and so they have to be consumed on the main thread.

The best way to dispatch events to the main thread depends on the platform, and the Android branch of libpd takes a different approach from the iOS branch. Invoking callback methods in Objective-C typically involves some memory allocation, which is slow and may even block for an indeterminate amount of time. In the audio thread, we cannot afford blocking calls, and so in the iOS branch of libpd, the audio thread just writes an efficient binary representation of a Pd message to a lock-free buffer shared by the two threads.

The main thread polls this buffer every 20ms, converts Pd messages to Objective-C datatypes, and invokes message callbacks as needed. Like this, all potentially slow operations occur on the main thread, and the audio thread does as little work as possible. The downside of this approach is that the shared message buffer may fill up if the main thread fails to consume messages in a timely fashion. In most cases, this is an indication that your patch generates messages at an unreasonable rate and needs to be revised, but if you’re really determined to send a torrent of messages from Pd to your app, you can try to remedy the occasional overflow by increasing the size of the message buffer. In order to do this, call [PdBase setMessageBufferSize:nBytes] before you install the dispatcher. If you don’t explicitly set the buffer size, the setDispatcher method will allocate a default buffer of 32K. Once the buffer has been allocated, further invocations of setMessageBufferSize have no effect.

For Android, the situation is different, for various reasons. One reason is that the Objective-C components of libpd will only be used with iOS and MacOS, and so it makes sense to bake Cocoa-specific optimizations right into PdBase.m. The Java bindings of libpd, on the other hand, are designed for use beyond the scope of Android, and we cannot make Android-specific assumptions about threading at this level. Hence, the Java bindings pay no attention to threads and just invoke message callbacks on the same thread where they also perform their audio processing. Fortunately, this is not much of a concern because object creation is cheap in Java, and so the conversion of Pd messages from C to Java puts little extra strain on the audio thread.

Note

If you are concerned about placing Java-related overhead on the audio thread, then you can implement your audio components entirely in C, bypassing the Java bindings and using the C API of libpd directly. That would be the topic of another book, though, and besides, it’s not clear how much there is to gain. Before you embark on such a project, you probably want to build a quick prototype in Java, profile it, and convince yourself that the expected performance gain will be worth the effort.

While most Pd messages will originate from the audio thread, they are usually meant to affect the user interface. Since most GUI methods can only be invoked on the main thread, most Pd messages need to be handled on the main thread, and that’s where the PdUiDispatcher class comes in. It’s a subclass of PdDispatcher that invokes listener callbacks on the main thread. If you use this class, you can implement your message handlers without having to pay any further attention to concurrency issues. We will focus on this approach because it is the appropriate one for most apps.

Note

There are rare cases where it is advantageous to have a choice as to which thread to execute a callback on. If you need this flexibility, you can instantiate PdDispatcher directly instead of using PdUiDispatcher, at the cost of having to be aware of threading when handling messages. It can be done, and if you’re versed in the dark art of Java concurrency and how it pertains to Android, then you can achieve the desired effect with fairly little code. The ScenePlayer app for Android, for instance, creates and updates its user interface without explicit thread management.

Now, in order to receive messages from send symbols in Pd, we need to implement a listener interface. This interface is quite similar to the message-sending API that we already discussed; the source parameter indicates the send symbol in Pd that the message was sent from. You generally won’t explicitly call those methods from your code. Instead, libpd will call them when a message from Pd arrives.

Listener interface in Java

public interface PdListener {
    public void receiveBang(String source);
    public void receiveFloat(String source, float x);
    public void receiveSymbol(String source, String symbol);
    public void receiveList(String source, Object... args);
    public void receiveMessage(String source, String symbol, Object... args);
}

Listener interface in Objective-C

@protocol PdListener
@optional
- (void)receiveBangFromSource:(NSString *)source;
- (void)receiveFloat:(float)received fromSource:(NSString *)source;
- (void)receiveSymbol:(NSString *)symbol fromSource:(NSString *)source;
- (void)receiveList:(NSArray *)list fromSource:(NSString *)source;
- (void)receiveMessage:(NSString *)message withArguments:(NSArray *)arguments
            fromSource:(NSString *)source;
@end

Most implementations of the listener interface will not need to handle all types of messages. In Objective-C, all methods are optional, and so you are free to pick and choose the methods you want to implement. For convenience, the listener interface in Java comes with an adapter class that achieves the same effect.

Let’s assume that we have a class, say SpamListener, that implements this interface for the purpose of receiving messages from the send symbol spam. Now we need to create an instance and register it with the dispatcher.

Registering a listener in Java

SpamListener spamListener = new SpamListener();
dispatcher.addListener("spam", spamListener);

Registering a listener in Objective-C

SpamListener *spamListener = [[SpamListener alloc] init];
[dispatcher addListener:spamListener forSource:@"spam"];

Behind the scenes, the dispatcher will subscribe to messages for the symbol spam. From now on, messages sent to spam in Pd will be received by our listener. Of course, you can add listeners for other symbols as well as multiple listeners for the same symbol. You can also remove listeners that are no longer needed.

The Objective-C version of PdDispatcher retains each registered listener, i.e., listeners will not be released as long as they haven’t been removed from the dispatcher, regardless of whether you’re using ARC.

Reading and Writing Arrays in Pd

Read and write access to arrays in libpd is provided by the following three methods. Veteran C programmers will notice that the Java methods mimic the venerable memcpy function of ANSI C.

Accessing Pd arrays in Java

static int arraySize(String name);
static int readArray(float[] destination, int destOffset,
                     String source, int srcOffset, int n);
static int writeArray(String destination, int destOffset,
                      float[] source, int srcOffset, int n);

Accessing Pd arrays in Objective-C

+(int)arraySizeForArrayNamed:(NSString *)arrayName;
+(int)copyArrayNamed:(NSString *)arrayName withOffset:(int)offset
             toArray:(float *)destinationArray count:(int)n;
+(int)copyArray:(float *)sourceArray toArrayNamed:(NSString *)arrayName
     withOffset:(int)offset count:(int)n;

Most of the time, you will probably want to copy entire arrays, in which case both source and target arrays will be of the same size, the offsets will be zero, and the count n will be the size. In some cases, however, you may only want to work with small regions of a large array, and then you can specify the regions in terms of offset and count.

Warning

It is easy to change the size of an array in Pd, and Pd patches do this often. If you are working in a threaded setting, libpd cannot guarantee that the array size will remain the same between invocations of methods of PdBase. All array access methods of PdBase check the bounds of arrays, and so there is no risk of crashes due to mismatched array sizes. Still, the application code will be much easier to write if the sound designer agrees not to resize arrays.

MIDI Support in libpd

Recent versions of libpd support MIDI using an API that’s similar to the API for exchanging messages with Pd, consisting of a set of functions for sending MIDI messages to Pd as well as support for receivers that handle MIDI messages from Pd. If you’re thinking about using the MIDI capabilities of libpd, though, you should take a moment and ask yourself whether this is really necessary. In many cases, you will be better off using Open Sound Control instead. Support for OSC is provided by externals that are easy to add to any libpd-based app.

As we saw in Chapter 2, there are really only two realistic use cases for MIDI in a mobile musical app. You may want to control external MIDI hardware with your app, or you may want to have your app control a patch that was designed to take its input from a MIDI device.

In the first case, you will be responsible for writing the boilerplate that connects libpd to the MIDI API of your platform (if any). It’s not hard, but since this is a rare requirement for mobile apps, we won’t discuss it any further.

Note

Shameless plug: If you’re developing for Android and you’re handy with a soldering iron, then you can build your own Bluetooth-MIDI adapter. You can find links to instructions, schematics, and software on the resource page of this book online, at http://shop.oreilly.com/product/0636920022503.do.

The second case may occur if the sound designer chooses to use a MIDI controller when patching for your app. In that situation, the easiest way to deploy the patch is to have the app assume the role of the MIDI controller, and so we need a way to send MIDI events to Pd. Pd and libpd support all MIDI event types; commonly used ones are represented by dedicated objects in Pd and corresponding functions and callbacks in libpd. For the less common event types, Pd and libpd provide access to raw MIDI bytes, although it is not clear whether anyone will ever need this. We’ll just list the functions in libpd that send channel voice messages to Pd.

Sending MIDI messages in Java

static int sendNoteOn(int channel, int pitch, int velocity);
static int sendControlChange(int channel, int controller, int value);
static int sendProgramChange(int channel, int value);
static int sendPitchBend(int channel, int value);
static int sendAftertouch(int channel, int value);
static int sendPolyAftertouch(int channel, int pitch, int value);

Sending MIDI messages in Objective-C

+(int)sendNoteOn:(int)channel pitch:(int)pitch velocity:(int)velocity;
+(int)sendControlChange:(int)channel controller:(int)controller value:(int)value;
+(int)sendProgramChange:(int)channel value:(int)value;
+(int)sendPitchBend:(int)channel value:(int)value;
+(int)sendAftertouch:(int)channel value:(int)value;
+(int)sendPolyAftertouch:(int)channel pitch:(int)pitch value:(int)value;

Most of the parameters are unsigned 7-bit integers, i.e., their values range from 0 to 127. The only exceptions are channel numbers and pitch bend values. Those two tend to cause some confusion because their binary representation in the MIDI wire format differs from their musical interpretation. Even worse, the pitch bend objects in Pd are inconsistent in their interpretation of pitch bend values.

We resolve this problem by choosing sanity over MIDI specs or consistency with Pd. To wit, channel numbers range from 0 to 15 as far as libpd is concerned, and pitch bend values range from -8192 to 8191, with 0 representing neutral pitch bend. In other words, parameters behave the way a programmer would expect them to, and libpd handles any conversions behind the scenes.

Odds and Ends

When your app is done with Pd, you should release the resources held by libpd. While it’s always a good idea to free up resources as soon as you don’t need them anymore, calling PdBase.release() is critically important for certain Android apps, for reasons we’ll discuss in Chapter 5. When working in Objective-C, you should close all patches and release the dispatcher object by calling [PdBase setDispatcher:nil].

This completes the list of methods in PdBase that you will commonly use when making musical apps. If you take a close look at the class definition, however, you will find a few more methods. Some of them are there for completeness but are rarely needed; the rest are used by the audio glue and will not be called by application code.

Externals in libpd

The functionality of Pd is commonly enhanced by the addition of externals, i.e., objects that are not built into Pd but loaded at runtime, as needed. In fact, many users of Pd do not use Pd Vanilla but Pd Extended, which comes with a wide range of additional externals that provide many useful features.

Patching for libpd tends to involve fewer externals than patching for Pd itself. One reason is that many externals enhance the user interface of Pd, which libpd discards. Another reason is that with libpd, you don’t always have to use an external if you need additional functionality. In many cases, it is easier to implement the desired functionality in your application code instead.

If you want to use externals with libpd, however, you can, both with Android and iOS. The details are highly platform-dependent, and so we will discuss them in Chapter 5 and Chapter 6.

Warning

Unlike the core of Pd and libpd, which have been released under a BSD license, many externals are covered by the GNU General Public License or the GNU Lesser General Public License. Surprisingly, that includes two of the externals that come with Pd Vanilla, expr and expr~. This is an important concern to keep in mind, especially if you intend to submit your app to Apple’s App Store. If you’re thinking about using externals in your app, check their licenses and make sure that they are compatible with the license of your app.

Audio Glue

The audio architectures of Android and iOS are quite different, but libpd aims to provide a coherent interface across platforms, without sacrificing platform-specific functionality. We discuss the common features here and leave most platform-specific considerations for Chapter 5 and Chapter 6.

The common features of the audio glue include methods for initializing, starting, and stopping the audio components, as well as a method that checks whether the audio components are currently active. In Java, the audio glue is provided by a class called PdAudio; in Objective-C, a class called PdAudioController plays a similar role.

Audio glue in Java

public class PdAudio {
  static void initAudio(int sampleRate, int inChannels, int outChannels,
                        int ticksPerBuffer, boolean restart) throws IOException;
  static void startAudio(Context context);
  static void stopAudio();
  static boolean isRunning();
}

Audio glue in Objective-C

@interface PdAudioController : NSObject <AVAudioSessionDelegate>

@property(nonatomic, readonly) int sampleRate;
@property(nonatomic, readonly) int numberChannels;
@property(nonatomic, readonly) BOOL inputEnabled;
@property(nonatomic, readonly) BOOL mixingEnabled;
@property(nonatomic, readonly) int ticksPerBuffer;

@property (nonatomic, getter=isActive) BOOL active;

-(PdAudioStatus)configurePlaybackWithSampleRate:(int)sampleRate
                                 numberChannels:(int)numChannels
                                   inputEnabled:(BOOL)inputEnabled
                                  mixingEnabled:(BOOL)mixingEnabled;

-(PdAudioStatus)configureAmbientWithSampleRate:(int)sampleRate
                                numberChannels:(int)numChannels
                                 mixingEnabled:(BOOL)mixingEnabled;

-(PdAudioStatus)configureTicksPerBuffer:(int)ticksPerBuffer;

@end

In order to initialize the audio glue, you need to specify a number of parameters. Most of them (sample rate, number of channels) are obvious, but one, the number of ticks per buffer, requires an explanation.

Pd computes audio in chunks of 64 frames, known as ticks. When specifying the number of ticks per buffer, you are effectively choosing the duration of the audio buffer through which Pd will exchange audio samples with the operating system. For example, if you request four ticks per buffer at a sample rate of 44100Hz, then the duration will be 4 * 64 / 44100Hz = 5.8ms. Note that this is only a request; PdAudio and PdAudioController will negotiate with the audio subsystem to get a buffer size that’s as close as possible to your request, but depending on the capabilities of your platform, the actual buffer size may be different.

In Objective-C, you don’t have to explicitly specify the number of ticks per buffer because Core Audio will provide a usable default: If you don’t set the number of ticks per buffer, the buffer size will be 512 frames, i.e., eight ticks per buffer.

One minor difference between the Android version and the iOS version is that the Android version will let you choose any combination of input and output channels as long as the requested channel numbers are available, while most audio configurations for iOS will only allow audio output. When audio input is enabled, the number of input channels must equal the number of output channels because the audio unit that connects libpd to Core Audio uses the same channel configuration and buffer for both input and output.

Another difference is the way PdAudio and PdAudioController handle configuration failures. In Java, PdAudio will either give you the configuration you requested, or it will fail and throw an IOException. In Objective-C, the configuration method returns a value of type PdAudioStatus, which is an enum with three elements: PdAudioOK, PdAudioError, and PdAudioPropertyChanged. The first two indicate success or failure, as one might expect. The third one, PdAudioPropertyChanged, indicates partial success, i.e., the controller was able to configure the audio session and create an audio unit, but it had to adjust some parameters. For example, if the requested sample rate is not available, the controller will use the current hardware sample rate instead. When asked to configure input channels on a system that does not provide audio inputs, the controller will configure the audio without inputs. If an audio configuration method returns PdAudioPropertyChanged, you can query the properties of the controller in order to determine whether the outcome is acceptable, or you can just treat it as a failure.

Not only does the audio glue protect you from having to deal with the complexity of the audio subsystem, it also lets your app benefit from the evolution of the underlying technology. You specify the buffer size (essentially, the latency) that you want, and PdAudio and PdAudioController will get you as close to that as currently possible.

Once the audio glue has been initialized, all you need to do is activate and deactivate it as needed. In Java, this is accomplished by calling startAudio and stopAudio. In Objective-C, a setter for the active property serves the same purpose. Finally, if you want to change the audio settings, you can call PdAudio.initAudio(…) again (make sure to set the restart parameter to true) or reconfigure your instance of PdAudioController. Keep in mind, however, that some patches configure themselves at load time, and they may malfunction if you change the sample rate after they have been loaded.

Note

In Chapter 2, I mentioned that the DSP toggle of Pd is redundant when working with libpd. Now we see why. With libpd, it is preferable to start or stop the audio thread instead of toggling the DSP state of Pd. Using two controls for the same purpose would be a recipe for confusion. PdAudio and PdAudioController will automatically enable DSP in Pd upon initialization; I strongly recommend that you don’t touch the DSP toggle of Pd, neither in your patch nor in your application code.

In both Java and Objective-C, the methods for starting and stopping the audio thread simply turn audio processing on and off. This can result in discontinuities in the sound, which will be audible as clicks. The basic audio glue makes no attempt to avoid clicks because there are many different ways of dealing with clicks, and different apps will have different requirements. If clicks on start or stop turn out to be a concern for you, you are responsible for dealing with them. A common technique is to ramp the audio output down before stopping the audio thread, and then ramp it back up when starting the thread again; you can implement this in a subclass of PdAudio or PdAudioController. Miller Puckette’s book, The Theory and Technique of Electronic Music, discusses clicks and their suppression in great depth.

Launch Sequence

When initializing a libpd-based app, it is important to perform the setup in the correct order. Most apps should stick to the following sequence:

  1. Initialize the audio components.

  2. Create a dispatcher and register it with PdBase.

  3. Add listeners, if any.

  4. Load your patch or patches.

  5. Start the audio components.

This order is not cast in stone. The most important rule is that you should initialize the audio components before you open any patches because some patches will query Pd for audio properties like the sample rate upon loading. Everything else is flexible, and you can disregard even this rule if you know that your patches will not query audio properties on load.

Still, it’s a good idea to register a dispatcher early on, even if you don’t intend to add any listeners, because it will log console message from Pd that may give you useful debugging information. You can add or remove listeners at any time, but if you open patches before your listeners are in place, you may miss messages from Pd.

Also, as long as your audio is initialized, you can open or close patches at any time, but if you load a large patch while another patch is playing, you may get audio dropouts. Then again, some developers have created apps that open lots of patches on the fly without ill effects, while the audio thread is running. Don’t be afraid to experiment.

Get Making Musical Apps now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.