Chapter 4. Embedding Pure Data with libpd
Pure Data was originally designed to be an interactive tool for computer music and multimedia, tightly integrating a dataflow programming language and signal processing with a graphical user interface and support for various audio and MIDI interfaces. In this capacity, Pd runs as the top-level application, managing most aspects of its operation and only intermittently delegating control to the audio subsystem of the operating system. The usual way to port Pd to a new platform is by extending Pd, adding support for a new audio API and possibly new objects, known as externals, that make special capabilities of the new platform available to Pd.
When I started thinking about porting Pd to Android, I quickly realized that the usual approach wouldn’t work. At the time, it was possible for Android apps to have native components written in C, but the main body of an app still had to be written in Java. This restriction has since been lifted, but at the time it was impossible to run an extension of Pd as an Android app. Rather, I had to turn the usual model on its head; instead of extending Pd for a new purpose, I had to find a way to embed Pd into Android apps.
Introducing libpd
After much refactoring, the first prototype of an Android port of Pd fell into five main pieces: Pd itself; a thin wrapper on top of Pd that turns it into an embeddable audio library; Java bindings for this library; some platform-specific glue that ties the Java bindings into the audio architecture of Android and provides some common utilities; and the top-level application code. This structure turned out to be highly versatile and applicable to a range of uses far beyond Android, and virtually all libpd-based applications adhere to the same layer model (Figure 4-1).
In the strictest sense, the term libpd only refers to the second layer (the library wrapper for Pd), but we will play fast and loose with the terminology and include language bindings and audio glue when we talk about libpd.
In this model, each layer only communicates with the layers immediately next to it. Pure Data is written in C, using custom datatypes. The libpd wrapper is also written in C, but its API uses standard datatypes as much as possible. In those few cases where libpd exposes pointers to a Pd-specific datatype, it also provides convenience functions that allow developers to treat those pointers as opaque, without requiring any knowledge of the underlying datatype. The language bindings do away with custom datatypes altogether and exclusively use built-in types of the target language, such as floats, strings, arrays, and lists.
Note
We won’t be writing any code in this chapter, but if you want to
download a copy of libpd for closer examination, you can open a terminal
and say git clone
git://github.com/libpd/libpd.git. Keep in mind,
however, that in subsequent chapters we’ll pull in a separate copy of
libpd as a submodule of Android or iOS projects. If you clone libpd now,
you’ll have two copies of it later, and you’ll need to be careful not to
get them mixed up.
API Overview
In this chapter, we will discuss the parts of libpd that are common to both Android and iOS, up to some minor concessions to language and idiom. Those parts include methods for opening and closing patches, for sending messages to and receiving messages from Pd, and for reading and writing arrays in Pd. That’s almost all of libpd, except for the most important parts, the methods that perform the actual audio processing. Musical apps will never call those directly; rather, they will use them by way of platform-specific audio glue.
In both Java and Objective-C, the low-level bindings for libpd are
provided by a class called PdBase that
is designed to be as small and thin as possible. Its methods are meant to
preserve the flavor of old-school C programming that marks Pd itself. In
particular, all methods of PdBase are
static, and most of them don’t throw exceptions; if something goes wrong,
they return a nonzero error code. In some cases, it may make sense to
create a class that wraps PdBase in a
more object-oriented way; this can be useful if you want to mock out the
Pd components for testing, or if you want to provide high-level exception
handling. Still, in the spirit of minimality, libpd refrains from
introducing unnecessary layers of indirection.
All public methods in PdBase are
synchronized, providing a basic level of thread safety. In particular,
they guarantee that you can use PdBase
in a threaded setting without having to worry about crashes due to race
conditions or visibility issues. This is an important point because
virtually all libpd-based apps will have at least two threads, one for the
user interface and one for audio.
Warning
For the vast majority of apps, the synchronization provided by
PdBase is all the thread safety
you’ll need. It is not a silver bullet, however, and concurrency-related
issues may still sneak up on you. In particular, if you need to be sure
that the state of Pd won’t change between subsequent invocations of
methods in PdBase, you have to
consider additional synchronization.
Opening Patches
static int openPatch(File file) throws IOException; static int openPatch(String path) throws IOException; static void closePatch(int handle);
Opening patches in Objective-C.
+(void *)openFile:(NSString *)baseName path:(NSString *)pathName; +(void)closeFile:(void *)handle; +(int)dollarZeroForFile:(void *)handle;
The calls for opening and closing patches differ slightly between Java and Objective-C, but the basic behavior is the same. You open a patch by sending the path to the patch to libpd, and you get back a handle that identifies the patch. When you want to close the patch, you pass its handle to the close method. You can open multiple copies of the same patch and tell them apart by their handles.
In Objective-C, the handle is simply a pointer to the data structure
that represents the patch in Pd, but you can treat it as an opaque pointer
in your application code. If you pass this pointer to dollarZeroForFile, you will get the $0 tag of the topmost patch in your file.
As we saw in Chapter 2, the $0 tag is a unique identifier that Pd assigns to
a patch, and it is frequently used when creating symbols that are local to
a patch. In Java, we can’t use pointers to refer to a patch, and so the
openPatch methods in the Java version
of PdBase simply return the $0 tag as a handle. Unlike the rest of PdBase, the openPatch methods in Java will throw an
exception if something goes wrong.
You don’t have to worry about initializing libpd; the first time you
call a method in PdBase, libpd will be
initialized automatically.
Finding Resources
Setting the search path in Java.
static void clearSearchPath(); static void addToSearchPath(String path);
Setting the search path in Objective-C.
+(void)clearSearchPath; +(void)addToSearchPath:(NSString *)path;
If your patch uses additional resources, such as wav files or abstractions, then it is good practice to package those resources with your patch and to refer to them by relative paths only. In some situations, however, you may have to keep some resources separate from your patch. In this case, you need to add their locations to the search path of Pd so that Pd will be able to find them.
Note
You may be aware that Pd has two search paths, the regular one and the extra path. The extra path exists to solve a problem that won’t occur when working with libpd, and so libpd just leaves it blank.
Sending Messages to Pd
static int sendBang(String receiver); static int sendFloat(String receiver, float value); static int sendSymbol(String receiver, String symbol); static int sendList(String receiver, Object... list); static int sendMessage(String receiver, String message, Object... list);
Sending messages in Objective-C.
+(int)sendBangToReceiver:(NSString *)receiverName;
+(int)sendFloat:(float)value toReceiver:(NSString *)receiverName;
+(int)sendSymbol:(NSString *)symbol toReceiver:(NSString *)receiverName;
+(int)sendList:(NSArray *)list toReceiver:(NSString *)receiverName;
+(int)sendMessage:(NSString *)message withArguments:(NSArray *)list
toReceiver:(NSString *)receiverName;Sending messages to Pd is perfectly straightforward. For each
supported message type (bang, float, symbol, list, typed message), there
is a method that sends a message to a receive symbol in Pd. For instance,
in order to send a pitch value of 72 to our synthesizer patch from Chapter 2, we would say PdBase.sendFloat("midinote", 72) in Java, or
[PdBase sendFloat:72
toReceiver:@"midinote"] in Objective-C.
The elements of list or typed messages must be strings or numbers,
i.e., they must be of type String,
Integer, Float, or Double in Java, and NSString or NSNumber in Objective-C. The return value is an
error code that will be nonzero if something went wrong, e.g., if the
given receiver does not exist or if list objects are of the wrong type.
Failures of send methods are rare and usually benign, and so most apps
will just ignore their return values.
Note
libpd supports all message types in Pd except pointer messages. The omission of pointer messages was deliberate because Pd pointers have no semantics outside of Pd itself. If you need your app to interact with pointers in Pd, you can store them in a pointer object in your patch and trigger them with a bang from libpd.
Receiving Messages from Pd
Receiving messages from Pd requires several steps. First we need to create a receiver class that implements callbacks for handling messages from Pd, then we register an instance of this class with libpd, and finally we let libpd know which send symbols in Pd we want to subscribe to. This quickly leads to repetitive code, especially when we are receiving messages from multiple send symbols in Pd, and so we won’t discuss the general approach in detail.
Instead, we’ll focus on a special case, a utility class that
encapsulates the routine aspects of handling messages from Pd and lets the
developer focus on the interesting bits. This utility class, PdDispatcher, is included in the distribution of
libpd, for both Java and Objective-C. It implements a publisher-subscriber
(pub/sub) pattern for routing messages from Pd.
In order to receive messages from Pd, we register an instance of
PdDispatcher with libpd. For each send
symbol in Pd that we’re interested in, we register one or more listeners
with the dispatcher; the listeners implement callback methods that are
ultimately responsible for handling events from Pd (Figure 4-2). Behind the scenes, the dispatcher will
subscribe to messages from those send symbols. When a Pd message is sent
to one of those symbols, the dispatcher will look up the associated
listeners and invoke their callback methods for this type of
message.
Setting up PdDispatcher in Java.
PdDispatcher dispatcher = new PdUiDispatcher(); // Note that we're instantiating a subclass, PdUiDispatcher, // for reasons explained later on in the text. PdBase.setReceiver(dispatcher);
Setting up PdDispatcher in Objective-C.
dispatcher = [[PdDispatcher alloc] init]; [PdBase setDelegate:dispatcher];
The code snippets in the sections Setting up PdDispatcher in Java
and Setting up PdDispatcher in Objective-C create a dispatcher object and
register it with PdBase. Note that the
setDelegate method in Objective-C
retains the dispatcher object. If you aren’t using automatic reference
counting (ARC), you’re free to call [dispatcher
release] after you’ve registered it with PdBase. Regardless of whether you’re using ARC,
you will need to call [PdBase
setDispatcher:nil] if you want to unregister and release the
dispatcher object altogether.
In addition to the message types we already discussed, dispatchers will also handle printing from Pd, i.e., the messages that appear in the main window when you’re working with Pd itself. Printing is generally not suitable for sending data from Pd to the application code, but it pays to log what Pd prints because you occasionally get useful debugging output that way.
When working with audio, you need to have some awareness of concurrency issues because audio apps typically have at least two threads, the audio thread and the main thread. (In practice, the operating system may spin off many more threads for each app, but these two are the ones that most developers should know of.) Messages generated by Pd originate from the audio thread, but in the vast majority of cases they will affect the user interface, and so they have to be consumed on the main thread.
The best way to dispatch events to the main thread depends on the platform, and the Android branch of libpd takes a different approach from the iOS branch. Invoking callback methods in Objective-C typically involves some memory allocation, which is slow and may even block for an indeterminate amount of time. In the audio thread, we cannot afford blocking calls, and so in the iOS branch of libpd, the audio thread just writes an efficient binary representation of a Pd message to a lock-free buffer shared by the two threads.
The main thread polls this buffer every 20ms, converts Pd messages
to Objective-C datatypes, and
invokes message callbacks as needed. Like this, all potentially slow
operations occur on the main thread, and the audio thread does as little
work as possible. The downside of this approach is that the shared message
buffer may fill up if the main thread fails to consume messages in a
timely fashion. In most cases, this is an indication that your patch
generates messages at an unreasonable rate and needs to be revised, but if
you’re really determined to send a torrent of messages from Pd to your
app, you can try to remedy the occasional overflow by increasing the size
of the message buffer. In order to do this, call [PdBase setMessageBufferSize:nBytes] before you
install the dispatcher. If you don’t explicitly set the buffer size, the
setDispatcher method will allocate a default buffer of
32K. Once the buffer has been allocated, further invocations of setMessageBufferSize have no effect.
For Android, the situation is different, for various reasons. One
reason is that the Objective-C components of libpd will only be used with
iOS and MacOS, and so it makes sense to bake Cocoa-specific optimizations
right into PdBase.m. The Java bindings
of libpd, on the other hand, are designed for use beyond the scope of
Android, and we cannot make Android-specific assumptions about threading
at this level. Hence, the Java bindings pay no attention to threads and
just invoke message callbacks on the same thread where they also perform
their audio processing. Fortunately, this is not much of a concern because
object creation is cheap in Java, and so the conversion of Pd messages
from C to Java puts little extra strain on the audio thread.
Note
If you are concerned about placing Java-related overhead on the audio thread, then you can implement your audio components entirely in C, bypassing the Java bindings and using the C API of libpd directly. That would be the topic of another book, though, and besides, it’s not clear how much there is to gain. Before you embark on such a project, you probably want to build a quick prototype in Java, profile it, and convince yourself that the expected performance gain will be worth the effort.
While most Pd messages will originate from the audio thread, they
are usually meant to affect the user interface. Since most GUI methods can
only be invoked on the main thread, most Pd messages need to be handled on
the main thread, and that’s where the PdUiDispatcher class comes in. It’s a subclass
of PdDispatcher that invokes listener
callbacks on the main thread. If you use this class, you can implement
your message handlers without having to pay any further attention to
concurrency issues. We will focus on this approach because it is the
appropriate one for most apps.
Note
There are rare cases where it is advantageous to have a choice as
to which thread to execute a callback on. If you need this flexibility,
you can instantiate PdDispatcher
directly instead of using PdUiDispatcher, at the cost of having to be
aware of threading when handling messages. It can be done, and if you’re
versed in the dark art of Java concurrency and how it pertains to
Android, then you can achieve the desired effect with fairly little
code. The ScenePlayer app for Android, for instance, creates and updates
its user interface without explicit thread management.
Now, in order to receive messages from send symbols in Pd, we need to implement a listener interface. This interface is quite similar to the message-sending API that we already discussed; the source parameter indicates the send symbol in Pd that the message was sent from. You generally won’t explicitly call those methods from your code. Instead, libpd will call them when a message from Pd arrives.
public interface PdListener {
public void receiveBang(String source);
public void receiveFloat(String source, float x);
public void receiveSymbol(String source, String symbol);
public void receiveList(String source, Object... args);
public void receiveMessage(String source, String symbol, Object... args);
}Listener interface in Objective-C.
@protocol PdListener
@optional
- (void)receiveBangFromSource:(NSString *)source;
- (void)receiveFloat:(float)received fromSource:(NSString *)source;
- (void)receiveSymbol:(NSString *)symbol fromSource:(NSString *)source;
- (void)receiveList:(NSArray *)list fromSource:(NSString *)source;
- (void)receiveMessage:(NSString *)message withArguments:(NSArray *)arguments
fromSource:(NSString *)source;
@endMost implementations of the listener interface will not need to handle all types of messages. In Objective-C, all methods are optional, and so you are free to pick and choose the methods you want to implement. For convenience, the listener interface in Java comes with an adapter class that achieves the same effect.
Let’s assume that we have a class, say SpamListener, that implements this interface for
the purpose of receiving messages from the send symbol spam. Now we need to create an instance and
register it with the dispatcher.
Registering a listener in Java.
SpamListener spamListener = new SpamListener();
dispatcher.addListener("spam", spamListener);Registering a listener in Objective-C.
SpamListener *spamListener = [[SpamListener alloc] init]; [dispatcher addListener:spamListener forSource:@"spam"];
Behind the scenes, the dispatcher will subscribe to messages for the
symbol spam. From now on, messages sent
to spam in Pd will be received by our
listener. Of course, you can add listeners for other symbols as well as
multiple listeners for the same symbol. You can also remove listeners that
are no longer needed.
The Objective-C version of PdDispatcher retains each registered listener,
i.e., listeners will not be released as long as they haven’t been removed
from the dispatcher, regardless of whether you’re using ARC.
Reading and Writing Arrays in Pd
Read and write access to arrays in libpd is provided by the
following three methods. Veteran C programmers will notice that the Java
methods mimic the venerable memcpy function of ANSI C.
static int arraySize(String name);
static int readArray(float[] destination, int destOffset,
String source, int srcOffset, int n);
static int writeArray(String destination, int destOffset,
float[] source, int srcOffset, int n);Accessing Pd arrays in Objective-C.
+(int)arraySizeForArrayNamed:(NSString *)arrayName;
+(int)copyArrayNamed:(NSString *)arrayName withOffset:(int)offset
toArray:(float *)destinationArray count:(int)n;
+(int)copyArray:(float *)sourceArray toArrayNamed:(NSString *)arrayName
withOffset:(int)offset count:(int)n;Most of the time, you will probably want to copy entire arrays, in
which case both source and target arrays will be of the same size, the
offsets will be zero, and the count n
will be the size. In some cases, however, you may only want to work with
small regions of a large array, and then you can specify the regions in
terms of offset and count.
Warning
It is easy to change the size of an array in Pd, and Pd patches do
this often. If you are working in a threaded setting, libpd cannot
guarantee that the array size will remain the same between invocations
of methods of PdBase. All array
access methods of PdBase check the
bounds of arrays, and so there is no risk of crashes due to mismatched
array sizes. Still, the application code will be much easier to write if
the sound designer agrees not to resize arrays.
MIDI Support in libpd
Recent versions of libpd support MIDI using an API that’s similar to the API for exchanging messages with Pd, consisting of a set of functions for sending MIDI messages to Pd as well as support for receivers that handle MIDI messages from Pd. If you’re thinking about using the MIDI capabilities of libpd, though, you should take a moment and ask yourself whether this is really necessary. In many cases, you will be better off using Open Sound Control instead. Support for OSC is provided by externals that are easy to add to any libpd-based app.
As we saw in Chapter 2, there are really only two realistic use cases for MIDI in a mobile musical app. You may want to control external MIDI hardware with your app, or you may want to have your app control a patch that was designed to take its input from a MIDI device.
In the first case, you will be responsible for writing the boilerplate that connects libpd to the MIDI API of your platform (if any). It’s not hard, but since this is a rare requirement for mobile apps, we won’t discuss it any further.
Note
Shameless plug: If you’re developing for Android and you’re handy with a soldering iron, then you can build your own Bluetooth-MIDI adapter. You can find links to instructions, schematics, and software on the resource page of this book online, at http://shop.oreilly.com/product/0636920022503.do.
The second case may occur if the sound designer chooses to use a MIDI controller when patching for your app. In that situation, the easiest way to deploy the patch is to have the app assume the role of the MIDI controller, and so we need a way to send MIDI events to Pd. Pd and libpd support all MIDI event types; commonly used ones are represented by dedicated objects in Pd and corresponding functions and callbacks in libpd. For the less common event types, Pd and libpd provide access to raw MIDI bytes, although it is not clear whether anyone will ever need this. We’ll just list the functions in libpd that send channel voice messages to Pd.
Sending MIDI messages in Java.
static int sendNoteOn(int channel, int pitch, int velocity); static int sendControlChange(int channel, int controller, int value); static int sendProgramChange(int channel, int value); static int sendPitchBend(int channel, int value); static int sendAftertouch(int channel, int value); static int sendPolyAftertouch(int channel, int pitch, int value);
Sending MIDI messages in Objective-C.
+(int)sendNoteOn:(int)channel pitch:(int)pitch velocity:(int)velocity; +(int)sendControlChange:(int)channel controller:(int)controller value:(int)value; +(int)sendProgramChange:(int)channel value:(int)value; +(int)sendPitchBend:(int)channel value:(int)value; +(int)sendAftertouch:(int)channel value:(int)value; +(int)sendPolyAftertouch:(int)channel pitch:(int)pitch value:(int)value;
Most of the parameters are unsigned 7-bit integers, i.e., their values range from 0 to 127. The only exceptions are channel numbers and pitch bend values. Those two tend to cause some confusion because their binary representation in the MIDI wire format differs from their musical interpretation. Even worse, the pitch bend objects in Pd are inconsistent in their interpretation of pitch bend values.
We resolve this problem by choosing sanity over MIDI specs or consistency with Pd. To wit, channel numbers range from 0 to 15 as far as libpd is concerned, and pitch bend values range from -8192 to 8191, with 0 representing neutral pitch bend. In other words, parameters behave the way a programmer would expect them to, and libpd handles any conversions behind the scenes.
Odds and Ends
When your app is done with Pd, you should release the resources held
by libpd. While it’s always a good idea to free up resources as soon as
you don’t need them anymore, calling PdBase.release() is critically important for
certain Android apps, for reasons we’ll discuss in Chapter 5. When working in Objective-C, you should close all
patches and release the dispatcher object by calling [PdBase setDispatcher:nil].
This completes the list of methods in PdBase that you will commonly use when making
musical apps. If you take a close look at the class definition, however,
you will find a few more methods. Some of them are there for completeness
but are rarely needed; the rest are used by the audio glue and will not be
called by application code.
Externals in libpd
The functionality of Pd is commonly enhanced by the addition of externals, i.e., objects that are not built into Pd but loaded at runtime, as needed. In fact, many users of Pd do not use Pd Vanilla but Pd Extended, which comes with a wide range of additional externals that provide many useful features.
Patching for libpd tends to involve fewer externals than patching for Pd itself. One reason is that many externals enhance the user interface of Pd, which libpd discards. Another reason is that with libpd, you don’t always have to use an external if you need additional functionality. In many cases, it is easier to implement the desired functionality in your application code instead.
If you want to use externals with libpd, however, you can, both with Android and iOS. The details are highly platform-dependent, and so we will discuss them in Chapter 5 and Chapter 6.
Warning
Unlike the core of Pd and libpd, which have been released under a
BSD license, many externals are covered by the GNU General Public
License or the GNU Lesser General Public License. Surprisingly, that
includes two of the externals that come with Pd Vanilla, expr and expr~. This is an important concern to keep in
mind, especially if you intend to submit your app to Apple’s App Store.
If you’re thinking about using externals in your app, check their
licenses and make sure that they are compatible with the license of your
app.
Audio Glue
The audio architectures of Android and iOS are quite different, but libpd aims to provide a coherent interface across platforms, without sacrificing platform-specific functionality. We discuss the common features here and leave most platform-specific considerations for Chapter 5 and Chapter 6.
The common features of the audio glue include methods for
initializing, starting, and stopping the audio components, as well as a
method that checks whether the audio components are currently active. In
Java, the audio glue is provided by a class called PdAudio; in Objective-C, a class called PdAudioController plays a similar role.
public class PdAudio {
static void initAudio(int sampleRate, int inChannels, int outChannels,
int ticksPerBuffer, boolean restart) throws IOException;
static void startAudio(Context context);
static void stopAudio();
static boolean isRunning();
}@interface PdAudioController : NSObject <AVAudioSessionDelegate>
@property(nonatomic, readonly) int sampleRate;
@property(nonatomic, readonly) int numberChannels;
@property(nonatomic, readonly) BOOL inputEnabled;
@property(nonatomic, readonly) BOOL mixingEnabled;
@property(nonatomic, readonly) int ticksPerBuffer;
@property (nonatomic, getter=isActive) BOOL active;
-(PdAudioStatus)configurePlaybackWithSampleRate:(int)sampleRate
numberChannels:(int)numChannels
inputEnabled:(BOOL)inputEnabled
mixingEnabled:(BOOL)mixingEnabled;
-(PdAudioStatus)configureAmbientWithSampleRate:(int)sampleRate
numberChannels:(int)numChannels
mixingEnabled:(BOOL)mixingEnabled;
-(PdAudioStatus)configureTicksPerBuffer:(int)ticksPerBuffer;
@endIn order to initialize the audio glue, you need to specify a number of parameters. Most of them (sample rate, number of channels) are obvious, but one, the number of ticks per buffer, requires an explanation.
Pd computes audio in chunks of 64 frames, known as
ticks. When specifying the number of ticks per
buffer, you are effectively choosing the duration of the audio buffer
through which Pd will exchange audio samples with the operating system.
For example, if you request four ticks per buffer at a sample rate of
44100Hz, then the duration will be 4 * 64 / 44100Hz = 5.8ms. Note that
this is only a request; PdAudio and
PdAudioController will negotiate with
the audio subsystem to get a buffer size that’s as close as possible to
your request, but depending on the capabilities of your platform, the
actual buffer size may be different.
In Objective-C, you don’t have to explicitly specify the number of ticks per buffer because Core Audio will provide a usable default: If you don’t set the number of ticks per buffer, the buffer size will be 512 frames, i.e., eight ticks per buffer.
One minor difference between the Android version and the iOS version is that the Android version will let you choose any combination of input and output channels as long as the requested channel numbers are available, while most audio configurations for iOS will only allow audio output. When audio input is enabled, the number of input channels must equal the number of output channels because the audio unit that connects libpd to Core Audio uses the same channel configuration and buffer for both input and output.
Another difference is the way PdAudio and PdAudioController handle configuration failures.
In Java, PdAudio will either give you
the configuration you requested, or it will fail and throw an IOException. In Objective-C, the configuration
method returns a value of type PdAudioStatus, which is an enum with three
elements: PdAudioOK, PdAudioError, and PdAudioPropertyChanged. The first two indicate
success or failure, as one might expect. The third one, PdAudioPropertyChanged, indicates partial
success, i.e., the controller was able to configure the audio session and
create an audio unit, but it had to adjust some parameters. For example,
if the requested sample rate is not available, the controller will use the
current hardware sample rate instead. When asked to configure input
channels on a system that does not provide audio inputs, the controller
will configure the audio without inputs. If an audio configuration method
returns PdAudioPropertyChanged, you can
query the properties of the controller in order to determine whether the
outcome is acceptable, or you can just treat it as a failure.
Not only does the audio glue protect you from having to deal with
the complexity of the audio subsystem, it also lets your app benefit from
the evolution of the underlying technology. You specify the buffer size
(essentially, the latency) that you want, and PdAudio and PdAudioController will get you as close to that
as currently possible.
Once the audio glue has been initialized, all you need to do is
activate and deactivate it as needed. In Java, this is accomplished by
calling startAudio and stopAudio. In Objective-C, a setter for the active property serves the same purpose.
Finally, if you want to change the audio settings, you can call PdAudio.initAudio(…) again (make sure to set the
restart parameter to true) or reconfigure your instance of PdAudioController. Keep in mind, however, that
some patches configure themselves at load time, and they may malfunction
if you change the sample rate after they have been loaded.
Note
In Chapter 2, I mentioned that the DSP toggle of
Pd is redundant when working with libpd. Now we see why. With libpd, it
is preferable to start or stop the audio thread instead of toggling the
DSP state of Pd. Using two controls for the same purpose would be a
recipe for confusion. PdAudio and
PdAudioController will automatically
enable DSP in Pd upon initialization; I strongly recommend that you
don’t touch the DSP toggle of Pd, neither in your patch nor in your
application code.
In both Java and Objective-C, the methods for starting and stopping
the audio thread simply turn audio processing on and off. This can result
in discontinuities in the sound, which will be audible as clicks. The
basic audio glue makes no attempt to avoid clicks because there are many
different ways of dealing with clicks, and different apps will have
different requirements. If clicks on start or stop turn out to be a
concern for you, you are responsible for dealing with them. A common
technique is to ramp the audio output down before stopping the audio
thread, and then ramp it back up when starting the thread again; you can
implement this in a subclass of PdAudio
or PdAudioController. Miller Puckette’s
book, The Theory and Technique of Electronic Music,
discusses clicks and their suppression in great depth.
Launch Sequence
When initializing a libpd-based app, it is important to perform the setup in the correct order. Most apps should stick to the following sequence:
Initialize the audio components.
Create a dispatcher and register it with
PdBase.Add listeners, if any.
Load your patch or patches.
Start the audio components.
This order is not cast in stone. The most important rule is that you should initialize the audio components before you open any patches because some patches will query Pd for audio properties like the sample rate upon loading. Everything else is flexible, and you can disregard even this rule if you know that your patches will not query audio properties on load.
Still, it’s a good idea to register a dispatcher early on, even if you don’t intend to add any listeners, because it will log console message from Pd that may give you useful debugging information. You can add or remove listeners at any time, but if you open patches before your listeners are in place, you may miss messages from Pd.
Also, as long as your audio is initialized, you can open or close patches at any time, but if you load a large patch while another patch is playing, you may get audio dropouts. Then again, some developers have created apps that open lots of patches on the fly without ill effects, while the audio thread is running. Don’t be afraid to experiment.
Get Making Musical Apps now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
