Parallel R

Chapter 3. multicore

multicore is a popular parallel programming package for use on multiprocessor and multicore computers. It was written by Simon Urbanek, and first released on CRAN in 2009. It immediately became popular because its clever use of the fork() system call allows it to implement a parallel lapply() operation that is even easier to use than snow’s parLapply().

Unfortunately, because fork() is a Posix system call, multicore can’t really be used on Windows machines.^[33] Fork() can also cause problems for functions that use resources that were allocated or initialized exclusively for the master, or parent process. This is particularly a problem with graphics functions, so it isn’t generally recommended to use multicore with an R GUI.^[34] Nevertheless, multicore works perfectly for most R functions on Posix systems, such as Linux and Mac OS X, and its use of fork() makes it very efficient and convenient, as we’ll see in this chapter.

Quick Look

Motivation: You have an R script that spends an hour executing a function using lapply() on your laptop.

Solution: Replace lapply() with the mclapply() function from the multicore package.

Good because: It’s easy to install, easy to use, and makes use of hardware that you probably already own.

How It Works

multicore is intended to run on Posix-based multiprocessor and multicore systems. This includes almost all modern Mac OS X and Linux desktop and laptop computers. It can also be used on single nodes of a Linux cluster, for example, but ...