Parallel Computing for Data Science

40 CHAPTER 3. SCHEDULING

So, again thinking of the case of 500 rows and 4 workers, the code

ichunk s <− c l u s t e r S p l i t ( cl s , 1 : ( nr −1))

t o t s <− c l u s t e r A p p l y ( c l s , ichunks , doichunk )

will send the chunk 1:125 to the first worker, 126:250 to the second, 251:375

to the third, and 375:499 to the fourth. The return list, assigned to tots,

will now consist of four elements, rather than 499 as before.

Again, the only change from the previous version of this code was to add

real chunks. This ought to help, because it allows us to better leverage

the fact that R can do matrix multiplication fairly quickly. Let’s see if this