book

Parallel R

by Q. Ethan McCallum, Stephen Weston

October 2011

Intermediate to advanced

126 pages

3h 10m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Conventions Used in This BookUsing Code ExamplesSafari® Books OnlineHow to Contact UsAcknowledgmentsQ. Ethan McCallumStephen Weston
Why R?Why Not R?The Solution: Parallel ExecutionA Road Map for This BookWhat We’ll CoverLooking Forward…What We’ll Assume You Already KnowIn a Hurry?snowmulticoreparallelR+HadoopRHIPESegueSummary
Quick LookHow It WorksSetting UpWorking with ItCreating Clusters with makeClusterParallel K-MeansInitializing WorkersLoad Balancing with clusterApplyLBTask Chunking with parLapplyVectorizing with clusterSplitLoad Balancing ReduxFunctions and EnvironmentsRandom Number Generationsnow ConfigurationInstalling RmpiExecuting snow Programs on a Cluster with RmpiExecuting snow Programs with a Batch Queueing SystemTroubleshooting snow ProgramsWhen It Works……And When It Doesn’tThe Wrap-up
Quick LookHow It WorksSetting UpWorking with ItThe mclapply FunctionThe mc.cores OptionThe mc.set.seed OptionLoad Balancing with mclapplyThe pvec FunctionThe parallel and collect FunctionsUsing collect OptionsParallel Random Number GenerationThe Low-Level APIWhen It Works……And When It Doesn’tThe Wrap-up
Quick LookHow It WorksSetting UpWorking with ItGetting StartedCreating Clusters with makeClusterParallel Random Number GenerationSummary of DifferencesWhen It Works……And When It Doesn’tThe Wrap-up
Hadoop at Cruising AltitudeA MapReduce PrimerThinking in MapReduce: Some Pseudocode ExamplesCalculate Average Call Length for Each DateNumber of Calls by Each User, on Each DateRun a Special Algorithm on Each RecordBinary and Whole-File Data: SequenceFilesNo Cluster? No Problem! Look to the Clouds…The Wrap-up
Quick LookHow It WorksSetting UpWorking with ItSimple Hadoop Streaming (All Text)Streaming, Redux: Indirectly Working with Binary DataThe Java API: Binary Input and OutputProcessing Related Groups (the Full Map and Reduce Phases)When It Works……And When It Doesn’tThe Wrap-up

Quick LookHow It WorksSetting UpWorking with ItPhone Call Records, ReduxTweet BrevityMore Complex Tweet AnalysisWhen It Works……And When It Doesn’tThe Wrap-up
Quick LookHow It WorksSetting UpWorking with ItModel Testing: Parameter SweepWhen It Works……And When It Doesn’tThe Wrap-up
doRedisRevoScale R and RevoConnectR (RHadoop)cloudNumbers.com

Content preview from Parallel R

About the Authors

Q Ethan McCallum is a consultant, writer, and technology enthusiast, though perhaps not in that order. His work has appeared online on The O’Reilly Network and Java.net, and also in print publications such as C/C++ Users Journal, Doctor Dobb’s Journal, and Linux Magazine. In his professional roles, he helps companies to make smart decisions about data and technology.

Stephen Weston has been working in high performance and parallelcomputing for over 25 years. He was employed at Scientific Computing Associates in the 90's, working on the Linda programming system, invented by David Gelernter. He was also a founder of Revolution Computing, leading the development of parallel computing packages for R, including nws, foreach, doSNOW, and doMC. He works at Yale University as an HPC Specialist.