O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Raspberry Pi Super Cluster

Book Description

As a Raspberry Pi enthusiast have you ever considered increasing their performance with parallel computing? Discover just how easy it can be with the right help – this guide takes you through the process from start to finish.

  • Learn about parallel computing by building your own system using Raspberry Pi
  • Build a two-node parallel computing cluster
  • Integrate Raspberry Pi with Hadoop to build your own super cluster

In Detail

A cluster is a type of parallel/distributed processing system which consists of a collection of interconnected stand-alone computers cooperatively working together. Using Raspberry Pi computers, you can build a two-node parallel computing cluster which enhances performance and availability.

This practical, example-oriented guide will teach you how to set up the hardware and operating systems of multiple Raspberry Pi computers to create your own cluster. It will then navigate you through how to install the necessary software to write your own programs such as Hadoop and MPICH before moving on to cover topics such as MapReduce. Throughout this book, you will explore the technology with the help of practical examples and tutorials to help you learn quickly and efficiently.

Starting from a pile of hardware, with this book, you will be guided through exciting tutorials that will help you turn your hardware into your own super-computing cluster. You’ll start out by learning how to set up your Raspberry Pi cluster’s hardware. Following this, you will be taken through how to install the operating system, and you will also be given a taste of what parallel computing is about. With your Raspberry Pi cluster successfully set up, you will then install software such as MPI and Hadoop. Having reviewed some examples and written some programs that explore these two technologies, you will then wrap up with some fun ancillary projects. Finally, you will be provided with useful links to help take your projects to the next step.

Table of Contents

  1. Raspberry Pi Super Cluster
    1. Table of Contents
    2. Raspberry Pi Super Cluster
    3. Credits
    4. About the Author
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers and more
        1. Why Subscribe?
        2. Free Access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    8. 1. Clusters, Parallel Computing, and Raspberry Pi – A Brief Background
      1. A very short history of parallel computing
        1. Supercomputers
        2. Multi-core and multiprocessor machines
        3. Commodity hardware clusters
        4. Cloud computing
        5. Big data
      2. Raspberry Pi and parallel computing
        1. Programming languages and frameworks
      3. Summary
    9. 2. Setting Up your Raspberry Pi Software and Hardware for Parallel Computing
      1. Setting up our work environment
        1. HDMI-capable monitor or VGA/DVI monitor and adapter
        2. USB keyboard and mouse
        3. Two micro-USB power units
        4. A desk-mounted power strip with both USB and mains outlets (optional)
        5. Three Ethernet/RJ45 network cables
        6. A small network switch
        7. An existing Internet connection
        8. Two SD cards that are compatible with the Raspberry Pi
        9. Housing units for the Raspberry Pi boards and Lego (optional)
        10. USB hard drives (optional)
      2. Future expansion and a scalable setup
      3. Completing the initial setup
      4. Using an SD card as our Raspberry Pi's storage device
        1. SD card setup
        2. Formatting our card
          1. Mac OS X SD card formatting instructions
          2. Windows 8 SD card formatting instructions
          3. Linux instructions for SD card formatting
      5. BerryBoot version 2
        1. Downloading the BerryBoot version 2 ZIP file
          1. Mac OS X
          2. Windows 8
          3. Linux
      6. Starting up the Raspberry Pi
        1. The installation process
        2. Installation complete
        3. Testing SSH and setting up keys
        4. Connecting via SSH
          1. Mac OS X and Linux users
          2. Windows 8 users with PuTTY
            1. SSH running successfully
        5. Setting up your SSH RSA keys
          1. The ssh-agent and ssh-add tools
          2. SSH setup complete
      7. Wrapping up
        1. Editing text files on Raspbian
        2. Installing Fortran
        3. Terminal multiplexing with Screen
      8. Summary
    10. 3. Parallel Computing – MPI on the Raspberry Pi
      1. MPI – Message Passing Interface
      2. MPI implementations – MPICH and OpenMPI
      3. Creating an environment and downloading MPICH
        1. Building and installing MPICH
        2. Configuring your Raspberry Pi to run with MPICH
        3. Testing our MPICH installation
        4. Building our second Raspberry Pi
          1. Windows 8
          2. Mac OS X
          3. Linux
        5. Powering up the second Raspberry Pi
        6. RSA key setup for SSH
      4. Writing an MPI-based application
        1. MPI – point-to-point communication
      5. Summary
    11. 4. Hadoop – Distributed Applications on the Raspberry Pi
      1. A brief introduction to Apache Hadoop
      2. Installing Java
      3. Installing Apache Hadoop
        1. Hadoop configuration
        2. Testing our Hadoop server
        3. Setting up our second Raspberry Pi
      4. Summary
    12. 5. MapReduce Applications with Hadoop and Java
      1. MapReduce
      2. MapReduce in Hadoop
        1. HDFS – The Hadoop distributed file system
      3. The WordCount MapReduce program
      4. Testing our application
      5. Summary
    13. 6. Calculate Pi with Hadoop and MPI
      1. Monte Carlo simulators
      2. A Hadoop application to calculate Pi
      3. Pi with C language and MPI
      4. Summary
    14. 7. Going Further
      1. Booting from an external USB HDD
      2. Building a Lego enclosure
      3. Experimenting with MPI and Fortran
      4. Power for multiple devices
        1. USB wall plates
        2. Battery power
        3. Using a PC power supply
        4. Power over Ethernet
      5. Summary
    15. A. Appendix
      1. Fortran and C/C++
      2. MPI, Hadoop, and parallel computing
      3. Raspberry Pi cases and clusters
    16. Index