Making Things See

Making Things See

by Greg Borenstein
Released January 2012
Publisher(s): Make: Community
ISBN: 9781449327781

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

This detailed, hands-on guide provides the technical and conceptual information you need to build cool applications with Microsoft’s Kinect, the amazing motion-sensing device that enables computers to see. Through half a dozen meaty projects, you’ll learn how to create gestural interfaces for software, use motion capture for easy 3D character animation, 3D scanning for custom fabrication, and many other applications.

Perfect for hobbyists, makers, artists, and gamers, Making Things See shows you how to build every project with inexpensive off-the-shelf components, including the open source Processing programming language and the Arduino microcontroller. You’ll learn basic skills that will enable you to pursue your own creative applications with Kinect.

  • Create Kinect applications on Mac OS X, Windows, or Linux
  • Track people with pose detection and skeletonization, and use blob tracking to detect objects
  • Analyze and manipulate point clouds
  • Make models for design and fabrication, using 3D scanning technology
  • Use MakerBot, RepRap, or Shapeways to print 3D objects
  • Delve into motion tracking for animation and games
  • Build a simple robot arm that can imitate your arm movements
  • Discover how skilled artists have used Kinect to build fascinating projects

Table of contents

  1. Making Things See
  2. Dedication
  3. Preface
    1. Why the Kinect Matters
    2. Who This Book Is For
    3. The Structure of This Book
      1. Working with the Depth Camera
      2. Working with Point Clouds
      3. Working with the Skeleton Data
      4. 3D Scanning for Digital Fabrication
      5. Computer Vision for Robotics
    4. Acknowledgments
    5. Using Code Examples
    6. Conventions Used in This Book
    7. Safari® Books Online
    8. How to Contact Us
  4. 1. What Is the Kinect?
    1. How Does It Work? Where Did It Come From?
      1. What Does the Kinect Do?
      2. What’s Inside? How Does It Work?
      3. Who Made the Kinect?
    2. Kinect Artists
      1. Kyle McDonald
      2. Robert Hodgin
      3. Elliot Woods
      4. blablabLAB
      5. Nicolas Burrus
      6. Oliver Kreylos
      7. Alejandro Crawford
      8. Josh Blake
      9. Adafruit
  5. 2. Working with the Depth Image
    1. Images and Pixels
    2. Project 1: Installing the SimpleOpenNI Processing Library
      1. Installing OpenNI on OS X
      2. Installing OpenNI on Windows
      3. Installing OpenNI on Linux
      4. Installing the Processing Library
    3. Project 2: Your First Kinect Program
      1. Understanding the Code
    4. Project 3: Looking at a Pixel
      1. Color Pixels
      2. Depth Pixels
    5. Converting to Real-World Distances
    6. Project 4: A Wireless Tape Measure
      1. Higher-Resolution Depth Data
    7. Project 5: Tracking the Nearest Object
      1. Finding the Closest Pixel
      2. Using Variable Scope
    8. Projects
    9. Project 6: Invisible Pencil
    10. Project 7: Minority Report Photos
      1. Basic Version: One Image
      2. Advanced Version: Multiple Images and Scale
    11. Exercises
  6. 3. Working with Point Clouds
    1. What You’ll Learn in This Chapter
    2. Welcome to the Third Dimension
    3. Drawing Our First Point Cloud
    4. Making the Point Cloud Move
    5. Viewing the Point Cloud in Color
    6. Making the Point Cloud Interactive
    7. Projects
    8. Project 8: Air Drum Kit
      1. Adding Audio with Minim
      2. Connecting Our First Air Drum
      3. Filling Out Our Air Drum Kit
    9. Project 9: Virtual Kinect
      1. Displaying a 3D Object
      2. 3D Model in the Point Cloud
      3. Variation: Visualizing the Kinect’s Line of Sight
      4. Camera Control with PeasyCam
    10. Conclusion
  7. 4. Working with the Skeleton Data
    1. A Note About Calibration
    2. Stages in the Calibration Process
    3. User Detection
    4. Accessing Joint Positions
    5. Skeleton Anatomy Lesson
    6. Measuring the Distance Between Two Joints
    7. Transferring Orientation in 3D
      1. Orientation from a Single Joint
      2. Orientation from Any Vector with Axis-Angle
    8. Background Removal, User Pixels, and the Scene Map
    9. Tracking Without Calibration: Hand Tracking and Center of Mass
      1. Hand Tracking
      2. Center of Mass
    10. Projects
    11. Project 10: Exercise Measurement
      1. The SkeletonRecorder Class
      2. Exercises
    12. Project 11: “Stayin’ Alive” Dance Move Triggers MP3
      1. Describing a Pose as Joint Relationships
      2. Basic Version
      3. Advanced Version
      4. The SkeletonPoser and PoseRule Classes
      5. Exercises
    13. Conclusion
  8. 5. Scanning for Fabrication
    1. Scan and Print: Rapid Prototyping Objects
      1. Scan and Print: Rapid Prototyping Objects
      2. From a Point Cloud to an Object
      3. Outline of This Chapter
    2. Intro to Modelbuilder
    3. Intro to MeshLab
      1. Your First Visit to MeshLab
    4. Making a Mesh from the Kinect Data
    5. Looking at Our First Scan
    6. Cleaning Up the Mesh
    7. Looking at Our Corrected Model
    8. Prepping for Printing
    9. Reduce Polygons in MeshLab
    10. Printing Our Model on a MakerBot
      1. Printing Our Scan
    11. Sending Our Model to Shapeways
    12. Conclusion: Comparing Prints
  9. 6. Using the Kinect for Robotics
    1. Forward Kinematics
      1. Calculating the Angles of Limbs
      2. Getting Started with Arduino: The Brain of Our Robot Arm
      3. Acquiring the Arduino and the Servos
      4. Plugging in the Servos
      5. Downloading the Arduino Development Environment
      6. Programming the Arduino
      7. Testing Our Arduino Program: Serial in Processing
      8. Building Our Robot Arm
      9. Putting It All Together: Connecting Our Robot Arm to Our Processing Sketch
    2. Inverse Kinematics
      1. Inverse Kinematics Calculations without the Kinect
      2. Controlling Our Robot Arm with Inverse Kinematics
      3. Updating Our Hardware
      4. Updating Our Code
    3. Conclusion
  10. 7. Conclusion: What’s Next?
    1. Beyond Processing: Other Frameworks and Languages
      1. Microsoft
      2. C++ Creative Coding Frameworks: openFrameworks and Cinder
      3. Data Flow: MAX/MSP and PureData
      4. Game Engines: Unity3D
    2. Topics in 3D Programming to Explore
      1. Graphics Programming: OpenGL and Shaders
      2. Computer Vision: OpenCV, Feature Tracking, and Beyond
      3. 3D Modeling and Animation
    3. Ideas for Projects
      1. A Physical Therapy Game
      2. Kinect Stereo Viewer
      3. Animated 3D Puppet
      4. Rolling Person-Chasing Robot
  11. A. Appendix
    1. SimpleOpenNI Cheat Sheet
    2. Chapter 2
      1. Closest Pixel with Running Average
      2. Limiting the Depth Range
    3. Chapter 4
      1. Save Calibration
      2. Load Calibration
      3. Extended SkeletonRecorder That Can Track Multiple Joints
    4. Chapter 6
      1. Complete Forward Kinematics Example
      2. Inverse Kinematics Geometrical Proof
  12. Index
  13. About the Author
  14. Colophon
  15. Copyright

Product information

  • Title: Making Things See
  • Author(s): Greg Borenstein
  • Release date: January 2012
  • Publisher(s): Make: Community
  • ISBN: 9781449327781