Making Things See

Table of contents

1. Making Things See
2. Dedication
3. Preface
   1. Why the Kinect Matters
   2. Who This Book Is For
   3. The Structure of This Book
      1. Working with the Depth Camera
      2. Working with Point Clouds
      3. Working with the Skeleton Data
      4. 3D Scanning for Digital Fabrication
      5. Computer Vision for Robotics
   4. Acknowledgments
   5. Using Code Examples
   6. Conventions Used in This Book
   7. Safari® Books Online
   8. How to Contact Us
4. 1. What Is the Kinect?
   1. How Does It Work? Where Did It Come From?
      1. What Does the Kinect Do?
      2. What’s Inside? How Does It Work?
      3. Who Made the Kinect?
   2. Kinect Artists
      1. Kyle McDonald
      2. Robert Hodgin
      3. Elliot Woods
      4. blablabLAB
      5. Nicolas Burrus
      6. Oliver Kreylos
      7. Alejandro Crawford
      8. Josh Blake
      9. Adafruit
5. 2. Working with the Depth Image
   1. Images and Pixels
   2. Project 1: Installing the SimpleOpenNI Processing Library
      1. Installing OpenNI on OS X
      2. Installing OpenNI on Windows
      3. Installing OpenNI on Linux
      4. Installing the Processing Library
   3. Project 2: Your First Kinect Program
      1. Understanding the Code
   4. Project 3: Looking at a Pixel
      1. Color Pixels
      2. Depth Pixels
   5. Converting to Real-World Distances
   6. Project 4: A Wireless Tape Measure
      1. Higher-Resolution Depth Data
   7. Project 5: Tracking the Nearest Object
      1. Finding the Closest Pixel
      2. Using Variable Scope
   8. Projects
   9. Project 6: Invisible Pencil
   10. Project 7: Minority Report Photos
       1. Basic Version: One Image
       2. Advanced Version: Multiple Images and Scale
   11. Exercises
6. 3. Working with Point Clouds
   1. What You’ll Learn in This Chapter
   2. Welcome to the Third Dimension
   3. Drawing Our First Point Cloud
   4. Making the Point Cloud Move
   5. Viewing the Point Cloud in Color
   6. Making the Point Cloud Interactive
   7. Projects
   8. Project 8: Air Drum Kit
      1. Adding Audio with Minim
      2. Connecting Our First Air Drum
      3. Filling Out Our Air Drum Kit
   9. Project 9: Virtual Kinect
      1. Displaying a 3D Object
      2. 3D Model in the Point Cloud
      3. Variation: Visualizing the Kinect’s Line of Sight
      4. Camera Control with PeasyCam
   10. Conclusion
7. 4. Working with the Skeleton Data
   1. A Note About Calibration
   2. Stages in the Calibration Process
   3. User Detection
   4. Accessing Joint Positions
   5. Skeleton Anatomy Lesson
   6. Measuring the Distance Between Two Joints
   7. Transferring Orientation in 3D
      1. Orientation from a Single Joint
      2. Orientation from Any Vector with Axis-Angle
   8. Background Removal, User Pixels, and the Scene Map
   9. Tracking Without Calibration: Hand Tracking and Center of Mass
      1. Hand Tracking
      2. Center of Mass
   10. Projects
   11. Project 10: Exercise Measurement
       1. The SkeletonRecorder Class
       2. Exercises
   12. Project 11: “Stayin’ Alive” Dance Move Triggers MP3
       1. Describing a Pose as Joint Relationships
       2. Basic Version
       3. Advanced Version
       4. The SkeletonPoser and PoseRule Classes
       5. Exercises
   13. Conclusion
8. 5. Scanning for Fabrication
   1. Scan and Print: Rapid Prototyping Objects
      1. Scan and Print: Rapid Prototyping Objects
      2. From a Point Cloud to an Object
      3. Outline of This Chapter
   2. Intro to Modelbuilder
   3. Intro to MeshLab
      1. Your First Visit to MeshLab
   4. Making a Mesh from the Kinect Data
   5. Looking at Our First Scan
   6. Cleaning Up the Mesh
   7. Looking at Our Corrected Model
   8. Prepping for Printing
   9. Reduce Polygons in MeshLab
   10. Printing Our Model on a MakerBot
       1. Printing Our Scan
   11. Sending Our Model to Shapeways
   12. Conclusion: Comparing Prints
9. 6. Using the Kinect for Robotics
   1. Forward Kinematics
      1. Calculating the Angles of Limbs
      2. Getting Started with Arduino: The Brain of Our Robot Arm
      3. Acquiring the Arduino and the Servos
      4. Plugging in the Servos
      5. Downloading the Arduino Development Environment
      6. Programming the Arduino
      7. Testing Our Arduino Program: Serial in Processing
      8. Building Our Robot Arm
      9. Putting It All Together: Connecting Our Robot Arm to Our Processing Sketch
   2. Inverse Kinematics
      1. Inverse Kinematics Calculations without the Kinect
      2. Controlling Our Robot Arm with Inverse Kinematics
      3. Updating Our Hardware
      4. Updating Our Code
   3. Conclusion
10. 7. Conclusion: What’s Next?
    1. Beyond Processing: Other Frameworks and Languages
       1. Microsoft
       2. C++ Creative Coding Frameworks: openFrameworks and Cinder
       3. Data Flow: MAX/MSP and PureData
       4. Game Engines: Unity3D
    2. Topics in 3D Programming to Explore
       1. Graphics Programming: OpenGL and Shaders
       2. Computer Vision: OpenCV, Feature Tracking, and Beyond
       3. 3D Modeling and Animation
    3. Ideas for Projects
       1. A Physical Therapy Game
       2. Kinect Stereo Viewer
       3. Animated 3D Puppet
       4. Rolling Person-Chasing Robot
11. A. Appendix
    1. SimpleOpenNI Cheat Sheet
    2. Chapter 2
       1. Closest Pixel with Running Average
       2. Limiting the Depth Range
    3. Chapter 4
       1. Save Calibration
       2. Load Calibration
       3. Extended SkeletonRecorder That Can Track Multiple Joints
    4. Chapter 6
       1. Complete Forward Kinematics Example
       2. Inverse Kinematics Geometrical Proof
12. Index
13. About the Author
14. Colophon
15. Copyright