book

Start Here!™ Learn Microsoft® Kinect API

by Rob Miles

July 2012

Beginner to intermediate

272 pages

7h 17m

English

Microsoft Press

Read now

Unlock full access

Who Should Read This BookAssumptionsWho Should Not Read This BookOrganization of This BookConventions and Features in This BookSystem RequirementsCode SamplesInstalling the Code SamplesUsing the Code SamplesAcknowledgmentsErrata and Book SupportWe Want to Hear from YouStay in Touch
The Kinect SensorGetting Inside a Kinect SensorThe Depth SensorThe Kinect MicrophonesRecognizing People with KinectProgramming the KinectKinect for Xbox and Kinect for WindowsSummary
Kinect for Windows SDK PrerequisitesKinect DeviceVisual StudioDirectX StudioInstalling the Kinect for Windows SDKInstalling the Kinect SDKInstalling the Kinect SDKInstalling the Kinect SDKConnecting the Kinect Sensor BarPowering the Kinect SensorInstalling the Kinect Sensor USB DriversTesting the Kinect Sensor BarThe Kinect SDK Sample BrowserTroubleshooting Your Kinect InstallationRemove Old SDK InstallationsEnsure That Visual Studio 2010 Is Installed but Not Running During InstallationEnsure That There Are No Windows Updates in ProgressEnsure That the Kinect Is Powered CorrectlyRemove Any Old USB DriversSummary
Making a Kinect Video CameraCreating a New Visual Studio Project for KinectCreating a new Kinect ProgramCreating a new Kinect ProgramGetting the Kinect Sensor WorkingUsing the KinectSensor Class to Control the Kinect SensorUsing the KinectSensor Class to Control the Kinect SensorDisplaying a Video FrameCreating a WPF Image Display Element for the Kinect CameraAdding an Image Display Element to the WPF WindowDisplaying the Kinect Camera PictureObtaining the Kinect Color Camera Video DataTesting the Kinect CameraAdding Error HandlingDetecting the Presence of a KinectHandling Setup ErrorsSummary
Image Storage in ComputersGetting the Kinect Image Data onto the ScreenControlling the Color of the PixelsCreating a Color Adjustment ProgramImproving the Speed by Writing Unsafe CodeManaged Code and Unsafe CodeReferences and PointersPointers and Fixed Memory LocationsUsing 32-Bit Integer PointersSaving the Image to a FileImproving Video QualityImproving Performance by Waiting for Each Kinect FrameCreating a Video Display ThreadUpdating the Image from a Different ThreadStopping the Background ThreadSummary
Detecting Movement in Video ImagesStoring a Video Image in Program MemoryDetecting Changes in Video ImagesDealing with NoiseUsing Change Thresholds to Filter Out NoiseSounding the AlarmAdding Sound Playback to a ProjectAdding Sound Playback to a ProjectA Complete Alarm ProgramSwitching to Black and WhiteTriggering Pictures with Motion CaptureCapturing Multiple FramesSummary
Visualizing Kinect Depth InformationThe Kinect Depth SensorObtaining Depth Information from the SensorDepth Data ValuesVisualizing Depth InformationUsing the Depth Information to Detect IntrudersUsing the Depth and Video Sensors at the Same TimeDrawing in the AirDetecting ObjectsCounting Depth ValuesMaking You into the ControllerUsing the Kinect Sensor with an XNA GameGetting the Control Value from the KinectDrawing the Kinect Depth Image in XNASummary

Capturing Sound Using KinectSound and ComputersReceiving Sound Signals from KinectPlaying Sound Using XNAStopping the ProgramSound Signals and LatencyVisualizing a Sound Signal in XNAStoring Sound Data in a File and Replaying ItCreating a WAV FilePlaying a Recorded SoundTriggering the Sound PlaybackSound Playback ManagementSummary
Kinect Body TrackingKinect Skeleton InformationKinect Joint PositionsA Head Tracking ProgramThe Joints Collection and C# DictionariesUsing Format Strings to Build a MessageSkeleton Information QualityJoint Tracking StateDrawing a SkeletonDrawing Lines in WPFConverting Joint Positions to Image CoordinatesClearing the CanvasTurning Yourself into a PaintbrushDrawing a Complete SkeletonDetecting GesturesCalculating the Distance Between Two Points in SpaceUsing a Gesture to Trigger an ActionBiometric Recognition with KinectCreating a “Kiss-Detecting” ProgramFinding Two Skeletons That Are Being TrackedSummary
Using the Microsoft Speech PlatformTesting Voice RecognitionCreating a Program That Recognizes Color NamesAdding the Speech Platform SDK Assemblies to a ProjectCreating a Speech Recognition EngineBuilding the CommandsCreating a GrammarGetting Audio into the Speech RecognizerResponding to Recognized WordsCreating a Voice-Controlled Painting ProgramSpeech CommandsDrawing a Skeleton CursorDrawing Using the Artist’s HandSaving the Drawing Canvas to a FileRendering a Canvas to a BitmapSaving a Bitmap into a Named FileTidying Up When the Program EndsImproving the Drawing ProgramAdding Speech Output to ProgramsFeedback ProblemsSummary
An Augmented-Reality GameCreating SpritesCreating Sprite GraphicsAdding Images to a ProjectDrawing Sprite ImagesSetting the Sprite PositionMaking a Sprite MovePutting the Bug Back on the TopCreating Augmented RealityDisplaying Computer Graphics on Top of the Video ImageAdding a Mallet for the PlayerMatching Together Screen and Depth CoordinatesDrawing the MalletSplatting BugsIsolating the Player Image from the BackgroundUsing a Display MaskFinding the Player PixelsUsing Depth Information to Make a MaskDrawing the Mask ImageSetting up the Game ImagePutting the Whole Game TogetherThe Kinect Manager ClassCreating a KinectManager InstanceUsing Kinect DataDisplaying the Kinect StatusPolling the SensorStarting and Stopping the KinectManager SensorUsing the KinectManager Class in Your ProgramsImproving the GameSummary
Controlling MIDI Devices with KinectThe MIDI ProtocolCreating a Class to Manage a MIDI ConnectionUsing MIDI in Windows PC ProgramsConstructing a MIDI Connection ClassCreating a MIDIControl InstanceCreating MIDI MessagesThe MIDI Command ByteThe Note ValueThe Velocity ValueSending MIDI MessagesPlaying MIDI NotesReleasing MIDI NotesCreating the MIDI ConnectionSending Note MessagesMaking a Multi-Note PianoCreating the Keyboard Display ElementsResponding to Keyboard EventsPlaying a Proper ScaleCreating a Human MIDI KeyboardCreating a List of MIDI Note KeysControlling Note PlaybackUsing the Player Position to Control NotesDeveloping the MIDI ProgramUsing the Kinect with a Serial PortSerial Ports and DevicesSerial Ports and DevicesThe Robot SlaveLinking a Kinect Program to a Serial PortCreating a Serial Port ConnectionSending Messages to the Robot Using the Serial PortCreating Command Touch AreasReceiving Messages from the Robot Using the Serial PortSummary
Adjusting the Sensor AngleUsing Kinect to Track Multiple PeopleIdentifying Particular People in a SceneCombining Skeleton and Person Depth InformationSound Location with the Kinect Microphone ArrayUsing Kinect with the Microsoft Robotics Development StudioMobile Autonomous Reference Using KinectEmulating a Robot EnvironmentRobots and Kinect in the FutureTaking Kinect FurtherMount the Sensor in Different OrientationsUse Multiple SensorsMove the Sensor AroundUse Skeleton Tracking to Measure ThingsInvestigate TransformSmoothParametersUse Voice Response to Do AnythingHave Fun Playing with VideoMake More of MIDIGood Luck and Have Fun!Summary

Overview

Ready to learn Kinect programming? Start Here!

Learn the fundamentals of programming with the Kinect API—and begin building apps that use motion tracking, voice recognition, and more. If you have experience programming with C#—simply start here! This book introduces must-know concepts and techniques through easy-to-follow explanations, examples, and exercises.

Here’s where you start learning Kinect

Build an application to display Kinect video on your PC

Have Kinect take photographs when it detects movement

Draw on a computer screen by moving your finger in the air

Track your body gestures and use them to control a program

Make a program that understands your speech and talks back to you

Play a part in your own augmented reality game

Create an "air piano" using Kinect with a MIDI device