book

Start Here!™ Learn Microsoft® Kinect API

Name: Start Here!™ Learn Microsoft® Kinect API
Author: Rob Miles
ISBN: 9780735673984

by Rob Miles

July 2012

Beginner to intermediate

272 pages

7h 17m

English

Microsoft Press

Read now

Unlock full access

Dedication
Introduction
Who Should Read This BookAssumptionsWho Should Not Read This BookOrganization of This BookConventions and Features in This BookSystem RequirementsCode SamplesInstalling the Code SamplesUsing the Code SamplesAcknowledgmentsErrata and Book SupportWe Want to Hear from YouStay in Touch
I. Getting Started
1. An Introduction to Kinect
The Kinect SensorGetting Inside a Kinect SensorThe Depth SensorThe Kinect MicrophonesRecognizing People with KinectProgramming the KinectKinect for Xbox and Kinect for WindowsSummary
2. Getting Started with Kinect
Kinect for Windows SDK PrerequisitesKinect DeviceVisual StudioDirectX StudioInstalling the Kinect for Windows SDKInstalling the Kinect SDKInstalling the Kinect SDKInstalling the Kinect SDKConnecting the Kinect Sensor BarPowering the Kinect SensorInstalling the Kinect Sensor USB DriversTesting the Kinect Sensor BarThe Kinect SDK Sample BrowserTroubleshooting Your Kinect InstallationRemove Old SDK InstallationsEnsure That Visual Studio 2010 Is Installed but Not Running During InstallationEnsure That There Are No Windows Updates in ProgressEnsure That the Kinect Is Powered CorrectlyRemove Any Old USB DriversSummary
3. Writing Software for Kinect
Making a Kinect Video CameraCreating a New Visual Studio Project for KinectCreating a new Kinect ProgramCreating a new Kinect ProgramGetting the Kinect Sensor WorkingUsing the KinectSensor Class to Control the Kinect SensorUsing the KinectSensor Class to Control the Kinect SensorDisplaying a Video FrameCreating a WPF Image Display Element for the Kinect CameraAdding an Image Display Element to the WPF WindowDisplaying the Kinect Camera PictureObtaining the Kinect Color Camera Video DataTesting the Kinect CameraAdding Error HandlingDetecting the Presence of a KinectHandling Setup ErrorsSummary
II. Using the Kinect Sensor
4. Your First Kinect Application—Video Snapshots
Image Storage in ComputersGetting the Kinect Image Data onto the ScreenControlling the Color of the PixelsCreating a Color Adjustment ProgramImproving the Speed by Writing Unsafe CodeManaged Code and Unsafe CodeReferences and PointersPointers and Fixed Memory LocationsUsing 32-Bit Integer PointersSaving the Image to a FileImproving Video QualityImproving Performance by Waiting for Each Kinect FrameCreating a Video Display ThreadUpdating the Image from a Different ThreadStopping the Background ThreadSummary
5. Moving Pictures
Detecting Movement in Video ImagesStoring a Video Image in Program MemoryDetecting Changes in Video ImagesDealing with NoiseUsing Change Thresholds to Filter Out NoiseSounding the AlarmAdding Sound Playback to a ProjectAdding Sound Playback to a ProjectA Complete Alarm ProgramSwitching to Black and WhiteTriggering Pictures with Motion CaptureCapturing Multiple FramesSummary
6. Fun with the Depth Sensor
Visualizing Kinect Depth InformationThe Kinect Depth SensorObtaining Depth Information from the SensorDepth Data ValuesVisualizing Depth InformationUsing the Depth Information to Detect IntrudersUsing the Depth and Video Sensors at the Same TimeDrawing in the AirDetecting ObjectsCounting Depth ValuesMaking You into the ControllerUsing the Kinect Sensor with an XNA GameGetting the Control Value from the KinectDrawing the Kinect Depth Image in XNASummary

7. Fun with the Sound Sensor
Capturing Sound Using KinectSound and ComputersReceiving Sound Signals from KinectPlaying Sound Using XNAStopping the ProgramSound Signals and LatencyVisualizing a Sound Signal in XNAStoring Sound Data in a File and Replaying ItCreating a WAV FilePlaying a Recorded SoundTriggering the Sound PlaybackSound Playback ManagementSummary
III. Creating Advanced User Interfaces
8. Body Tracking with Kinect
Kinect Body TrackingKinect Skeleton InformationKinect Joint PositionsA Head Tracking ProgramThe Joints Collection and C# DictionariesUsing Format Strings to Build a MessageSkeleton Information QualityJoint Tracking StateDrawing a SkeletonDrawing Lines in WPFConverting Joint Positions to Image CoordinatesClearing the CanvasTurning Yourself into a PaintbrushDrawing a Complete SkeletonDetecting GesturesCalculating the Distance Between Two Points in SpaceUsing a Gesture to Trigger an ActionBiometric Recognition with KinectCreating a “Kiss-Detecting” ProgramFinding Two Skeletons That Are Being TrackedSummary
9. Voice Control with Kinect
Using the Microsoft Speech PlatformTesting Voice RecognitionCreating a Program That Recognizes Color NamesAdding the Speech Platform SDK Assemblies to a ProjectCreating a Speech Recognition EngineBuilding the CommandsCreating a GrammarGetting Audio into the Speech RecognizerResponding to Recognized WordsCreating a Voice-Controlled Painting ProgramSpeech CommandsDrawing a Skeleton CursorDrawing Using the Artist’s HandSaving the Drawing Canvas to a FileRendering a Canvas to a BitmapSaving a Bitmap into a Named FileTidying Up When the Program EndsImproving the Drawing ProgramAdding Speech Output to ProgramsFeedback ProblemsSummary
10. Augmented Reality with Kinect
An Augmented-Reality GameCreating SpritesCreating Sprite GraphicsAdding Images to a ProjectDrawing Sprite ImagesSetting the Sprite PositionMaking a Sprite MovePutting the Bug Back on the TopCreating Augmented RealityDisplaying Computer Graphics on Top of the Video ImageAdding a Mallet for the PlayerMatching Together Screen and Depth CoordinatesDrawing the MalletSplatting BugsIsolating the Player Image from the BackgroundUsing a Display MaskFinding the Player PixelsUsing Depth Information to Make a MaskDrawing the Mask ImageSetting up the Game ImagePutting the Whole Game TogetherThe Kinect Manager ClassCreating a KinectManager InstanceUsing Kinect DataDisplaying the Kinect StatusPolling the SensorStarting and Stopping the KinectManager SensorUsing the KinectManager Class in Your ProgramsImproving the GameSummary
VI. Kinect in the Real World
11. Real-World Control with Kinect
Controlling MIDI Devices with KinectThe MIDI ProtocolCreating a Class to Manage a MIDI ConnectionUsing MIDI in Windows PC ProgramsConstructing a MIDI Connection ClassCreating a MIDIControl InstanceCreating MIDI MessagesThe MIDI Command ByteThe Note ValueThe Velocity ValueSending MIDI MessagesPlaying MIDI NotesReleasing MIDI NotesCreating the MIDI ConnectionSending Note MessagesMaking a Multi-Note PianoCreating the Keyboard Display ElementsResponding to Keyboard EventsPlaying a Proper ScaleCreating a Human MIDI KeyboardCreating a List of MIDI Note KeysControlling Note PlaybackUsing the Player Position to Control NotesDeveloping the MIDI ProgramUsing the Kinect with a Serial PortSerial Ports and DevicesSerial Ports and DevicesThe Robot SlaveLinking a Kinect Program to a Serial PortCreating a Serial Port ConnectionSending Messages to the Robot Using the Serial PortCreating Command Touch AreasReceiving Messages from the Robot Using the Serial PortSummary
12. Taking Kinect Further
Adjusting the Sensor AngleUsing Kinect to Track Multiple PeopleIdentifying Particular People in a SceneCombining Skeleton and Person Depth InformationSound Location with the Kinect Microphone ArrayUsing Kinect with the Microsoft Robotics Development StudioMobile Autonomous Reference Using KinectEmulating a Robot EnvironmentRobots and Kinect in the FutureTaking Kinect FurtherMount the Sensor in Different OrientationsUse Multiple SensorsMove the Sensor AroundUse Skeleton Tracking to Measure ThingsInvestigate TransformSmoothParametersUse Voice Response to Do AnythingHave Fun Playing with VideoMake More of MIDIGood Luck and Have Fun!Summary
Index
About the Author
Copyright

Overview

Ready to learn Kinect programming? Start Here!

Learn the fundamentals of programming with the Kinect API—and begin building apps that use motion tracking, voice recognition, and more. If you have experience programming with C#—simply start here! This book introduces must-know concepts and techniques through easy-to-follow explanations, examples, and exercises.

Here’s where you start learning Kinect

Build an application to display Kinect video on your PC

Have Kinect take photographs when it detects movement

Draw on a computer screen by moving your finger in the air

Track your body gestures and use them to control a program

Make a program that understands your speech and talks back to you

Play a part in your own augmented reality game

Create an "air piano" using Kinect with a MIDI device

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Beginning Kinect Programming with the Microsoft Kinect SDK

Publisher Resources

ISBN: 9780735673984Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills