My First Experiences with Kinect for Windows – @KinectWindows

I recently had the pleasure of some free time recently – with three kids this does not come often – so I decided to finally plug my Kinect into my PC. I find it kind of funny how long it actually took for this to happen. Over a year ago I finally joined the real world and made the decision to finally make a new console gaming purchase.

Like everyone, I had to choose between the PS3 and Xbox 360. Each of course has their pros and cons. But when it came down to it, I could develop for the Xbox and Kinect for free and this became the deciding decision factor.

So finally over a year later I finally plugged my Kinect into my PC’s USB port!

The first thing I learned was I couldn’t directly plug my Kinect into my USB port. I actually had to go out and buy a cable for it. The Kinect requires additional power that cannot be provided by a USB port itself. Instead, you need to buy a cable similar to this one. This cable contains a plug that needs to plug into an outlet as well as the Kinect itself and then provides a USB extension that will complete the Kinect’s connection to your PC.

Before continuing, it’s important that you need to install the Kinect SDK before plugging the Kinect into your USB port.

The above link contains options to download the latest Kinect SDK’s as well as the Kinect Developers Toolkit. The toolkit contains some excellent sample examples to allow you too quickly and easily test out several great features of the Kinect.

Now that everything should be setup and running, I decided to jump into some example code. Here are the examples that I started with:

Color Basics

I chose this example for its simplicity. It provided me two pieces of important information:

How to setup the base code to detect and select a Kinect sensor that is connected to the PC.
How to activate the camera on the Kinect and display it on an Image element in a WPF application.

With this ammo, I had a great of understanding of how the basic code structure is setup and used on Windows.

Not much more can be said about this example because there is no real interaction here other than the thrill of seeing yourself on your computer screen through the eyes of the Kinect camera…

Speech Basics

This example I found quite nice and simple again. This time it shows you how to leverage the Kinect’s microphone. The example even goes one step farther and leverages Microsoft’s Speech Recognition Engine to detect words.

By running this example, I learned – what I found an important piece of information – that the speech engine doesn’t work on sentences or conversations. This might not seem important to everyone, but I thought it was significant because I was experimenting with building a system that would “listen” to conversations and act upon certain keywords.

Instead the speech engine seems to be more single-word driven. For example, I need to say “Hello” directly to it. Whereas, if the word “hello” appears in a sentence, the speech engine will not recognize it.

Not the end of the world, but simply changes how you might wish to use the microphone in conjunction with the speech recognition engine.

The other nice code that was demonstrated in the example was how to load the list of grammar words via an XML file and not just via hard-coded words in the application.

Skeleton Basics

Another great example where it begins to use the Skeleton features provided by the Kinect SDK. From what I recall, this example only shows your skeleton and doesn’t use the color camera at the same time.

I found this a little less useful, so I immediately edited the above Color Basics example and display the skeleton on top of the color image.

Once this was setup I had a much better understanding of where the joints in the skeleton are being drawn in correspondence to my actual body.

This was really important to me because it becomes extremely clear right away about how inaccurate the points in the skeleton are. For example, the top of my head did not line up at all when drawn on top of the color video. As I moved around, the updating was a bit inaccurate and could not maintain the updates on screen compared to me moving.

Nonetheless, the example was useful in drawing and seeing my skeleton on screen, but I had to edit the example to use the color video as well.

Avateering

This by far was my favorite example! Before running this example, you need to have the XNA Game Studio 4.0 SDK installed as well.

If you are using Windows 8, you will probably need this helpful blog on MSDN to install Game Studio on Windows 8.

Using the skeleton joints from the example above, this example allows you to control “Dude” – a fully textured 3D model.

Right out of the box it allows you to see how you could leverage this technology to animate any 3D model with your Kinect.

The only thing I really learned with this example was when I let my 3 year old twins control “Dude” that it was not able to detect their legs very much at all. So if you plan to make Kinect games for your young kids, you might have to stick with the “seated” skeleton joints.

Summary

Not being a game developer and understanding the many concepts around it, I really think Kinect development for me probably won’t extend very far. I did have fun dancing around my living room controlling “Dude” and seeing my green skeleton laid on top of a real-time image of myself though!

And…</end thoughts>