Sensor: KINECT

kinect

Kinect is a motion sensing input device originally created to play with Microsoft’s Xbox 360 video game console. It is actually many sensors in one. It contains a RGB camera, an infrared (IR) emitter, an IR depth sensor and a microphone array as well as a software pipeline that processes color, depth, and skeleton data.



COMPONENT DETAILS

kinect-comp

1. IR emitter: Projects a pattern of infrared light into a room. As the light hits a surface, the pattern becomes distorted, and the distortion is read by the depth camera.

2. IR depth camera: Analyzes IR patterns to build a 3-D map of the room and all objects and people within it, up to 4.0 m.

3. RGB camera: Like a webcam, this captures a 640×480 RGB color video image. This information can be used to get details about objects and people in the room.

4. Microphone array: Four microphones returning audio can be used to detect where voices or sounds are coming from.

5. Tilt motor: This can be used to adjust the angle of the device based on the object in front of it.

6. USB cable: Transmits data via an unencrypted feed, which makes it relatively easy to use with other devices than the Xbox, like a regular PC or Mac computer.

The following table shows the device’s specifications.

kinect_specs

And here you can see a disassembled Kinect to know its various components and parts:

kinect-parts



APPLICATIONS

Although its intended purpose is for playing videogames, many developers have been researching other possible applications of the system that make use of its ability to map and undestand its environment and users.

Researchers and artists started developing projects and posting the results online since the week the device was released on November 2010. Today it is widely used in artistic installations, publicity kiosks, robotics and even medicine, allowing to guide imaging during surgery.

None of these kinds of projects are sanctioned by Microsoft but you have to install drivers and libraries in order to make it work properly, using some algorithms that help the machine recognize individual bodies and track motion.



ALGORITHMS USED IN OUR PROJECT

kinect-void

In our final project “Void”, we used the position of the user to generate different visual effects in space. We used Processing as a programming language enhancing its capabilities with SimpleOpenNI library.
The main variable we tracked was the distance of the user from the sensor, making use of the depth sensor in the Kinect. We used the numerical value of this variable (VarZ) to modify the particles in the screen over time, mapping it into new values.

The following code changes the speed of stars using 1500 as a reference value for VarZ:

if (VarZ<1500) { stateX = map(VarZ, 0, 1500, width-width/4, 6000); stateY = map(VarZ, 0, 1500, width-width/4, 6000); } else if (VarZ>1500) { stateX = map(VarZ, 1500, 3500, 6000, width-width/4); stateY = map(VarZ, 1500, 3500, 6000, width-width/4); }

And the following code draws a line between stars depending on their distance, using VarZ as a reference to how close the stars must be:

if (line) { for (int i = 0; i < starArray.size (); i++) { for (int j = 0; j < starArray.size (); j++) { if (VarZ < 1500) { starDistA=0; starDistB=40; mapVal1=0; mapVal2=1500; } else if (VarZ > 1500) { starDistA=40; starDistB=0; mapVal1=1500; mapVal2=3500; } if (dist(starArray.get(i).x, starArray.get(i).y, starArray.get(j).x, starArray.get(j).y) < map(VarZ, mapVal1, mapVal2, starDistA, starDistB)) { starArray.remove(map(VarZ, 0, 1500, 0, 1000)); stroke(random(150, 200), 50); line( starArray.get(i).x, starArray.get(i).y, starArray.get(j).x, starArray.get(j).y); } } } }

During this project we didn’t cover all the aspects or possibilities of this sensor. Rather we focused on the specific features needed to reach our goal. However, I think from here many more opportunities for further exploration will arise.

One Comment

  1. Kyung-nyer Ku

    Hello.

    My name is Kyung-nyer Ku. I am an MS student and currently working for my thesis project.

    A part of my project is using a structured-light 3D scanner, e.g. Kinect, and I am trying to explain the mechanism that Kinect obtains the 3D structure of the object in the introduction part of my thesis.

    I posted this message to ask you if I can use your picture in this post (a picture a person is standing in the middle of the IR light projection) in my thesis.

    It is a bit difficult to explain the light projection pattern of Kinect using only words. I think if I can put your photograph, it will be very helpful to give a clear idea about the light pattern of Kinect to people who are not familiar with 3D imaging.

    Please let me know if it is OK to use your photograph.

    I am looking forward to your reply.

    Sincerely,
    Kyung-nyer Ku

    MS student
    Colorado State University
    USA

Leave a Reply