Cursor Control with the Kinect

Overview

The Kinect program can now control the cursor by following the right hand of the user and perform left clicks if the users hand if in a certain position.

Background

To interact with the program they user does not want to keep swapping between using the Kinect and mouse, rather just use one. So controlling the mouse via the Kinect seemed an appropriate solution.

Activity

To control the mouse the program had to detect the position of the hand and use this. By using much of the code that has already been implemented the position is simple to get. However the position had to be scaled to the screen size and desired reach.
By using the "ScaleTo" function build in to the CodingForFun library, it is possible to set the position of the hand join in relation to a screen size. Below is the code scaling the right hand to the screen resolution. The last two section .2f, .2f are how much the user must stretch to reach the furthest points.

 first.Joints[JointType.HandRight].ScaleTo(1680, 1050, .2f, .2f);

From this the X and Y can be determined and set the mouse cursor. By importing the User32.dll this allows access to the users controls such as the cursor position.
Now that the cursor could be controlled from the Kinect, the action of a mouse clicked needed to be implemented.
Using the code seen below if the left hand if higher up then the right hand then perform the a left click. The presented a problem as the mouse click kept clicking until the hand had moved. So I implemented a boolean that would say whether the hand position had been reset.

             if (first.Joints[JointType.HandRight].Position.Y < first.Joints[JointType.HandLeft].Position.Y && active == false)
            {
                mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
                Thread.Sleep(1);
                active = true;
            }
            else if (first.Joints[JointType.HandRight].Position.Y > first.Joints[JointType.HandLeft].Position.Y && active == true)
            {
                active = false;
            }

The Next Step

The next step is to design the interface that was be easily navigable.