Software Design
Software Overview:
The software can be divided into 4 major parts, the part that went on our linux box to interact with the Kinect,
the part that went on the CMUCam3 to transmit information about what it sees to the car, and the parts that went on
each of the two Actel SmartFusion boards we used to receive, interpret, and transmit this data.
On the Linux Box
We used a set of open source Kinect drivers called OpenNI to get the computer to read
in data from the Kinect, and an open source gesture recognition library called NITE to track the users skeleton
and detect specific gestures within that data.
- Once we had all this working we modified and expanded one of the examples that displayed the results of the
skeletal tracking onto the screen so that it would send some of this information through the computer's serial
port to one of our Actel SmartFusion boards. The code itself is in C++ and uses a callback event based loop
architecture so that when specific actions or changes occur, the corresponding functions are called.
On the CMUCam3:
The software we wrote for the CMUCam is fairly simple. Each time the camera captures a frame it checks
to see where the color red is detected, then it sends the centroid
of where it saw that color through its serial port to the Actel board. It encodes this into a single byte with the most significant 4 bits being the X
position, and the least significant 4 being the Y position.
The SmartFusion Connected to the Linux Box:
For this board we implemented a simple state structure in which it sends information over the radio to the
board on the car, and then waits to receive information back before it sends again. This was needed in order
to make sure that both boards do not cause too much interference with each other. Because of the fact that some radio messages
are lost, we also use the MSS timer to timeout if it sits in receive longer than 100ms on this board, at which
point it automatically swaps to transmit mode in order to make sure we cannot get stuck with both cars waiting
for a message.
- In transmit mode, this board reads from the UART that the linux box sends the Kinect information to and transmits
the message leaving all interpretation of the data to the board on the car.
- In receive mode this board gets the current centroid for where the CMUCam sees the color red in a radio message
from the board on the car. It must first check the special case where the message is just a 0, which means that there
is no red in the image, in which case it sends a command to the LCD screen to clear itself. Otherwise, it takes the
byte it received and splits it into the X and Y positions transmitted by the camera. It then converts those
coordinates to positions on the LCD screen and sends out commands to the screen to draw a box around that point.
The SmartFusion on the Car:
- This board is running the software that makes the car drive. There are a couple state machines being implemented simultaneously.
The first state machine is to differentiate between transmit and receive modes. The functionality of these two states mirrors those on
the Kinect-side board. When in transmit, the code reads from the UART that is being used to receive camera data and sends this data back
to the other board using a radio send function. When in receive, the car waits to be sent a message from the Kinect-side board. This message
that it receives is then interpreted to determine whether the message is indicating new drive or camera controls, or to simply idle.
Whenever new control data is received, the appropriate I/O registers are updated by the software.
The other state machine being used by this board's software is to determine whether the car is free to be driven or if it is recovering
from a detected hazard. This state can only be changed by an interrupt that is generated by the hardware. When the car goes into recovery
mode, it uses a system timer to recover away from the hazard for one second after which it goes back to drive mode. The timer is also used
to implement a blinking reverse light.