Kinect for the Blind

Our project uses Kinect for Xbox 360 sensor, which is capable of viewing depth directly and not by reconstructing it from stereo images.

Here is an example how Kinect sees outer world. Depth scene on the right corresponds to the image on the left. Note that the image is not used to produce depth map as Kinect 'sees' distances directly by using infrared flash and receiver that measures per-pixel light delay.

Using CL NUI Platform driver, our application gets depth map from the sensor, makes some heuristic filtering procedures and scales it down to the 8x4 depth matrix for presenting it on the tactile device.

Belly-mounted tactile matrix is controlled by ATmega32 board. Tactile data is transmitted via USB-to-UART FTDI interface so that virtual COM port is available when tatcile device is connected via USB cable.

Each tactile pixel is disassembled electromagnetic relay much like this with a curved paper-clip attached. Voltage is applied periodically so that pixel prods person several times a second. The more frequency is, the closer is the object under that pixel. Depth perception made that way is not very accurate, but even non-trained person could easily distinguish 'very close' from 'about one meter' from 'about three metres'.

Also, as tactile data is sent to the device at a rather high speed (baudrate of 19200 bits per second corresponds to about 480 tactile frames per second in our format) and because pixel pins are quite massive, it's also possible to slightly control how powerful are the prods by varying amount of data frames each pixel is 'on' within a prod period.

Optionally pixel could be softened by attaching pieces of a napkin.

Finally we won 2-nd place on Russian Finals of Imagine Cup 2011.
The 'Tactile Viewers' team (in the middle), left to right: Mila Shilova, Sergey Svita, Vasily Tarasov, Sergey Sahno.