ZoneOS home

Kinect for the Blind

Tactile matrix transmitting depth data from Kinect sensor (Vasily Tarasov is simulated blind)

Kinect for the Blind

Kinect for the Blind is Imagine Cup 2011 Software Design project of Tactile Viewers team, Moscow Institute of Physics and Technology: Vasily Tarasov, Mila Shilova, Sergey Svita, and Sergey Sahno.

It was designed to help visually impaired people (including completely blind) orient on streets. Particularly, with our device person can 'feel' direction, distance and overall dimensions of obstacles at a distances greater than the length of a white cane. Second, our device allows track obstacles on all directions simultaneously (left, right, up, down), while cane is tracking only one direction at a time.

How it Works

Our project uses Kinect for Xbox 360 sensor, which is capable of viewing depth directly and not by reconstructing it from stereo images.

Here is an example how Kinect sees outer world. Depth scene on the right corresponds to the image on the left. Note that the image is not used to produce depth map as Kinect 'sees' distances directly by using infrared flash and receiver that measures per-pixel light delay.

Using CL NUI Platform driver, our application gets depth map from the sensor, makes some heuristic filtering procedures and scales it down to the 8x4 depth matrix for presenting it on the tactile device.

Belly-mounted tactile matrix is controlled by ATmega32 board. Tactile data is transmitted via USB-to-UART FTDI interface so that virtual COM port is available when tatcile device is connected via USB cable.

Each tactile pixel is disassembled electromagnetic relay much like this with a curved paper-clip attached. Voltage is applied periodically so that pixel prods person several times a second. The more frequency is, the closer is the object under that pixel. Depth perception made that way is not very accurate, but even non-trained person could easily distinguish 'very close' from 'about one meter' from 'about three metres'.

Also, as tactile data is sent to the device at a rather high speed (baudrate of 19200 bits per second corresponds to about 480 tactile frames per second in our format) and because pixel pins are quite massive, it's also possible to slightly control how powerful are the prods by varying amount of data frames each pixel is 'on' within a prod period.

Optionally pixel could be softened by attaching pieces of a napkin.

Finally we won 2-nd place on Russian Finals of Imagine Cup 2011.
The 'Tactile Viewers' team (in the middle), left to right: Mila Shilova, Sergey Svita, Vasily Tarasov, Sergey Sahno.

Related Projects

NAVI – Navigational Aids for the Visually Impaired
Vision Substitution
AuxDeco - Forehead Sensory Recognition
The vOICe - See with your Ears

Home | Projects | Videos

Copyright © 2011 Tactile Viewers.