We used our stereo vision algorithm for three applications.  The first one is for human gesture recognition for human computer interface.  The second one is to help with visually impaired to detect and avoid obstacles.  The third application is for finger tracking that provides touch-free interface with hand-held devices. 

    Many image and signal processing techniques have been applied to medical and health care applications in recent years. We developed a robust signal processing approach that can be used to solve the correspondence problem for an embedded stereo vision sensor to provide real-time visual guidance to the visually impaired. This approach is based on our new one-dimensional (1D) spline-based genetic algorithm to match signals. The algorithm processes image data lines as 1D signals to generate a dense disparity map, from which 3D information can be extracted. With recent advances in electronics technology, this 1D signal matching technique can be implemented and executed in parallel in hardware such as field-programmable gate arrays (FPGAs) to provide real-time feedback about the environment to the user. In order to complement (not replace) traditional aids for the visually impaired such as canes and Seeing Eyes dogs, vision systems that provide guidance to the visually impaired must be affordable, easy to use, compact, and free from attributes that are awkward or embarrassing to the user. “Seeing Eye Glasses,” an embedded stereo vision system utilizing our new algorithm, meets all these requirements.

    Hundreds of millions of people use hand-held devices frequently and control them by touching the screen with their fingers. If this method of operation is being used by people who are driving, the probability of deaths and accidents occurring substantially increases. With a non-contact control interface, people do not need to touch the screen. As a result, people will not need to pay as much attention to their phones and thus drive more safely than they would otherwise. This interface can be achieved with real-time stereo vision. A novel Intensity Profile Shape-Matching Algorithm is able to obtain 3-D information from a pair of stereo images in real time. While this algorithm does have a trade-off between accuracy and processing speed, the result of this algorithm proves the accuracy is sufficient for the practical use of recognizing human poses and finger movement tracking.  By choosing an interval of disparity, an object at a certain distance range can be segmented. In other words, we detect the object by its distance to the cameras. The advantage of this profile shape-matching algorithm is that detection of correspondences relies on the shape of profile and not on intensity values, which are subjected to lighting variations.  Based on the resulting 3-D information, the movement of fingers in space from a specific distance can be determined.  Finger location and movement can then be analyzed for non-contact control of hand-held devices.

 Project Sponsors:

David and Deborah Huber Scholarship

 Graduate Students:

Yung-Ping Chang, Beau Tippetts, Jonathan Anderson

1. D. Zhang, D.J. Lee, Y.P. Chang, "A new profile shape matching stereo vision algorithm for real-time gesture and motion estimation, International Journal of Advanced Robotic Systems, vol. 11, February 2014.
   
2. Y.P. Chang, D.J. Lee, J.A. Moore, A. Desai, and B.J. Tippetts, “Finger Tracking for Hand-held Device Interface Using Profile-matching Stereo Vision,” SPIE Electronic Imaging, Intelligent Robots and Computer Vision XXX: Algorithms and Techniques, San Francisco, CA, USA, February 3-7, 2013.
3. B.J. Tippetts, D. J. Lee, J.K Archibald, and K.D. Lillywhite, “Dense Disparity Real-time Stereo Vision Algorithm for Resource Limited Systems,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 21/10, p. 1547-1555, October 2011.
4. D.J. Lee, J.D. Anderson, and J.K. Archibald, “Hardware Implementation of Spline-based Genetic Algorithm for Embedded Stereo Vision Sensor Providing Real-time Visual Guidance to the Visually Impaired”, special issue on “Signal Processing for Applications in Healthcare Systems (AHS)” of the EURASIP Journal on Advances in Signal Processing, vol. 2008, 10 pages, June 2008.
5. J.D. Anderson, D.J. Lee, and J.K. Archibald, “Embedded Stereo Vision System Providing Visual Guidance to the Visually Impaired,” The Third IEEE/NIH Life Science Systems and Application Workshop (LISSA), p. 229-232, Bethesda, MD, USA, November 8-9, 2007.