[Google]Gesturing is a natural means of communication for humans. Hand gestures, in particular, are often used to express ideas, to refer to objects, to attract attention or to signal turn taking. To recreate this communication channel over distance, real-time groupware systems usually display telepointers that participants can move over the shared view. However, standard telepointers usually lack semantic information: they have a fixed size and always point in the same direction. At best, the user can select among a predefined set of shapes and/or colors. These limitations hinder gesture communication between participants. They make it hard to draw attention, to designate several objects at the same time or to express an idea.
A mouse cursor is a poor substitute for the hand for gesture communication. In 1999, Michel Beaudouin-Lafon suggested to me that the image of the hand itself might do a better telepointer. We set up a camera above a desk covered by a large blue sheet of paper and recorded several sequences showing some hand gestures over this solid-color background. Ghislain Nouvel developed a chroma-keying filter to extract the image of the hands from these sequences and I created a few prototypes to display these chroma-keyed images over other image streams or running applications.
This project was developped with the videoSpace toolkit and was sponsored by France Télécom R&D. For more information on the chroma-keying algorithm or the prototypes we developed, see [1, 2, 3]. A few pictures and screenshots and a short video (14 Mb) are also available.
[Google] [Google] [Google]