Supervisor: Nicolas Roussel, senior researcher at INRIA Lille
Keywords: Human-computer interaction (HCI), direct manipulation hybrids, mixed-initiative interaction, tactile and gestural interaction
Direct manipulation refers to the interaction style characterised by a continuous representation of the objects of interest, physical actions instead of a complex syntax, and incremental and reversible operations whose impact on the objects is immediately visible [1]. Predominant in graphical user interfaces, this interaction style played an important part in making them faster, easier, less error prone and more stimulating over the years. Yet even today, certain tasks remain difficult to do by manipulation.
A problem with physical actions, for example, is they can take some time. Further complicating the issue is the fact that for many computing activities, we redo more than we do. As the number of objects and actions increases the direct manipulation approach can become unwieldy: it makes it difficult to refer to parts of previous interactions, to schedule actions to take place in the future, to perform repetitive actions, or to do several things at the same time [2]. Direct manipulation interfaces are limited in that they are essentially equivalent to programming languages that only support sequencing, i.e. they do not have anything even as simple as a test-and-branch, much less more structured control constructs [3].
Different approaches have been proposed that aim at facilitating human-computer interactions by analyzing what the user is doing and automatically providing him/her with appropriate aids [4]. But the diversity and complexity of our computing activities make their artificial perception and understanding very difficult, which often leads to wrong or unpredictable system behaviours. Nothing is more frustrating than an "intelligent system" that behaves in undesired ways, especially in the unusual contexts causing these undesired behaviours which are likely to be stressful.
Mixed-initiative interaction can be seen as the creative integration of direct manipulation and automated services. It refers to a flexible interaction strategy at the boundary of Artificial Intelligence and Human-Computer Interaction in which the human and the computer contribute what it is best suited at the most appropriate time [5].
The UIMarks system that we recently proposed lets users specify on-screen targets and associated actions that can range from basic pointing facilitation to complex sequences possibly involving user interaction [6]. This system constitutes an interesting progress in the long standing effort to provide user programmable automation over direct manipulation. The goal of the proposed work is to further explore hybrid interaction styles by investigating the combined use of direct manipulation, programmed actions and automated ones following a mixed-initiative approach.
MINT, a joint research team between INRIA Lille and the LIFL (UMR CNRS 8022) and L2EP laboratories of Lille 1 University focused on tactile and gestural interaction.
The successful PhD candidate will work on:
To these ends, (s)he will more particularly investigate the use of tactile and gestural interactions and the ways in which users can control the initiative cursor, i.e. the level of initiative they are willing to transfer to an automated system.
Participatory design workshops and possibly longitudinal studies will be used to determine users' tasks, problems and needs. Standard prototyping tools and techniques from the HCI community will be used to generate new ideas, get feedback from users and make informed choices between alternative solutions. Controlled studies based on experimental and quasi-experimental designs will be used to evaluate the chosen solutions.
Ideal candidates will have a Master in Computer Science, proficient English language skills (written and spoken) and a background in Human-Computer Interaction, Computer Graphics or Virtual Reality as documented by a Master thesis (or equivalent) or by successfully completed projects in the area.