SOMBRERO
EMBODYING SOCIO-COMMUNICATIVE BEHAVIORS
IN HUMANOID ROBOTS

Research topics

WP
The project is organized along six main tasks:
  1. Management and exploitation of the project.
  2. Interaction scenarios & psychosociological profiles: We will conduct a first exploratory study to collect the performances, the impressions and general attitudes of the people in front of robots coaches/therapists. A scenario will be built from points identified during a field study and according the scientific, experimental and technical constraints. We will compare subjective and objective characteristics of interactive tests performed under four experimental conditions: (a) human coaches in presence; (b) robots teleoperated by human coaches; (c) faked autonomous robots teleoperated by human coaches, i.e. wizard of Oz experiments (wOz) and (d) fully autonomous robots
  3. Immersive teleoperation (Beaming): We will study the accommodation patterns of several pilots driving the two robots. The performances of the mature control will be tested, in particular with tasks involving multi-joint (eye/head/hand/speech) coordination for pointing. Pilots will participate in tasks of robot-mediated human-pilot interactions. The focus will be on the importance of the socio-communicative behavior during finalized tasks involving objects of interest of the human-robot joint environment, i.e. test material displayed on a graphic tablet
  4. Active perception & annotation of socio-communicative intentions: The raw multimodal data (audiovisual signals, motion capture) monitored during beaming experiments will be automatically processed to feed multimodal scores with verbal and co-verbal cues such as identifying the robot’s eye fixations and attributing them to audiovisual regions of interest, labeling speech or identifying gestures performed by the user. We will develop incremental techniques able to process perceptual data and output actions as the interaction progresses.
  5. Human-aware decision making & socio-communicative behaviors: we will develop user-adapted behavioral models that best estimate the appropriate actions given the perceived user’s actions and the current estimation of the mutual intention of the interacting agents. We will consider the intricacy of two basic interaction structures: task-oriented dialogue and socio-communicative behaviors. The originality of our approach relies on its capacity to learn from the demonstration of socio-communicative behaviours based on the active perception (see Task 4) and the annotations (i.e., sequence of states and action labels) of tele-operated interactions (see Task 3).
  6. Autonomous robotics: we will implement the results of the previous tasks into the two robots: iCub2 and Roméo. These results are not only new features or improvements of existing features, but also the mechanisms, identified during the learning stages, implementing the connection between the perception of the context and the behavior of the robot. This task is made up of three sub-tasks: the definition of the integration architecture, the integration of innovative developments made by the partners, the development of the application that makes the robot autonomous to implement the scenarios