Towards individualized affective human-machine interaction

Robots and other autonomous systems interacting with humans should customize their behaviour to their human partner's preferences. We propose a method for learning and generating robot movement customized to individual preferences. Within a reinforcement learning framework, we generate rewards based on facial expressions observed during the robot's motion. Robot motions are parametrized; the rewards are used to modify these motion parameters using Q learning. The proposed approach is evaluated in a user study, using an interactive kinetic sculpture. The system interacts with participants and evolves its motion based on the rewards estimated from the participants' facial expressions. Our results show that, for a subset of participants, the system was able to successfully generate actions that resulted in higher than random rewards. The ability to successfully generate high-reward actions depends on: being able to recognize positive affect from the face, being able to generate actions that are pleasing to the participant, and being able to learn the mapping from rewards to actions.