Modeling nonconvex workspace constraints from diverse demonstration sets for Constrained Manipulator Visual Servoing

Ambrose Chan, Elizabeth A. Croft, James J. Little

Research output: Chapter in Book/Report/Conference proceedingConference PaperOther

5 Citations (Scopus)


This paper presents a novel framework for solving the Constrained Manipulator Visual Servoing (CMVS) problem. Classical eye-in-hand visual servoing relies on a reference image to capture the end-effector positioning task, but non-convex workspace constraints (such as whole-arm collision and camera occlusion constraints) are not represented. An explicit CAD model of the workspace is typically required for collision avoidance and visibility planning algorithms. In our novel CMVS framework, during the reference image capture process, we leverage the user's kinesthetic and visual capabilities to obtain a set of qualitatively-diverse demonstrations that provide information about the robot's work environment. We investigate methods for identifying the topology of the feasible regions represented directly in the control space of the robot (i.e., image-space and joint-space). We use a combination of stochastic modeling and graphical methods to describe the feasible space, capturing both the inter-group and intra-group variations. Specifically, our method uses the inter-groups variations to build a map that describes the global connectivity of the space, while exploiting the intra-group variations to automatically derive the appropriate gains in the control law. For a given target object, we apply online Gaussian Mixture Regression to the relevant feasible space regions to provide an idealized trajectory for tracking in image-space and in joint-space. We illustrate the key advantages of our approach through a set of visual servoing experiments on a Barrett WAM 7-DOF manipulator with a Sony XC-HR70 camera.

Original languageEnglish
Title of host publication2013 IEEE International Conference on Robotics and Automation, ICRA 2013
Number of pages7
Publication statusPublished - 14 Nov 2013
Externally publishedYes
EventIEEE International Conference on Robotics and Automation 2013 - Karlsruhe, Germany
Duration: 6 May 201310 May 2013 (Proceedings)


ConferenceIEEE International Conference on Robotics and Automation 2013
Abbreviated titleICRA 2013
Internet address

Cite this