Collision-free visual servoing of an eye-in-hand manipulator via constraint-aware planning and control

Ambrose Chan, Simon Leonard, Elizabeth A. Croft, James J. Little

Research output: Chapter in Book/Report/Conference proceedingConference PaperOther

7 Citations (Scopus)


A constraint-aware eye-in-hand visual servoing control law is proposed. The control law is designed for a robot manipulator with an uncalibrated camera mounted on its end-effector. The method uses an image-based command to describe the desired end-effector position with respect to an object with an unknown position. When starting from an unknown position, the control law uses feedback from the camera to move the robot towards the reference image while satisfying a set of system constraints. The visual servoing control law is implemented via a nonlinear model predictive control framework to generate feasible and realistic robot trajectories that respect the robot's joint limits and velocity limits. The control law explicitly keeps the target object within the camera's field of view and avoids potential collisions with workspace obstacles. An appropriate representation of the robot's whole-arm collision constraints is extracted from well-known path planning methods, such as probabilistic road maps and dynamic collision checking algorithms. Experiments using an uncalibrated eyein-hand platform demonstrate the ability of the visual servoing control law to achieve closed-loop positioning via collision-free trajectories, even when the initial object location is uncertain.

Original languageEnglish
Title of host publicationProceedings of the 2011 American Control Conference, ACC 2011
Number of pages7
Publication statusPublished - 29 Sep 2011
Externally publishedYes
EventAmerican Control Conference 2011 - San Francisco Hilton on O'Farrell Street, San Francisco, United States of America
Duration: 29 Jun 20111 Jul 2011


ConferenceAmerican Control Conference 2011
Abbreviated titleACC 2011
CountryUnited States of America
CitySan Francisco
Internet address

Cite this