In-hand gravitational pivoting using tactile sensing

Jason Toskov, Rhys Newbury, Mustafa Mukadam, Dana Kulić, Akansel Cosgun

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

3 Citations (Scopus)


We study gravitational pivoting, a constrained version of in-hand manipulation, where we aim to control the rotation of an object around the grip point of a parallel gripper. To achieve this, instead of controlling the gripper to avoid slip, we embrace slip to allow the object to rotate in-hand. We collect two real-world datasets, a static tracking dataset and a controller-in-the-loop dataset, both annotated with object angle and angular velocity labels. Both datasets contain force-based tactile information on ten different household objects. We train an LSTM model to predict the angular position and velocity of the held object from purely tactile data. We integrate this model with a controller that opens and closes the gripper allowing the object to rotate to desired relative angles. We conduct real-world experiments where the robot is tasked to achieve a relative target angle. We show that our approach outperforms a sliding-window based MLP in a zero-shot generalization setting with unseen objects. Furthermore, we show a 16.6% improvement in performance when the LSTM model is fine-tuned on a small set of data collected with both the LSTM model and the controller in-the-loop. Code and videos are available at

Original languageEnglish
Title of host publicationProceedings of The 6th Conference on Robot Learning
EditorsKaren Liu, Dana Kulic, Jeff Ichnowski
Place of PublicationLondon UK
PublisherProceedings of Machine Learning Research (PMLR)
Number of pages10
Publication statusPublished - 2023
EventConference on Robot Learning 2022 - Auckland, New Zealand
Duration: 14 Dec 202218 Dec 2022
Conference number: 6th (Website) (Proceedings)


ConferenceConference on Robot Learning 2022
Abbreviated titleCoRL 2022
Country/TerritoryNew Zealand
Internet address


  • In Hand Manipulation
  • Tactile Pose Sensing

Cite this