Cross-domain transfer learning and state inference for soft robots via a semi-supervised sequential variational bayes framework

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)


Recently, data-driven models such as deep neural networks have shown to be promising tools for modelling and state inference in soft robots. However, voluminous amounts of data are necessary for deep models to perform effectively, which requires exhaustive and quality data collection, particularly of state labels. Consequently, obtaining labelled state data for soft robotic systems is challenged for various reasons, including difficulty in the sensorization of soft robots and the inconvenience of collecting data in unstructured environments. To address this challenge, in this paper, we propose a semi-supervised sequential variational Bayes (DSVB) framework for transfer learning and state inference in soft robots with missing state labels on certain robot configurations. Considering that soft robots may exhibit distinct dynamics under different robot configurations, a feature space transfer strategy is also incorporated to promote the adaptation of latent features across multiple configurations. Unlike existing transfer learning approaches, our proposed DSVB employs a recurrent neural network to model the nonlinear dynamics and temporal coherence in soft robot data. The proposed framework is validated on multiple setup configurations of a pneumatic-based soft robot finger. Experimental results on four transfer scenarios demonstrate that DSVB performs effective transfer learning and accurate state inference amidst missing state labels.

Original languageEnglish
Title of host publicationConference Proceedings - ICRA 2023
EditorsSeth Hutchinson
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9798350323658
ISBN (Print)9798350323665
Publication statusPublished - 2023
EventIEEE International Conference on Robotics and Automation 2023 - London, United Kingdom
Duration: 29 May 20232 Jun 2023 (Proceedings) (Website)


ConferenceIEEE International Conference on Robotics and Automation 2023
Abbreviated titleICRA 2023
Country/TerritoryUnited Kingdom
Internet address

Cite this