Shape-invariant indirect hardness estimation for a soft vacuum-actuated gripper with an onboard depth camera

Soft grippers have gained a lot of interest in the last decade. In addition to firmly grasping an object, the estimation of its hardness is also an important aspect in various soft robotic applications. This study proposes a shape-invariant indirect hardness estimation approach for a soft vacuum-actuated gripper with an embedded depth camera. The technique proposed herein would eliminate the need for invasive sensors, which may damage certain objects. The project focuses on a simultaneous grasping and sensing system for deformable objects, without visible markers on the gripper's membrane. The deformation of membrane, containing valuable information on the object's properties, is captured by a depth camera inside the gripper. A convolutional neural network-based hardness prediction model is created with a mean absolute percentage error (MAPE) of 0.37%, in the case of trained shapes and trained hardnesses. For untrained hardnesses, the error is observed to be 4.54%. Through comparison with conventional grayscale images, the experiments also showed that images with depth information are more preferable for hardness estimation.