Robotic perception and manipulation of granular materials

Abstract

In various domains such as agriculture, mining, archaeology, and civil engineering, granular materials (GMs) are commonly encountered, such as sand, beans, soil, and gravel. However, GMs as a distinct class of materials consisting of discrete, solid particles exhibit unique mechanical properties. Unlike fluids or solids, GMs possess a combination of solid-like behavior, where individual particles maintain their shape, and fluid-like behavior, where the material can flow and deform under stress. As such, the complex physical properties of GMs pose significant challenges for robots in perceiving and manipulating them, in contrast to the rigid-body perception and manipulation encountered in traditional industrial scenarios. This thesis builds upon the progress made in physics regarding granules and explores solutions for robots to perceive and manipulate granular media using robotics technology and artificial intelligence algorithms. The first part of the thesis, comprising Chapters 2 and 3, focuses on the application of robots in locating objects beneath granules. Leveraging the unique force information derived from granular jamming, a proximity sensing system is developed to enable safe and automated object localization and distribution estimation within the granular medium. The second part of the thesis, comprising Chapters 4 and 5, deepens the understanding of granules through the application of robotics technology. By leveraging multimodal data, models are trained to recognize and estimate granular material categories and properties. The generalization ability of these models is demonstrated, including their zero-shot capability in natural environments. The findings of this research offer potential applications for robots in granular media.