Data Efficient and Geometric Optimal Transport for Robust Machine Learning

Optimal transport (OT) based distributional robustness is a promising framework for robust machine
learning and laying foundation for novel regularization techniques. However, the existing OT-based
distributional robustness has some severe limitations. First, it is not computationally tractable due to the min-max form w.r.t. a regularization parameter. Second, it is not sufficiently rich to represent the local (e.g., finding the most challenging local examples) and global (e.g., data and label shifts) regularization, hence circumventing the applications to real-world tasks including domain adaptation, domain generalization, semi-supervised learning, and adversarial/trustworthy machine learning, which always require formulating local/global regularization terms. Third, it cannot exploit the geometry structures carried in data (e.g., images tend to lie in low-dimensional intrinsic manifolds embedded in a data space). Fourth, it is impossible to capture the model sharpness which is crucial to improve the generalization ability of models. Targeting these severe limitations and drawbacks, we propose novel OT-based distributional robustness frameworks that are computationally tractable, sufficiently enormous to capture local/global regularization terms, can exploit and harvest geometry structures carried in data, and can be globally sharpness-aware. The proposed OT-based distributional frameworks open doors and enable the applications of distributional robustness to real-world applications such as domain adaptation, domain generalization, semi-supervised learning, and adversarial/trustworthy machine learning.