Learning Debiased Representations via Conditional Attribute Interpolation

Abstract

An image is usually described by more than one attribute like “shape” and “color”. When a dataset is biased, i.e., most samples have attributes spuriously correlated with the target label, a Deep Neural Network (DNN) is prone to make predictions by the “unintended” attribute, especially if it is easier to learn. To improve the generalization ability when training on such a biased dataset, we propose a X2-model to learn debiased representations. First, we design a x-shape pattern to match the training dynamics of a DNN and find Intermediate Attribute Samples (IASs) — samples near the attribute decision boundaries, which indicate how the value of an attribute changes from one extreme to another. Then we rectify the representation with a X-structured metric learning objective. Conditional interpolation among IASs eliminates the negative effect of periph-eral attributes and facilitates retaining the intra-class compactness. Experiments show that X2-modellearns debiased representation effectively and achieves remarkable improvements on various datasets. Code is available at: https://github.com/ZhangYikaii/chi-square