Understanding Data Augmentation under Label Noise

Abstract

Data augmentation (DA) has become a fundamental component of modern machine learning systems and is widely regarded as an effective strategy for improving robustness and generalization. Despite its empirical success, the theoretical impact of DA under label noise remains poorly understood. In this work, we first theoretically explain when DA is beneficial and when it becomes detrimental, and provide sufficient conditions for distinguishing between these two regimes under both the i.i.d. learning setting and the more challenging domain generalization scenario. Then, we reveal a counterintuitive phenomenon: DA essentially acts as an uncertainty amplifier in the learning process, decreasing model stability and generalization performance. This finding is supported by the proposed learning and generalization bounds. Our analyses do not depend on specific DA techniques and require only mild assumptions, indicating that the results are applicable to a broad range of machine learning scenarios. Namely, we provide a unified theoretical foundation for understanding, analyzing, and designing DA methods in the presence of label noise.