Discover and Mitigate Multiple Biased Subgroups in Image Classifiers

Machine learning models can perform well on in-distribution data but oftenfail on biased subgroups that are underrepresented in the training data,hindering the robustness of models for reliable applications. Such subgroupsare typically unknown due to the absence of subgroup labels. Discovering biasedsubgroups is the key to understanding models' failure modes and furtherimproving models' robustness. Most previous works of subgroup discovery make animplicit assumption that models only underperform on a single biased subgroup,which does not hold on in-the-wild data where multiple biased subgroups exist. In this work, we propose Decomposition, Interpretation, and Mitigation (DIM),a novel method to address a more challenging but also more practical problem ofdiscovering multiple biased subgroups in image classifiers. Our approachdecomposes the image features into multiple components that represent multiplesubgroups. This decomposition is achieved via a bilinear dimension reductionmethod, Partial Least Square (PLS), guided by useful supervision from the imageclassifier. We further interpret the semantic meaning of each subgroupcomponent by generating natural language descriptions using vision-languagefoundation models. Finally, DIM mitigates multiple biased subgroupssimultaneously via two strategies, including the data- and model-centricstrategies. Extensive experiments on CIFAR-100 and Breeds datasets demonstratethe effectiveness of DIM in discovering and mitigating multiple biasedsubgroups. Furthermore, DIM uncovers the failure modes of the classifier onHard ImageNet, showcasing its broader applicability to understanding model biasin image classifiers. The code is available athttps://github.com/ZhangAIPI/DIM.