Re-weighting and Hierarchical Pre-training Boost 3D Medical Self-Supervised Representation.

The shortage of annotations and class imbalance in three-dimensional (3d) medical volume processing tasks downgrade the performance of supervised deep learning algorithms. We solve this problem via self-supervised learning and the hierarchical pre-training paradigm. This paper aims to improve the discriminability of self-supervised representation extracted by a 3d neural network from class-imbalanced medical volumes and to explore the potential of using deep learning features as complementary knowledge to radiomics features. We propose a self-supervised representation learning framework to extract deep-features from volume patches. Specifically, 1) we design a hyperparameter-free sample re-weighting module, which can embed into existing self-supervised contrastive learning architecture, and 2) we introduce the hierarchical pre-training paradigm into volume modality to improve the data utilization efficiency on limited samples. We conduct detailed experiments on public datasets of two modalities, computerized tomography (CT) and magnetic resonance imaging (MRI), to demonstrate the advantages of our architecture over the state-of-the-art counterpart in terms of feature discriminability.