Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces

Estimating depth from images nowadays yields outstanding results, both interms of in-domain accuracy and generalization. However, we identify two mainchallenges that remain open in this field: dealing with non-Lambertianmaterials and effectively processing high-resolution images. Purposely, wepropose a novel dataset that includes accurate and dense ground-truth labels athigh resolution, featuring scenes containing several specular and transparentsurfaces. Our acquisition pipeline leverages a novel deep space-time stereoframework, enabling easy and accurate labeling with sub-pixel precision. Thedataset is composed of 606 samples collected in 85 different scenes, eachsample includes both a high-resolution pair (12 Mpx) as well as an unbalancedstereo pair (Left: 12 Mpx, Right: 1.1 Mpx), typical of modern mobile devicesthat mount sensors with different resolutions. Additionally, we providemanually annotated material segmentation masks and 15K unlabeled samples. Thedataset is composed of a train set and two test sets, the latter devoted to theevaluation of stereo and monocular depth estimation networks. Our experimentshighlight the open challenges and future research directions in this field.