OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation

The scarcity of ground-truth labels poses one major challenge in developingoptical flow estimation models that are both generalizable and robust. Whilecurrent methods rely on data augmentation, they have yet to fully exploit therich information available in labeled video sequences. We propose OCAI, amethod that supports robust frame interpolation by generating intermediatevideo frames alongside optical flows in between. Utilizing a forward warpingapproach, OCAI employs occlusion awareness to resolve ambiguities in pixelvalues and fills in missing values by leveraging the forward-backwardconsistency of optical flows. Additionally, we introduce a teacher-studentstyle semi-supervised learning method on top of the interpolated frames. Usinga pair of unlabeled frames and the teacher model's predicted optical flow, wegenerate interpolated frames and flows to train a student model. The teacher'sweights are maintained using Exponential Moving Averaging of the student. Ourevaluations demonstrate perceptually superior interpolation quality andenhanced optical flow accuracy on established benchmarks such as Sintel andKITTI.