Towards Deep Reference Frame in Versatile Video Coding NNVC

In this paper, we propose a deep reference frame generation method that aims to enhance bi-direction inter prediction under random access configuration in the latest video coding standard, Versatile Video Coding. Specifically, a pair of neighboring reconstructed frames are selected from decoded picture buffer and put into an optical-flow-based interpolation network to synthesize a new frame, similar to the current to-be-coded frame. Subsequently, this synthesized frame is incorporated into two-sided picture reference lists as additional reference frames. The proposed method is employed in both the encoding and decoding processes to eliminate bitstream signaling for supplementary information. The Small Ad-hoc Deep-Learning Library is utilized for implementing the proposed method. Experimental results demonstrate 3.67%/7.34%/6.51% coding efficiency improvements for Y/U/V components under the random access configuration when compared to the Versatile Video Coding NNVC reference software VTM-11_NNVC-5.0.