An error-resilient video coding framework with soft reset and end-to-end distortion optimization
Temporal prediction plays a crucial role in most video coding applications. However, due to error propagation via the prediction loop, it also increases the vulnerability to channel loss. The standard counter measure to mitigate error propagation is the ‘intra refresh’ mode, which in effect resets temporal prediction to block error propagation, but at a significant rate overhead. This on/off switch for temporal prediction is overly crude to optimize the compression-resilience tradeoff. In this paper, we propose a novel framework that significantly expands the options available to counter error propagation by introducing optimally controlled soft resets, wherein intra and inter predictions are combined with adjustable weights to control the dependency on previous frames while accounting for the overall rate and distortion. Since the optimal control of such soft resets can only be achieved if the encoder can effectively estimate its impact on the end-to-end distortion (EED), we propose to extend the well known recursive optimal per-pixel estimation (ROPE) approach to accurately account for the soft reset mode, then optimize encoder mode decisions to minimize the estimated EED for the given rate. Experimental results show that the proposed framework achieves significant performance gains for video streaming over unreliable networks.