Research on Rate Scalable Video Coding

The transmission of video over heterogeneous networks for multimedia applications is an area of active research in the Video and Image Processing Laboratory (VIPER). Rate scalable video compression is appealing for applications where the bandwidth available cannot be guaranteed. Typical applications are Internet video, video telephony, wireless communication, and video databases.

We have developed a continuously rate scalable video coding technique known as Scalable Adaptive Motion Compensated Wavelet (SAMCoW) compression. SAMCoW uses embedded coding such that the data rate can be dynamically changed on a frame-by-frame basis, and does not require the use of separate layers for scalability. This work is under the direction of Prof. Edward Delp.

• Main Features of SAMCoW
• CEZW: Embedded Coding of Color Images
• Adaptive Motion Compensation
• SAMCoW+
• Publications

Main Features of SAMCoW

Meeting bandwidth requirements and maintaining acceptable image quality simultaneously is a challenge. Continuous rate scalable applications can prove valuable in scenarios where the channel is unable to provide a constant bandwidth to the application. Rather than terminating the session, a decoder can adjust the data rate to use the limited resources, yet produce video of acceptable quality. Such decoders are particularly attractive because of their flexibility.

The main features of SAMCoW are:

• A modified zerotree wavelet image compression scheme known as CEZW, used for coding intracoded and predictive error frames, that exploits the redundancy across color components
• Adaptive block-base motion compensation used in the spatial domain to reduce temporal redundancy, and to improve image quality at low data rates

These features are described in detail in the following sections.

CEZW: Embedded Coding of Color Images

CEZW uses a unique spatial orientation tree (SOT) in the YUV color space. It exploits the interdependence between color components to achieve a higher degree of compression. CEZW uses the concept that at spatial locations where chrominance components have large transitions, the luminance component also has large transitions. Therefore, each node in the SOT of the luminance component also has descendants in the chrominance components at the same spatial location. This is shown in the figure on the left.

More information is available on the CEZW page .

Adaptive Motion Compensation

In a scalable codec, the decoded frames have different distortions at different data rates, making it impossible for the encoder to generate the exact reference frames as in the decoder for all the possible data rates. One solution is to have the encoder lock on to a fixed data rate (usually the highest data rate) and let the decoder run freely. The codec will work exactly as the non-scalable codec when decoding at the highest data rate. However, when decoding at a low data rate, the quality of the decoded reference frames will deviate from that at the encoder. Hence, both the motion prediction and the decoding of the predicted error frames contribute to the increase in distortion of the decoded video sequence. This distortion also propagates from one frame to the next within a group of pictures (GOP). If the size of a GOP is large, the increase in distortion can be unacceptable.

SAMCoW uses a technique known as adaptive motion compensation, in which a feedback loop is added in the decoder, as shown in the figure on the right. The decoded reference frames at both the encoder and the decoder are then locked to the same data rate. This makes the reference frames at the encoder and the decoder identical, hence eliminating error propagation.

SAMCoW+

The performance of SAMCoW at data rates below 64 kbps is perceptibly lower than H.263+ in sequences with high motion. We have developed several techniques to improve the performance of SAMCoW at low data rates. This extension to SAMCoW is known as SAMCoW+.

In SAMCoW+, CEZW is used for coding I frames. A modified CEZW algorithm is used for predictive error frames (PEF), as shown in the figure on the left. The PEF is preprocessed by using feature emphasis techniques and the elimination of information that is not visually significant. The modified CEZW algorithm uses wavelet shrinkage to selectively encode spatial orientation trees.

In addition, advanced coding options, such as unrestricted motion vectors, half-pixel accuracy, and B frames, are used in SAMCoW+.

More information is available on the SAMCoW+ page.

Publications

The complete list of recent publications in Image and Video Coding in the Video and Image Processing Laboratory (VIPER).